2024-03-04T15:23:21.561 INFO:root:teuthology version: 0.0.1.dev260+gb1dac55 2024-03-04T15:23:21.562 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_tonay@teuthology --description orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} --name tonay-2024-03-04_15:07:38-orch:cephadm-wip-asm0deuz-testing-2024-03-04-1211-distro-default-smithi --verbose --archive /home/teuthworker/archive/tonay-2024-03-04_15:07:38-orch:cephadm-wip-asm0deuz-testing-2024-03-04-1211-distro-default-smithi/7581221 -- /home/teuthworker/archive/tonay-2024-03-04_15:07:38-orch:cephadm-wip-asm0deuz-testing-2024-03-04-1211-distro-default-smithi/7581221/orig.config.yaml 2024-03-04T15:23:21.594 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T15:23:21.705 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/tonay-2024-03-04_15:07:38-orch:cephadm-wip-asm0deuz-testing-2024-03-04-1211-distro-default-smithi/7581221 branch: wip-asm0deuz-testing-2024-03-04-1211 description: orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} email: tonay@ibm.com first_in_suite: false job_id: '7581221' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false machine_type: smithi name: tonay-2024-03-04_15:07:38-orch:cephadm-wip-asm0deuz-testing-2024-03-04-1211-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-asm0deuz-testing-2024-03-04-1211 ceph: conf: client: debug ms: 1 global: mon election default strategy: 1 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: 9ca0b5de35ed230e84387abdd8e619ed8e4f3282 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: cephadm-package install: ceph: flavor: default sha1: 9ca0b5de35ed230e84387abdd8e619ed8e4f3282 extra_packages: - cephadm workunit: branch: wip-asm0deuz-testing-2024-03-04-1211 sha1: 9ca0b5de35ed230e84387abdd8e619ed8e4f3282 owner: scheduled_tonay@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 3905 sha1: 9ca0b5de35ed230e84387abdd8e619ed8e4f3282 sleep_before_teardown: 0 subset: 1/14 suite: orch:cephadm suite_branch: wip-asm0deuz-testing-2024-03-04-1211 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_9ca0b5de35ed230e84387abdd8e619ed8e4f3282/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 9ca0b5de35ed230e84387abdd8e619ed8e4f3282 targets: smithi117.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHf1ehnhjHc10KhtKu4jwsfchliAUjFZXiiJi8z1Q9fdLAxIiu8KDLw6bdOoDiteRYYzxWxxnkYpz5bCyVgptCs= smithi151.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGnhs4B90sAmKtOUf5PPUHN5Bh0v5nKFlRDlAbSotJEf7zPCbtBZT2URm7gLcEtfK8Trbx0k8X847cqw8GxI7sI= tasks: - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: b1dac5519c57c269ea98a22fb7729016a1d0e4d2 timestamp: 2024-03-04_15:07:38 tube: smithi user: tonay verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-03-04T15:23:21.705 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_9ca0b5de35ed230e84387abdd8e619ed8e4f3282/qa; will attempt to use it 2024-03-04T15:23:21.706 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_9ca0b5de35ed230e84387abdd8e619ed8e4f3282/qa/tasks 2024-03-04T15:23:21.706 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-03-04T15:23:21.708 INFO:teuthology.task.internal:Checking packages... 2024-03-04T15:23:21.745 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '9ca0b5de35ed230e84387abdd8e619ed8e4f3282' 2024-03-04T15:23:21.745 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-03-04T15:23:21.745 INFO:teuthology.packaging:ref: None 2024-03-04T15:23:21.746 INFO:teuthology.packaging:tag: None 2024-03-04T15:23:21.746 INFO:teuthology.packaging:branch: wip-asm0deuz-testing-2024-03-04-1211 2024-03-04T15:23:21.746 INFO:teuthology.packaging:sha1: 9ca0b5de35ed230e84387abdd8e619ed8e4f3282 2024-03-04T15:23:21.746 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-asm0deuz-testing-2024-03-04-1211 2024-03-04T15:23:21.915 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-1826-g9ca0b5de-1jammy 2024-03-04T15:23:21.916 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-03-04T15:23:21.925 INFO:teuthology.task.internal:no buildpackages task found 2024-03-04T15:23:21.925 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-03-04T15:23:21.931 INFO:teuthology.task.internal:Saving configuration 2024-03-04T15:23:21.943 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-03-04T15:23:21.949 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-04T15:23:21.983 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/tonay-2024-03-04_15:07:38-orch:cephadm-wip-asm0deuz-testing-2024-03-04-1211-distro-default-smithi/7581221', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 15:17:30.128623', 'locked_by': 'scheduled_tonay@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIGFsovQ9QQzKuK7fEaY/gWVwIFHvLHaOG9wPEdgAu6cI'} 2024-03-04T15:23:22.014 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/tonay-2024-03-04_15:07:38-orch:cephadm-wip-asm0deuz-testing-2024-03-04-1211-distro-default-smithi/7581221', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 15:17:30.130277', 'locked_by': 'scheduled_tonay@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFMYop7iGeiufg2uVzMJYSdoimv+cqJjsW1YKthFoaYe'} 2024-03-04T15:23:22.014 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-03-04T15:23:22.020 INFO:teuthology.task.internal:roles: ubuntu@smithi117.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-03-04T15:23:22.020 INFO:teuthology.task.internal:roles: ubuntu@smithi151.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-03-04T15:23:22.020 INFO:teuthology.run_tasks:Running task console_log... 2024-03-04T15:23:22.218 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f40409d43a0>, signals=[15]) 2024-03-04T15:23:22.219 INFO:teuthology.run_tasks:Running task internal.connect... 2024-03-04T15:23:22.225 INFO:teuthology.task.internal:Opening connections... 2024-03-04T15:23:22.225 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi117.front.sepia.ceph.com 2024-03-04T15:23:22.227 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T15:23:22.408 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi151.front.sepia.ceph.com 2024-03-04T15:23:22.409 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T15:23:22.568 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-03-04T15:23:22.576 DEBUG:teuthology.orchestra.run.smithi117:> uname -m 2024-03-04T15:23:22.586 INFO:teuthology.orchestra.run.smithi117.stdout:x86_64 2024-03-04T15:23:22.587 DEBUG:teuthology.orchestra.run.smithi117:> cat /etc/os-release 2024-03-04T15:23:22.633 INFO:teuthology.orchestra.run.smithi117.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-04T15:23:22.633 INFO:teuthology.orchestra.run.smithi117.stdout:NAME="Ubuntu" 2024-03-04T15:23:22.633 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_ID="22.04" 2024-03-04T15:23:22.633 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-04T15:23:22.633 INFO:teuthology.orchestra.run.smithi117.stdout:VERSION_CODENAME=jammy 2024-03-04T15:23:22.633 INFO:teuthology.orchestra.run.smithi117.stdout:ID=ubuntu 2024-03-04T15:23:22.633 INFO:teuthology.orchestra.run.smithi117.stdout:ID_LIKE=debian 2024-03-04T15:23:22.633 INFO:teuthology.orchestra.run.smithi117.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-04T15:23:22.633 INFO:teuthology.orchestra.run.smithi117.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-04T15:23:22.634 INFO:teuthology.orchestra.run.smithi117.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-04T15:23:22.634 INFO:teuthology.orchestra.run.smithi117.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-04T15:23:22.634 INFO:teuthology.orchestra.run.smithi117.stdout:UBUNTU_CODENAME=jammy 2024-03-04T15:23:22.634 INFO:teuthology.lock.ops:Updating smithi117.front.sepia.ceph.com on lock server 2024-03-04T15:23:22.666 DEBUG:teuthology.orchestra.run.smithi151:> uname -m 2024-03-04T15:23:22.678 INFO:teuthology.orchestra.run.smithi151.stdout:x86_64 2024-03-04T15:23:22.678 DEBUG:teuthology.orchestra.run.smithi151:> cat /etc/os-release 2024-03-04T15:23:22.723 INFO:teuthology.orchestra.run.smithi151.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2024-03-04T15:23:22.723 INFO:teuthology.orchestra.run.smithi151.stdout:NAME="Ubuntu" 2024-03-04T15:23:22.723 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_ID="22.04" 2024-03-04T15:23:22.723 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2024-03-04T15:23:22.723 INFO:teuthology.orchestra.run.smithi151.stdout:VERSION_CODENAME=jammy 2024-03-04T15:23:22.723 INFO:teuthology.orchestra.run.smithi151.stdout:ID=ubuntu 2024-03-04T15:23:22.724 INFO:teuthology.orchestra.run.smithi151.stdout:ID_LIKE=debian 2024-03-04T15:23:22.724 INFO:teuthology.orchestra.run.smithi151.stdout:HOME_URL="https://www.ubuntu.com/" 2024-03-04T15:23:22.724 INFO:teuthology.orchestra.run.smithi151.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2024-03-04T15:23:22.724 INFO:teuthology.orchestra.run.smithi151.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2024-03-04T15:23:22.724 INFO:teuthology.orchestra.run.smithi151.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2024-03-04T15:23:22.724 INFO:teuthology.orchestra.run.smithi151.stdout:UBUNTU_CODENAME=jammy 2024-03-04T15:23:22.724 INFO:teuthology.lock.ops:Updating smithi151.front.sepia.ceph.com on lock server 2024-03-04T15:23:22.757 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-03-04T15:23:22.769 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-03-04T15:23:22.783 INFO:teuthology.task.internal:Checking for old test directory... 2024-03-04T15:23:22.783 DEBUG:teuthology.orchestra.run.smithi117:> test '!' -e /home/ubuntu/cephtest 2024-03-04T15:23:22.791 DEBUG:teuthology.orchestra.run.smithi151:> test '!' -e /home/ubuntu/cephtest 2024-03-04T15:23:22.803 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-03-04T15:23:22.812 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-03-04T15:23:22.812 DEBUG:teuthology.orchestra.run.smithi117:> test -z $(ls -A /var/lib/ceph) 2024-03-04T15:23:22.838 DEBUG:teuthology.orchestra.run.smithi151:> test -z $(ls -A /var/lib/ceph) 2024-03-04T15:23:22.909 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-03-04T15:23:22.982 INFO:teuthology.run_tasks:Running task kernel... 2024-03-04T15:23:23.001 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2024-03-04T15:23:23.002 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2024-03-04T15:23:23.002 DEBUG:teuthology.orchestra.run.smithi117:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-04T15:23:23.004 DEBUG:teuthology.orchestra.run.smithi151:> test -f /run/.containerenv -o -f /.dockerenv 2024-03-04T15:23:23.010 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T15:23:23.011 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-03-04T15:23:23.014 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T15:23:23.016 DEBUG:teuthology.orchestra.run.smithi151:> uname -r 2024-03-04T15:23:23.057 INFO:teuthology.orchestra.run.smithi117.stdout:5.15.0-56-generic 2024-03-04T15:23:23.057 INFO:teuthology.task.kernel:Running kernel on smithi117: 5.15.0-56-generic 2024-03-04T15:23:23.057 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get clean 2024-03-04T15:23:23.065 INFO:teuthology.orchestra.run.smithi151.stdout:5.15.0-56-generic 2024-03-04T15:23:23.065 INFO:teuthology.task.kernel:Running kernel on smithi151: 5.15.0-56-generic 2024-03-04T15:23:23.065 DEBUG:teuthology.orchestra.run.smithi151:> sudo apt-get clean 2024-03-04T15:23:23.179 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get update 2024-03-04T15:23:23.184 DEBUG:teuthology.orchestra.run.smithi151:> sudo apt-get update 2024-03-04T15:23:23.321 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-04T15:23:23.323 INFO:teuthology.orchestra.run.smithi117.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-04T15:23:23.328 INFO:teuthology.orchestra.run.smithi117.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-04T15:23:23.415 INFO:teuthology.orchestra.run.smithi117.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-04T15:23:23.465 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-04T15:23:23.478 INFO:teuthology.orchestra.run.smithi151.stdout:Get:2 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2024-03-04T15:23:23.511 INFO:teuthology.orchestra.run.smithi151.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2024-03-04T15:23:23.568 INFO:teuthology.orchestra.run.smithi117.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [413 kB] 2024-03-04T15:23:23.642 INFO:teuthology.orchestra.run.smithi117.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [579 kB] 2024-03-04T15:23:23.650 INFO:teuthology.orchestra.run.smithi117.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,205 kB] 2024-03-04T15:23:23.700 INFO:teuthology.orchestra.run.smithi117.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [219 kB] 2024-03-04T15:23:23.700 INFO:teuthology.orchestra.run.smithi117.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-04T15:23:23.700 INFO:teuthology.orchestra.run.smithi117.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,476 kB] 2024-03-04T15:23:23.715 INFO:teuthology.orchestra.run.smithi117.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,422 kB] 2024-03-04T15:23:23.719 INFO:teuthology.orchestra.run.smithi117.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-04T15:23:23.719 INFO:teuthology.orchestra.run.smithi117.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [244 kB] 2024-03-04T15:23:23.729 INFO:teuthology.orchestra.run.smithi117.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-04T15:23:23.729 INFO:teuthology.orchestra.run.smithi117.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [596 kB] 2024-03-04T15:23:23.731 INFO:teuthology.orchestra.run.smithi117.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [846 kB] 2024-03-04T15:23:23.742 INFO:teuthology.orchestra.run.smithi117.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [161 kB] 2024-03-04T15:23:23.767 INFO:teuthology.orchestra.run.smithi117.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-04T15:23:23.767 INFO:teuthology.orchestra.run.smithi117.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [279 kB] 2024-03-04T15:23:23.767 INFO:teuthology.orchestra.run.smithi117.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-04T15:23:23.767 INFO:teuthology.orchestra.run.smithi117.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-04T15:23:23.769 INFO:teuthology.orchestra.run.smithi117.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-04T15:23:23.769 INFO:teuthology.orchestra.run.smithi117.stdout:Get:23 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-04T15:23:23.769 INFO:teuthology.orchestra.run.smithi117.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-04T15:23:23.773 INFO:teuthology.orchestra.run.smithi117.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,504 kB] 2024-03-04T15:23:23.793 INFO:teuthology.orchestra.run.smithi117.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-04T15:23:23.794 INFO:teuthology.orchestra.run.smithi117.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [247 kB] 2024-03-04T15:23:23.798 INFO:teuthology.orchestra.run.smithi117.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-04T15:23:23.798 INFO:teuthology.orchestra.run.smithi117.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,052 kB] 2024-03-04T15:23:23.811 INFO:teuthology.orchestra.run.smithi117.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [691 kB] 2024-03-04T15:23:23.818 INFO:teuthology.orchestra.run.smithi117.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [237 kB] 2024-03-04T15:23:23.822 INFO:teuthology.orchestra.run.smithi117.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-04T15:23:23.824 INFO:teuthology.orchestra.run.smithi117.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.1 kB] 2024-03-04T15:23:23.825 INFO:teuthology.orchestra.run.smithi117.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,160 B] 2024-03-04T15:23:23.842 INFO:teuthology.orchestra.run.smithi117.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.1 kB] 2024-03-04T15:23:23.843 INFO:teuthology.orchestra.run.smithi117.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-04T15:23:23.843 INFO:teuthology.orchestra.run.smithi117.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [41.7 kB] 2024-03-04T15:23:23.843 INFO:teuthology.orchestra.run.smithi117.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [33.9 kB] 2024-03-04T15:23:23.843 INFO:teuthology.orchestra.run.smithi117.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2024-03-04T15:23:23.845 INFO:teuthology.orchestra.run.smithi117.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-04T15:23:23.845 INFO:teuthology.orchestra.run.smithi117.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [24.3 kB] 2024-03-04T15:23:23.846 INFO:teuthology.orchestra.run.smithi117.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [13.4 kB] 2024-03-04T15:23:23.863 INFO:teuthology.orchestra.run.smithi151.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2024-03-04T15:23:23.870 INFO:teuthology.orchestra.run.smithi151.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [413 kB] 2024-03-04T15:23:23.870 INFO:teuthology.orchestra.run.smithi117.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-03-04T15:23:23.870 INFO:teuthology.orchestra.run.smithi117.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-04T15:23:23.996 INFO:teuthology.orchestra.run.smithi151.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,422 kB] 2024-03-04T15:23:24.161 INFO:teuthology.orchestra.run.smithi151.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [1,205 kB] 2024-03-04T15:23:24.293 INFO:teuthology.orchestra.run.smithi151.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [579 kB] 2024-03-04T15:23:24.323 INFO:teuthology.orchestra.run.smithi151.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [219 kB] 2024-03-04T15:23:24.334 INFO:teuthology.orchestra.run.smithi151.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [279 kB] 2024-03-04T15:23:24.336 INFO:teuthology.orchestra.run.smithi151.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2024-03-04T15:23:24.339 INFO:teuthology.orchestra.run.smithi151.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [1,476 kB] 2024-03-04T15:23:24.347 INFO:teuthology.orchestra.run.smithi151.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.1 kB] 2024-03-04T15:23:24.348 INFO:teuthology.orchestra.run.smithi151.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [1,504 kB] 2024-03-04T15:23:24.407 INFO:teuthology.orchestra.run.smithi151.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [35.3 kB] 2024-03-04T15:23:24.408 INFO:teuthology.orchestra.run.smithi151.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [247 kB] 2024-03-04T15:23:24.414 INFO:teuthology.orchestra.run.smithi151.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [520 B] 2024-03-04T15:23:24.436 INFO:teuthology.orchestra.run.smithi151.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [33.7 kB] 2024-03-04T15:23:24.437 INFO:teuthology.orchestra.run.smithi151.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [244 kB] 2024-03-04T15:23:24.442 INFO:teuthology.orchestra.run.smithi151.stdout:Get:20 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [520 B] 2024-03-04T15:23:24.442 INFO:teuthology.orchestra.run.smithi151.stdout:Get:21 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [846 kB] 2024-03-04T15:23:24.457 INFO:teuthology.orchestra.run.smithi151.stdout:Get:22 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [596 kB] 2024-03-04T15:23:24.580 INFO:teuthology.orchestra.run.smithi151.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [1,052 kB] 2024-03-04T15:23:24.631 INFO:teuthology.orchestra.run.smithi151.stdout:Get:24 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [161 kB] 2024-03-04T15:23:25.027 INFO:teuthology.orchestra.run.smithi151.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.8 kB] 2024-03-04T15:23:25.034 INFO:teuthology.orchestra.run.smithi151.stdout:Get:26 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [37.1 kB] 2024-03-04T15:23:25.047 INFO:teuthology.orchestra.run.smithi151.stdout:Get:27 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,356 B] 2024-03-04T15:23:25.048 INFO:teuthology.orchestra.run.smithi151.stdout:Get:28 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,476 B] 2024-03-04T15:23:25.051 INFO:teuthology.orchestra.run.smithi151.stdout:Get:29 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2024-03-04T15:23:25.205 INFO:teuthology.orchestra.run.smithi151.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [691 kB] 2024-03-04T15:23:25.244 INFO:teuthology.orchestra.run.smithi151.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [237 kB] 2024-03-04T15:23:25.256 INFO:teuthology.orchestra.run.smithi151.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.1 kB] 2024-03-04T15:23:25.257 INFO:teuthology.orchestra.run.smithi151.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [42.1 kB] 2024-03-04T15:23:25.259 INFO:teuthology.orchestra.run.smithi151.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [4,160 B] 2024-03-04T15:23:25.260 INFO:teuthology.orchestra.run.smithi151.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [10.1 kB] 2024-03-04T15:23:25.260 INFO:teuthology.orchestra.run.smithi151.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2024-03-04T15:23:25.261 INFO:teuthology.orchestra.run.smithi151.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [33.9 kB] 2024-03-04T15:23:25.262 INFO:teuthology.orchestra.run.smithi151.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [41.7 kB] 2024-03-04T15:23:25.288 INFO:teuthology.orchestra.run.smithi151.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2024-03-04T15:23:25.452 INFO:teuthology.orchestra.run.smithi151.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2024-03-04T15:23:25.535 INFO:teuthology.orchestra.run.smithi151.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [24.3 kB] 2024-03-04T15:23:25.631 INFO:teuthology.orchestra.run.smithi151.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [13.4 kB] 2024-03-04T15:23:25.660 INFO:teuthology.orchestra.run.smithi151.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.5 kB] 2024-03-04T15:23:25.711 INFO:teuthology.orchestra.run.smithi151.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [644 B] 2024-03-04T15:23:29.117 INFO:teuthology.orchestra.run.smithi117.stdout:Fetched 11.9 MB in 2s (7,034 kB/s) 2024-03-04T15:23:30.017 INFO:teuthology.orchestra.run.smithi151.stdout:Fetched 11.9 MB in 2s (4,825 kB/s) 2024-03-04T15:23:30.425 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-03-04T15:23:30.442 DEBUG:teuthology.orchestra.run.smithi117:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-04T15:23:30.507 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-03-04T15:23:30.700 INFO:teuthology.orchestra.run.smithi117.stdout:Building dependency tree... 2024-03-04T15:23:30.700 INFO:teuthology.orchestra.run.smithi117.stdout:Reading state information... 2024-03-04T15:23:30.867 INFO:teuthology.orchestra.run.smithi117.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T15:23:30.867 INFO:teuthology.orchestra.run.smithi117.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T15:23:30.868 INFO:teuthology.orchestra.run.smithi117.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T15:23:30.868 INFO:teuthology.orchestra.run.smithi117.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T15:23:30.868 INFO:teuthology.orchestra.run.smithi117.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-04T15:23:30.868 INFO:teuthology.orchestra.run.smithi117.stdout: python2.7-minimal 2024-03-04T15:23:30.868 INFO:teuthology.orchestra.run.smithi117.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T15:23:30.869 INFO:teuthology.orchestra.run.smithi117.stdout:The following additional packages will be installed: 2024-03-04T15:23:30.870 INFO:teuthology.orchestra.run.smithi117.stdout: linux-generic linux-headers-5.15.0-97 linux-headers-5.15.0-97-generic 2024-03-04T15:23:30.870 INFO:teuthology.orchestra.run.smithi117.stdout: linux-headers-generic linux-image-5.15.0-97-generic 2024-03-04T15:23:30.870 INFO:teuthology.orchestra.run.smithi117.stdout: linux-modules-5.15.0-97-generic linux-modules-extra-5.15.0-97-generic 2024-03-04T15:23:30.872 INFO:teuthology.orchestra.run.smithi117.stdout:Suggested packages: 2024-03-04T15:23:30.872 INFO:teuthology.orchestra.run.smithi117.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-04T15:23:30.872 INFO:teuthology.orchestra.run.smithi117.stdout:Recommended packages: 2024-03-04T15:23:30.872 INFO:teuthology.orchestra.run.smithi117.stdout: thermald 2024-03-04T15:23:30.930 INFO:teuthology.orchestra.run.smithi117.stdout:The following NEW packages will be installed: 2024-03-04T15:23:30.930 INFO:teuthology.orchestra.run.smithi117.stdout: linux-headers-5.15.0-97 linux-headers-5.15.0-97-generic 2024-03-04T15:23:30.930 INFO:teuthology.orchestra.run.smithi117.stdout: linux-image-5.15.0-97-generic linux-modules-5.15.0-97-generic 2024-03-04T15:23:30.930 INFO:teuthology.orchestra.run.smithi117.stdout: linux-modules-extra-5.15.0-97-generic 2024-03-04T15:23:30.931 INFO:teuthology.orchestra.run.smithi117.stdout:The following packages will be upgraded: 2024-03-04T15:23:30.932 INFO:teuthology.orchestra.run.smithi117.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-04T15:23:30.989 INFO:teuthology.orchestra.run.smithi117.stdout:3 upgraded, 5 newly installed, 0 to remove and 288 not upgraded. 2024-03-04T15:23:30.989 INFO:teuthology.orchestra.run.smithi117.stdout:Need to get 113 MB of archives. 2024-03-04T15:23:30.989 INFO:teuthology.orchestra.run.smithi117.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-04T15:23:30.989 INFO:teuthology.orchestra.run.smithi117.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-97-generic amd64 5.15.0-97.107 [22.6 MB] 2024-03-04T15:23:31.322 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-03-04T15:23:31.339 DEBUG:teuthology.orchestra.run.smithi151:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-04T15:23:31.391 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-03-04T15:23:31.474 INFO:teuthology.orchestra.run.smithi117.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-97-generic amd64 5.15.0-97.107 [11.5 MB] 2024-03-04T15:23:31.561 INFO:teuthology.orchestra.run.smithi151.stdout:Building dependency tree... 2024-03-04T15:23:31.562 INFO:teuthology.orchestra.run.smithi151.stdout:Reading state information... 2024-03-04T15:23:31.596 INFO:teuthology.orchestra.run.smithi117.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-97-generic amd64 5.15.0-97.107 [63.8 MB] 2024-03-04T15:23:31.714 INFO:teuthology.orchestra.run.smithi151.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T15:23:31.714 INFO:teuthology.orchestra.run.smithi151.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T15:23:31.714 INFO:teuthology.orchestra.run.smithi151.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T15:23:31.714 INFO:teuthology.orchestra.run.smithi151.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T15:23:31.714 INFO:teuthology.orchestra.run.smithi151.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2024-03-04T15:23:31.714 INFO:teuthology.orchestra.run.smithi151.stdout: python2.7-minimal 2024-03-04T15:23:31.715 INFO:teuthology.orchestra.run.smithi151.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T15:23:31.715 INFO:teuthology.orchestra.run.smithi151.stdout:The following additional packages will be installed: 2024-03-04T15:23:31.716 INFO:teuthology.orchestra.run.smithi151.stdout: linux-generic linux-headers-5.15.0-97 linux-headers-5.15.0-97-generic 2024-03-04T15:23:31.716 INFO:teuthology.orchestra.run.smithi151.stdout: linux-headers-generic linux-image-5.15.0-97-generic 2024-03-04T15:23:31.716 INFO:teuthology.orchestra.run.smithi151.stdout: linux-modules-5.15.0-97-generic linux-modules-extra-5.15.0-97-generic 2024-03-04T15:23:31.717 INFO:teuthology.orchestra.run.smithi151.stdout:Suggested packages: 2024-03-04T15:23:31.718 INFO:teuthology.orchestra.run.smithi151.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2024-03-04T15:23:31.718 INFO:teuthology.orchestra.run.smithi151.stdout:Recommended packages: 2024-03-04T15:23:31.718 INFO:teuthology.orchestra.run.smithi151.stdout: thermald 2024-03-04T15:23:31.775 INFO:teuthology.orchestra.run.smithi151.stdout:The following NEW packages will be installed: 2024-03-04T15:23:31.776 INFO:teuthology.orchestra.run.smithi151.stdout: linux-headers-5.15.0-97 linux-headers-5.15.0-97-generic 2024-03-04T15:23:31.776 INFO:teuthology.orchestra.run.smithi151.stdout: linux-image-5.15.0-97-generic linux-modules-5.15.0-97-generic 2024-03-04T15:23:31.776 INFO:teuthology.orchestra.run.smithi151.stdout: linux-modules-extra-5.15.0-97-generic 2024-03-04T15:23:31.777 INFO:teuthology.orchestra.run.smithi151.stdout:The following packages will be upgraded: 2024-03-04T15:23:31.777 INFO:teuthology.orchestra.run.smithi151.stdout: linux-generic linux-headers-generic linux-image-generic 2024-03-04T15:23:31.956 INFO:teuthology.orchestra.run.smithi151.stdout:3 upgraded, 5 newly installed, 0 to remove and 288 not upgraded. 2024-03-04T15:23:31.956 INFO:teuthology.orchestra.run.smithi151.stdout:Need to get 113 MB of archives. 2024-03-04T15:23:31.956 INFO:teuthology.orchestra.run.smithi151.stdout:After this operation, 582 MB of additional disk space will be used. 2024-03-04T15:23:31.956 INFO:teuthology.orchestra.run.smithi151.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-97-generic amd64 5.15.0-97.107 [22.6 MB] 2024-03-04T15:23:32.954 INFO:teuthology.orchestra.run.smithi117.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.97.92 [1,696 B] 2024-03-04T15:23:32.954 INFO:teuthology.orchestra.run.smithi117.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.97.92 [2,494 B] 2024-03-04T15:23:32.954 INFO:teuthology.orchestra.run.smithi117.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-97 all 5.15.0-97.107 [12.3 MB] 2024-03-04T15:23:33.155 INFO:teuthology.orchestra.run.smithi117.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-97-generic amd64 5.15.0-97.107 [2,892 kB] 2024-03-04T15:23:33.200 INFO:teuthology.orchestra.run.smithi117.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.97.92 [2,342 B] 2024-03-04T15:23:33.371 INFO:teuthology.orchestra.run.smithi151.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-97-generic amd64 5.15.0-97.107 [11.5 MB] 2024-03-04T15:23:33.591 INFO:teuthology.orchestra.run.smithi117.stdout:Fetched 113 MB in 2s (50.0 MB/s) 2024-03-04T15:23:33.717 INFO:teuthology.orchestra.run.smithi151.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-97-generic amd64 5.15.0-97.107 [63.8 MB] 2024-03-04T15:23:33.766 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-modules-5.15.0-97-generic. 2024-03-04T15:23:35.696 INFO:teuthology.orchestra.run.smithi117.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-04T15:23:35.701 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../0-linux-modules-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T15:23:35.813 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-modules-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T15:23:35.952 INFO:teuthology.orchestra.run.smithi151.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.97.92 [1,696 B] 2024-03-04T15:23:35.953 INFO:teuthology.orchestra.run.smithi151.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.97.92 [2,494 B] 2024-03-04T15:23:35.953 INFO:teuthology.orchestra.run.smithi151.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-97 all 5.15.0-97.107 [12.3 MB] 2024-03-04T15:23:36.398 INFO:teuthology.orchestra.run.smithi151.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-97-generic amd64 5.15.0-97.107 [2,892 kB] 2024-03-04T15:23:36.485 INFO:teuthology.orchestra.run.smithi151.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.97.92 [2,342 B] 2024-03-04T15:23:36.879 INFO:teuthology.orchestra.run.smithi151.stdout:Fetched 113 MB in 5s (24.0 MB/s) 2024-03-04T15:23:37.099 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package linux-modules-5.15.0-97-generic. 2024-03-04T15:23:37.855 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-image-5.15.0-97-generic. 2024-03-04T15:23:37.877 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../1-linux-image-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T15:23:37.956 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-image-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T15:23:38.350 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-97-generic. 2024-03-04T15:23:38.374 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T15:23:38.408 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-modules-extra-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T15:23:39.122 INFO:teuthology.orchestra.run.smithi151.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2024-03-04T15:23:39.126 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../0-linux-modules-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T15:23:39.205 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-modules-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T15:23:41.564 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package linux-image-5.15.0-97-generic. 2024-03-04T15:23:41.585 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../1-linux-image-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T15:23:41.680 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-image-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T15:23:42.120 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-97-generic. 2024-03-04T15:23:42.138 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T15:23:42.167 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-modules-extra-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T15:23:44.848 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../3-linux-generic_5.15.0.97.92_amd64.deb ... 2024-03-04T15:23:44.967 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-generic (5.15.0.97.92) over (5.15.0.56.54) ... 2024-03-04T15:23:45.309 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.97.92_amd64.deb ... 2024-03-04T15:23:45.437 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-image-generic (5.15.0.97.92) over (5.15.0.56.54) ... 2024-03-04T15:23:45.748 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-headers-5.15.0-97. 2024-03-04T15:23:45.771 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../5-linux-headers-5.15.0-97_5.15.0-97.107_all.deb ... 2024-03-04T15:23:45.806 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-headers-5.15.0-97 (5.15.0-97.107) ... 2024-03-04T15:23:48.308 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../3-linux-generic_5.15.0.97.92_amd64.deb ... 2024-03-04T15:23:48.427 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-generic (5.15.0.97.92) over (5.15.0.56.54) ... 2024-03-04T15:23:48.792 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.97.92_amd64.deb ... 2024-03-04T15:23:48.922 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-image-generic (5.15.0.97.92) over (5.15.0.56.54) ... 2024-03-04T15:23:49.233 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package linux-headers-5.15.0-97. 2024-03-04T15:23:49.245 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../5-linux-headers-5.15.0-97_5.15.0-97.107_all.deb ... 2024-03-04T15:23:49.275 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-headers-5.15.0-97 (5.15.0-97.107) ... 2024-03-04T15:23:51.679 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package linux-headers-5.15.0-97-generic. 2024-03-04T15:23:51.717 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../6-linux-headers-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T15:23:51.754 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-headers-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T15:23:54.282 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.97.92_amd64.deb ... 2024-03-04T15:23:54.407 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking linux-headers-generic (5.15.0.97.92) over (5.15.0.56.54) ... 2024-03-04T15:23:54.526 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package linux-headers-5.15.0-97-generic. 2024-03-04T15:23:54.564 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../6-linux-headers-5.15.0-97-generic_5.15.0-97.107_amd64.deb ... 2024-03-04T15:23:54.601 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-headers-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T15:23:54.731 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-headers-5.15.0-97 (5.15.0-97.107) ... 2024-03-04T15:23:54.835 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-headers-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T15:23:54.936 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-headers-generic (5.15.0.97.92) ... 2024-03-04T15:23:55.039 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-image-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T15:23:56.379 INFO:teuthology.orchestra.run.smithi117.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-04T15:23:56.379 INFO:teuthology.orchestra.run.smithi117.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-04T15:23:56.379 INFO:teuthology.orchestra.run.smithi117.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-97-generic 2024-03-04T15:23:56.379 INFO:teuthology.orchestra.run.smithi117.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-97-generic 2024-03-04T15:23:56.532 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-modules-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T15:23:57.778 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.97.92_amd64.deb ... 2024-03-04T15:23:57.904 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking linux-headers-generic (5.15.0.97.92) over (5.15.0.56.54) ... 2024-03-04T15:23:57.964 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-modules-extra-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T15:23:58.245 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-headers-5.15.0-97 (5.15.0-97.107) ... 2024-03-04T15:23:58.368 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-headers-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T15:23:58.493 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-headers-generic (5.15.0.97.92) ... 2024-03-04T15:23:58.622 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-image-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T15:23:59.551 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-image-generic (5.15.0.97.92) ... 2024-03-04T15:23:59.653 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up linux-generic (5.15.0.97.92) ... 2024-03-04T15:23:59.754 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for linux-image-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T15:23:59.818 INFO:teuthology.orchestra.run.smithi117.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-04T15:23:59.818 INFO:teuthology.orchestra.run.smithi117.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-97-generic 2024-03-04T15:24:00.256 INFO:teuthology.orchestra.run.smithi151.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2024-03-04T15:24:00.256 INFO:teuthology.orchestra.run.smithi151.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2024-03-04T15:24:00.256 INFO:teuthology.orchestra.run.smithi151.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-97-generic 2024-03-04T15:24:00.256 INFO:teuthology.orchestra.run.smithi151.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-97-generic 2024-03-04T15:24:00.419 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-modules-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T15:24:01.858 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-modules-extra-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T15:24:03.276 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-image-generic (5.15.0.97.92) ... 2024-03-04T15:24:03.405 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up linux-generic (5.15.0.97.92) ... 2024-03-04T15:24:03.512 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for linux-image-5.15.0-97-generic (5.15.0-97.107) ... 2024-03-04T15:24:03.590 INFO:teuthology.orchestra.run.smithi151.stdout:/etc/kernel/postinst.d/initramfs-tools: 2024-03-04T15:24:03.590 INFO:teuthology.orchestra.run.smithi151.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-97-generic 2024-03-04T15:24:14.743 INFO:teuthology.orchestra.run.smithi117.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-04T15:24:14.744 INFO:teuthology.orchestra.run.smithi117.stdout:Sourcing file `/etc/default/grub' 2024-03-04T15:24:14.754 INFO:teuthology.orchestra.run.smithi117.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-04T15:24:14.779 INFO:teuthology.orchestra.run.smithi117.stdout:Generating grub configuration file ... 2024-03-04T15:24:15.037 INFO:teuthology.orchestra.run.smithi117.stdout:Found linux image: /boot/vmlinuz-5.15.0-97-generic 2024-03-04T15:24:15.051 INFO:teuthology.orchestra.run.smithi117.stdout:Found initrd image: /boot/initrd.img-5.15.0-97-generic 2024-03-04T15:24:15.349 INFO:teuthology.orchestra.run.smithi117.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-04T15:24:15.355 INFO:teuthology.orchestra.run.smithi117.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-04T15:24:15.424 INFO:teuthology.orchestra.run.smithi117.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-04T15:24:15.430 INFO:teuthology.orchestra.run.smithi117.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-04T15:24:15.602 INFO:teuthology.orchestra.run.smithi117.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-04T15:24:15.602 INFO:teuthology.orchestra.run.smithi117.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-04T15:24:15.602 INFO:teuthology.orchestra.run.smithi117.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-04T15:24:15.640 INFO:teuthology.orchestra.run.smithi117.stdout:done 2024-03-04T15:24:16.094 DEBUG:teuthology.orchestra.run.smithi117:> dpkg -s linux-image-generic 2024-03-04T15:24:16.113 INFO:teuthology.orchestra.run.smithi117.stdout:Package: linux-image-generic 2024-03-04T15:24:16.113 INFO:teuthology.orchestra.run.smithi117.stdout:Status: install ok installed 2024-03-04T15:24:16.113 INFO:teuthology.orchestra.run.smithi117.stdout:Priority: optional 2024-03-04T15:24:16.113 INFO:teuthology.orchestra.run.smithi117.stdout:Section: kernel 2024-03-04T15:24:16.113 INFO:teuthology.orchestra.run.smithi117.stdout:Installed-Size: 21 2024-03-04T15:24:16.113 INFO:teuthology.orchestra.run.smithi117.stdout:Maintainer: Ubuntu Kernel Team 2024-03-04T15:24:16.113 INFO:teuthology.orchestra.run.smithi117.stdout:Architecture: amd64 2024-03-04T15:24:16.113 INFO:teuthology.orchestra.run.smithi117.stdout:Source: linux-meta 2024-03-04T15:24:16.114 INFO:teuthology.orchestra.run.smithi117.stdout:Version: 5.15.0.97.92 2024-03-04T15:24:16.114 INFO:teuthology.orchestra.run.smithi117.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-97), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-04T15:24:16.114 INFO:teuthology.orchestra.run.smithi117.stdout:Depends: linux-image-5.15.0-97-generic, linux-modules-extra-5.15.0-97-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-04T15:24:16.114 INFO:teuthology.orchestra.run.smithi117.stdout:Recommends: thermald 2024-03-04T15:24:16.114 INFO:teuthology.orchestra.run.smithi117.stdout:Description: Generic Linux kernel image 2024-03-04T15:24:16.114 INFO:teuthology.orchestra.run.smithi117.stdout: This package will always depend on the latest generic kernel image 2024-03-04T15:24:16.114 INFO:teuthology.orchestra.run.smithi117.stdout: available. 2024-03-04T15:24:16.114 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-97-generic 2024-03-04T15:24:16.114 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-04T15:24:16.115 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-04T15:24:16.115 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-03-04T15:24:16.115 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi117.front.sepia.ceph.com, path=None, version=distro) 2024-03-04T15:24:16.115 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get clean 2024-03-04T15:24:16.292 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get update 2024-03-04T15:24:16.416 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-04T15:24:16.436 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-04T15:24:16.468 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-04T15:24:16.530 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-04T15:24:17.387 INFO:teuthology.orchestra.run.smithi151.stdout:/etc/kernel/postinst.d/zz-update-grub: 2024-03-04T15:24:17.387 INFO:teuthology.orchestra.run.smithi151.stdout:Sourcing file `/etc/default/grub' 2024-03-04T15:24:17.396 INFO:teuthology.orchestra.run.smithi151.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-04T15:24:17.421 INFO:teuthology.orchestra.run.smithi151.stdout:Generating grub configuration file ... 2024-03-04T15:24:17.718 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-03-04T15:24:17.724 INFO:teuthology.orchestra.run.smithi151.stdout:Found linux image: /boot/vmlinuz-5.15.0-97-generic 2024-03-04T15:24:17.735 DEBUG:teuthology.orchestra.run.smithi117:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-04T15:24:17.755 INFO:teuthology.orchestra.run.smithi151.stdout:Found initrd image: /boot/initrd.img-5.15.0-97-generic 2024-03-04T15:24:17.786 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-03-04T15:24:17.959 INFO:teuthology.orchestra.run.smithi117.stdout:Building dependency tree... 2024-03-04T15:24:17.959 INFO:teuthology.orchestra.run.smithi117.stdout:Reading state information... 2024-03-04T15:24:18.055 INFO:teuthology.orchestra.run.smithi151.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-04T15:24:18.061 INFO:teuthology.orchestra.run.smithi151.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-04T15:24:18.108 INFO:teuthology.orchestra.run.smithi117.stdout:linux-image-generic is already the newest version (5.15.0.97.92). 2024-03-04T15:24:18.108 INFO:teuthology.orchestra.run.smithi117.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T15:24:18.108 INFO:teuthology.orchestra.run.smithi117.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T15:24:18.108 INFO:teuthology.orchestra.run.smithi117.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T15:24:18.108 INFO:teuthology.orchestra.run.smithi117.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T15:24:18.108 INFO:teuthology.orchestra.run.smithi117.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-04T15:24:18.109 INFO:teuthology.orchestra.run.smithi117.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-04T15:24:18.109 INFO:teuthology.orchestra.run.smithi117.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T15:24:18.132 INFO:teuthology.orchestra.run.smithi151.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-04T15:24:18.137 INFO:teuthology.orchestra.run.smithi151.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-04T15:24:18.147 INFO:teuthology.orchestra.run.smithi117.stdout:0 upgraded, 0 newly installed, 0 to remove and 288 not upgraded. 2024-03-04T15:24:18.149 DEBUG:teuthology.orchestra.run.smithi117:> dpkg -s linux-image-generic 2024-03-04T15:24:18.207 INFO:teuthology.orchestra.run.smithi117.stdout:Package: linux-image-generic 2024-03-04T15:24:18.207 INFO:teuthology.orchestra.run.smithi117.stdout:Status: install ok installed 2024-03-04T15:24:18.207 INFO:teuthology.orchestra.run.smithi117.stdout:Priority: optional 2024-03-04T15:24:18.207 INFO:teuthology.orchestra.run.smithi117.stdout:Section: kernel 2024-03-04T15:24:18.207 INFO:teuthology.orchestra.run.smithi117.stdout:Installed-Size: 21 2024-03-04T15:24:18.208 INFO:teuthology.orchestra.run.smithi117.stdout:Maintainer: Ubuntu Kernel Team 2024-03-04T15:24:18.208 INFO:teuthology.orchestra.run.smithi117.stdout:Architecture: amd64 2024-03-04T15:24:18.208 INFO:teuthology.orchestra.run.smithi117.stdout:Source: linux-meta 2024-03-04T15:24:18.208 INFO:teuthology.orchestra.run.smithi117.stdout:Version: 5.15.0.97.92 2024-03-04T15:24:18.208 INFO:teuthology.orchestra.run.smithi117.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-97), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-04T15:24:18.208 INFO:teuthology.orchestra.run.smithi117.stdout:Depends: linux-image-5.15.0-97-generic, linux-modules-extra-5.15.0-97-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-04T15:24:18.208 INFO:teuthology.orchestra.run.smithi117.stdout:Recommends: thermald 2024-03-04T15:24:18.208 INFO:teuthology.orchestra.run.smithi117.stdout:Description: Generic Linux kernel image 2024-03-04T15:24:18.208 INFO:teuthology.orchestra.run.smithi117.stdout: This package will always depend on the latest generic kernel image 2024-03-04T15:24:18.208 INFO:teuthology.orchestra.run.smithi117.stdout: available. 2024-03-04T15:24:18.209 DEBUG:teuthology.orchestra.run.smithi117:> mktemp 2024-03-04T15:24:18.257 INFO:teuthology.orchestra.run.smithi117.stdout:/tmp/tmp.KCOcjxrJcW 2024-03-04T15:24:18.257 DEBUG:teuthology.orchestra.run.smithi117:> sudo cp /boot/grub/grub.cfg /tmp/tmp.KCOcjxrJcW 2024-03-04T15:24:18.313 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0666 /tmp/tmp.KCOcjxrJcW 2024-03-04T15:24:18.313 INFO:teuthology.orchestra.run.smithi151.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-04T15:24:18.314 INFO:teuthology.orchestra.run.smithi151.stdout:Systems on them will not be added to the GRUB boot configuration. 2024-03-04T15:24:18.314 INFO:teuthology.orchestra.run.smithi151.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-04T15:24:18.362 INFO:teuthology.orchestra.run.smithi151.stdout:done 2024-03-04T15:24:18.440 DEBUG:teuthology.orchestra.remote:smithi117:/tmp/tmp.KCOcjxrJcW is 10KB 2024-03-04T15:24:18.456 DEBUG:teuthology.orchestra.run.smithi117:> rm -fr /tmp/tmp.KCOcjxrJcW 2024-03-04T15:24:18.469 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-04T15:24:18.520 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-04T15:24:18.520 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-04T15:24:18.520 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-04T15:24:18.588 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-97-generic 2024-03-04T15:24:18.589 DEBUG:teuthology.orchestra.run.smithi117:> sudo update-grub 2024-03-04T15:24:18.783 DEBUG:teuthology.orchestra.run.smithi151:> dpkg -s linux-image-generic 2024-03-04T15:24:18.803 INFO:teuthology.orchestra.run.smithi151.stdout:Package: linux-image-generic 2024-03-04T15:24:18.803 INFO:teuthology.orchestra.run.smithi151.stdout:Status: install ok installed 2024-03-04T15:24:18.803 INFO:teuthology.orchestra.run.smithi151.stdout:Priority: optional 2024-03-04T15:24:18.803 INFO:teuthology.orchestra.run.smithi151.stdout:Section: kernel 2024-03-04T15:24:18.803 INFO:teuthology.orchestra.run.smithi151.stdout:Installed-Size: 21 2024-03-04T15:24:18.803 INFO:teuthology.orchestra.run.smithi151.stdout:Maintainer: Ubuntu Kernel Team 2024-03-04T15:24:18.803 INFO:teuthology.orchestra.run.smithi151.stdout:Architecture: amd64 2024-03-04T15:24:18.803 INFO:teuthology.orchestra.run.smithi151.stdout:Source: linux-meta 2024-03-04T15:24:18.803 INFO:teuthology.orchestra.run.smithi151.stdout:Version: 5.15.0.97.92 2024-03-04T15:24:18.803 INFO:teuthology.orchestra.run.smithi151.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-97), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-04T15:24:18.803 INFO:teuthology.orchestra.run.smithi151.stdout:Depends: linux-image-5.15.0-97-generic, linux-modules-extra-5.15.0-97-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-04T15:24:18.803 INFO:teuthology.orchestra.run.smithi151.stdout:Recommends: thermald 2024-03-04T15:24:18.803 INFO:teuthology.orchestra.run.smithi151.stdout:Description: Generic Linux kernel image 2024-03-04T15:24:18.804 INFO:teuthology.orchestra.run.smithi151.stdout: This package will always depend on the latest generic kernel image 2024-03-04T15:24:18.804 INFO:teuthology.orchestra.run.smithi151.stdout: available. 2024-03-04T15:24:18.804 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-97-generic 2024-03-04T15:24:18.804 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-03-04T15:24:18.804 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-03-04T15:24:18.804 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-03-04T15:24:18.804 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi151.front.sepia.ceph.com, path=None, version=distro) 2024-03-04T15:24:18.804 DEBUG:teuthology.orchestra.run.smithi151:> sudo apt-get clean 2024-03-04T15:24:18.959 DEBUG:teuthology.orchestra.run.smithi151:> sudo apt-get update 2024-03-04T15:24:19.130 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-04T15:24:19.154 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-04T15:24:19.196 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-04T15:24:19.197 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:4 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-04T15:24:19.734 INFO:teuthology.orchestra.run.smithi117.stderr:Sourcing file `/etc/default/grub' 2024-03-04T15:24:19.735 INFO:teuthology.orchestra.run.smithi117.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-04T15:24:19.737 INFO:teuthology.orchestra.run.smithi117.stderr:Generating grub configuration file ... 2024-03-04T15:24:19.931 INFO:teuthology.orchestra.run.smithi117.stderr:Found linux image: /boot/vmlinuz-5.15.0-97-generic 2024-03-04T15:24:19.938 INFO:teuthology.orchestra.run.smithi117.stderr:Found initrd image: /boot/initrd.img-5.15.0-97-generic 2024-03-04T15:24:20.177 INFO:teuthology.orchestra.run.smithi117.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-04T15:24:20.182 INFO:teuthology.orchestra.run.smithi117.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-04T15:24:20.224 INFO:teuthology.orchestra.run.smithi117.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-04T15:24:20.229 INFO:teuthology.orchestra.run.smithi117.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-04T15:24:20.347 INFO:teuthology.orchestra.run.smithi117.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-04T15:24:20.347 INFO:teuthology.orchestra.run.smithi117.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-04T15:24:20.347 INFO:teuthology.orchestra.run.smithi117.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-04T15:24:20.364 INFO:teuthology.orchestra.run.smithi117.stderr:done 2024-03-04T15:24:20.382 DEBUG:teuthology.orchestra.run.smithi117:> sudo shutdown -r now 2024-03-04T15:24:20.438 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-03-04T15:24:20.457 DEBUG:teuthology.orchestra.run.smithi151:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2024-03-04T15:24:20.517 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-03-04T15:24:20.708 INFO:teuthology.orchestra.run.smithi151.stdout:Building dependency tree... 2024-03-04T15:24:20.708 INFO:teuthology.orchestra.run.smithi151.stdout:Reading state information... 2024-03-04T15:24:20.870 INFO:teuthology.orchestra.run.smithi151.stdout:linux-image-generic is already the newest version (5.15.0.97.92). 2024-03-04T15:24:20.870 INFO:teuthology.orchestra.run.smithi151.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T15:24:20.870 INFO:teuthology.orchestra.run.smithi151.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T15:24:20.870 INFO:teuthology.orchestra.run.smithi151.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T15:24:20.870 INFO:teuthology.orchestra.run.smithi151.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T15:24:20.870 INFO:teuthology.orchestra.run.smithi151.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-04T15:24:20.870 INFO:teuthology.orchestra.run.smithi151.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-04T15:24:20.870 INFO:teuthology.orchestra.run.smithi151.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T15:24:20.907 INFO:teuthology.orchestra.run.smithi151.stdout:0 upgraded, 0 newly installed, 0 to remove and 288 not upgraded. 2024-03-04T15:24:20.912 DEBUG:teuthology.orchestra.run.smithi151:> dpkg -s linux-image-generic 2024-03-04T15:24:20.966 INFO:teuthology.orchestra.run.smithi151.stdout:Package: linux-image-generic 2024-03-04T15:24:20.966 INFO:teuthology.orchestra.run.smithi151.stdout:Status: install ok installed 2024-03-04T15:24:20.966 INFO:teuthology.orchestra.run.smithi151.stdout:Priority: optional 2024-03-04T15:24:20.966 INFO:teuthology.orchestra.run.smithi151.stdout:Section: kernel 2024-03-04T15:24:20.966 INFO:teuthology.orchestra.run.smithi151.stdout:Installed-Size: 21 2024-03-04T15:24:20.966 INFO:teuthology.orchestra.run.smithi151.stdout:Maintainer: Ubuntu Kernel Team 2024-03-04T15:24:20.966 INFO:teuthology.orchestra.run.smithi151.stdout:Architecture: amd64 2024-03-04T15:24:20.966 INFO:teuthology.orchestra.run.smithi151.stdout:Source: linux-meta 2024-03-04T15:24:20.966 INFO:teuthology.orchestra.run.smithi151.stdout:Version: 5.15.0.97.92 2024-03-04T15:24:20.966 INFO:teuthology.orchestra.run.smithi151.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.2), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-97), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.2) 2024-03-04T15:24:20.967 INFO:teuthology.orchestra.run.smithi151.stdout:Depends: linux-image-5.15.0-97-generic, linux-modules-extra-5.15.0-97-generic, linux-firmware, intel-microcode, amd64-microcode 2024-03-04T15:24:20.967 INFO:teuthology.orchestra.run.smithi151.stdout:Recommends: thermald 2024-03-04T15:24:20.967 INFO:teuthology.orchestra.run.smithi151.stdout:Description: Generic Linux kernel image 2024-03-04T15:24:20.967 INFO:teuthology.orchestra.run.smithi151.stdout: This package will always depend on the latest generic kernel image 2024-03-04T15:24:20.967 INFO:teuthology.orchestra.run.smithi151.stdout: available. 2024-03-04T15:24:20.967 DEBUG:teuthology.orchestra.run.smithi151:> mktemp 2024-03-04T15:24:21.014 INFO:teuthology.orchestra.run.smithi151.stdout:/tmp/tmp.WAlaOdTVHU 2024-03-04T15:24:21.015 DEBUG:teuthology.orchestra.run.smithi151:> sudo cp /boot/grub/grub.cfg /tmp/tmp.WAlaOdTVHU 2024-03-04T15:24:21.070 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0666 /tmp/tmp.WAlaOdTVHU 2024-03-04T15:24:21.197 DEBUG:teuthology.orchestra.remote:smithi151:/tmp/tmp.WAlaOdTVHU is 10KB 2024-03-04T15:24:21.210 DEBUG:teuthology.orchestra.run.smithi151:> rm -fr /tmp/tmp.WAlaOdTVHU 2024-03-04T15:24:21.216 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2024-03-04T15:24:21.275 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-04T15:24:21.275 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/grub.d/01_ceph_kernel 2024-03-04T15:24:21.275 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2024-03-04T15:24:21.344 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-97-generic 2024-03-04T15:24:21.345 DEBUG:teuthology.orchestra.run.smithi151:> sudo update-grub 2024-03-04T15:24:22.453 INFO:teuthology.orchestra.run.smithi151.stderr:Sourcing file `/etc/default/grub' 2024-03-04T15:24:22.455 INFO:teuthology.orchestra.run.smithi151.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2024-03-04T15:24:22.457 INFO:teuthology.orchestra.run.smithi151.stderr:Generating grub configuration file ... 2024-03-04T15:24:22.632 INFO:teuthology.orchestra.run.smithi151.stderr:Found linux image: /boot/vmlinuz-5.15.0-97-generic 2024-03-04T15:24:22.638 INFO:teuthology.orchestra.run.smithi151.stderr:Found initrd image: /boot/initrd.img-5.15.0-97-generic 2024-03-04T15:24:22.871 INFO:teuthology.orchestra.run.smithi151.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2024-03-04T15:24:22.877 INFO:teuthology.orchestra.run.smithi151.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2024-03-04T15:24:22.918 INFO:teuthology.orchestra.run.smithi151.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2024-03-04T15:24:22.923 INFO:teuthology.orchestra.run.smithi151.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2024-03-04T15:24:23.048 INFO:teuthology.orchestra.run.smithi151.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2024-03-04T15:24:23.048 INFO:teuthology.orchestra.run.smithi151.stderr:Systems on them will not be added to the GRUB boot configuration. 2024-03-04T15:24:23.048 INFO:teuthology.orchestra.run.smithi151.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2024-03-04T15:24:23.060 INFO:teuthology.orchestra.run.smithi151.stderr:done 2024-03-04T15:24:23.069 DEBUG:teuthology.orchestra.run.smithi151:> sudo shutdown -r now 2024-03-04T15:24:50.412 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-03-04T15:24:50.413 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-04T15:24:50.413 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T15:24:53.075 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-03-04T15:24:53.075 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-03-04T15:24:53.076 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T15:25:24.925 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-03-04T15:25:27.744 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.151 2024-03-04T15:25:33.932 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-04T15:25:33.933 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T15:25:36.746 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-03-04T15:25:36.747 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T15:25:52.316 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-03-04T15:26:04.329 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-04T15:26:04.330 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T15:26:06.083 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.117 2024-03-04T15:26:21.096 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-04T15:26:21.097 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T15:26:21.401 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-03-04T15:26:21.967 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-04T15:26:21.967 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-97-generic"... 2024-03-04T15:26:21.967 DEBUG:teuthology.orchestra.run.smithi117:> uname -r 2024-03-04T15:26:22.016 INFO:teuthology.orchestra.run.smithi117.stdout:5.15.0-97-generic 2024-03-04T15:26:22.017 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-97-generic vs 5.15.0-97-generic 2024-03-04T15:26:22.017 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-04T15:26:22.017 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-04T15:26:23.017 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-03-04T15:26:23.018 DEBUG:teuthology.orchestra.run.smithi117:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-04T15:26:23.393 INFO:teuthology.orchestra.run.smithi117.stdout:ttyS1 2024-03-04T15:26:23.426 DEBUG:teuthology.parallel:result is None 2024-03-04T15:26:36.763 DEBUG:teuthology.orchestra.remote:timed out 2024-03-04T15:26:48.765 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-03-04T15:26:48.766 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T15:26:49.049 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-03-04T15:26:49.573 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-03-04T15:26:49.573 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-97-generic"... 2024-03-04T15:26:49.574 DEBUG:teuthology.orchestra.run.smithi151:> uname -r 2024-03-04T15:26:49.621 INFO:teuthology.orchestra.run.smithi151.stdout:5.15.0-97-generic 2024-03-04T15:26:49.622 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-97-generic vs 5.15.0-97-generic 2024-03-04T15:26:49.622 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-03-04T15:26:49.622 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2024-03-04T15:26:50.623 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-03-04T15:26:50.623 DEBUG:teuthology.orchestra.run.smithi151:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-03-04T15:26:50.948 INFO:teuthology.orchestra.run.smithi151.stdout:ttyS1 2024-03-04T15:26:50.990 DEBUG:teuthology.parallel:result is None 2024-03-04T15:26:50.991 INFO:teuthology.run_tasks:Running task internal.base... 2024-03-04T15:26:50.998 INFO:teuthology.task.internal:Creating test directory... 2024-03-04T15:26:50.998 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-04T15:26:51.001 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-03-04T15:26:51.007 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-03-04T15:26:51.013 INFO:teuthology.run_tasks:Running task internal.archive... 2024-03-04T15:26:51.036 INFO:teuthology.task.internal:Creating archive directory... 2024-03-04T15:26:51.036 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-04T15:26:51.049 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-03-04T15:26:51.128 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-03-04T15:26:51.135 INFO:teuthology.task.internal:Enabling coredump saving... 2024-03-04T15:26:51.135 DEBUG:teuthology.orchestra.run.smithi117:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-04T15:26:51.137 DEBUG:teuthology.orchestra.run.smithi151:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-03-04T15:26:51.152 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T15:26:51.155 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T15:26:51.159 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T15:26:51.165 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-03-04T15:26:51.166 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-03-04T15:26:51.173 INFO:teuthology.task.internal:Configuring sudo... 2024-03-04T15:26:51.174 DEBUG:teuthology.orchestra.run.smithi117:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-04T15:26:51.206 DEBUG:teuthology.orchestra.run.smithi151:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-03-04T15:26:51.221 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-03-04T15:26:51.231 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-03-04T15:26:51.231 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-04T15:26:51.261 DEBUG:teuthology.orchestra.run.smithi151:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-03-04T15:26:51.268 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T15:26:51.315 DEBUG:teuthology.orchestra.run.smithi117:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T15:26:51.364 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-04T15:26:51.365 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-04T15:26:51.437 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-03-04T15:26:51.443 DEBUG:teuthology.orchestra.run.smithi151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-03-04T15:26:51.489 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-04T15:26:51.489 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-03-04T15:26:51.563 DEBUG:teuthology.orchestra.run.smithi117:> sudo service rsyslog restart 2024-03-04T15:26:51.565 DEBUG:teuthology.orchestra.run.smithi151:> sudo service rsyslog restart 2024-03-04T15:26:51.635 INFO:teuthology.run_tasks:Running task internal.timer... 2024-03-04T15:26:51.643 INFO:teuthology.task.internal:Starting timer... 2024-03-04T15:26:51.644 INFO:teuthology.run_tasks:Running task pcp... 2024-03-04T15:26:51.653 INFO:teuthology.run_tasks:Running task selinux... 2024-03-04T15:26:51.662 DEBUG:teuthology.task.selinux:Excluding smithi117: OS 'ubuntu' does not support SELinux 2024-03-04T15:26:51.662 DEBUG:teuthology.task.selinux:Excluding smithi151: OS 'ubuntu' does not support SELinux 2024-03-04T15:26:51.662 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-03-04T15:26:51.663 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2024-03-04T15:26:51.663 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-03-04T15:26:51.663 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-03-04T15:26:51.672 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-03-04T15:26:51.678 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-03-04T15:26:51.854 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-03-04T15:26:51.873 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-03-04T15:26:51.875 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi117.front.sepia.ceph.com,smithi151.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-03-04T15:33:55.408 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi117.front.sepia.ceph.com'), Remote(name='ubuntu@smithi151.front.sepia.ceph.com')] 2024-03-04T15:33:55.410 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-04T15:33:55.411 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi117.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T15:33:55.518 DEBUG:teuthology.orchestra.run.smithi117:> true 2024-03-04T15:33:55.596 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi117.front.sepia.ceph.com' 2024-03-04T15:33:55.596 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-03-04T15:33:55.596 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-03-04T15:33:55.680 DEBUG:teuthology.orchestra.run.smithi151:> true 2024-03-04T15:33:55.756 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi151.front.sepia.ceph.com' 2024-03-04T15:33:55.756 INFO:teuthology.run_tasks:Running task clock... 2024-03-04T15:33:55.766 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-03-04T15:33:55.766 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-04T15:33:55.767 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T15:33:55.769 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-03-04T15:33:55.769 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T15:33:55.794 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:33:55 ntpd[17142]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-04T15:33:55.794 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:33:55 ntpd[17142]: Command line: ntpd -gq 2024-03-04T15:33:55.794 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:33:55 ntpd[17142]: ---------------------------------------------------- 2024-03-04T15:33:55.794 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:33:55 ntpd[17142]: ntp-4 is maintained by Network Time Foundation, 2024-03-04T15:33:55.794 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:33:55 ntpd[17142]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-04T15:33:55.794 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:33:55 ntpd[17142]: corporation. Support and training for ntp-4 are 2024-03-04T15:33:55.795 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:33:55 ntpd[17142]: available at https://www.nwtime.org/support 2024-03-04T15:33:55.795 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:33:55 ntpd[17142]: ---------------------------------------------------- 2024-03-04T15:33:55.795 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:33:55 ntpd[17142]: proto: precision = 0.044 usec (-24) 2024-03-04T15:33:55.795 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:33:55 ntpd[17142]: basedate set to 2022-02-04 2024-03-04T15:33:55.795 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:33:55 ntpd[17142]: gps base set to 2022-02-06 (week 2196) 2024-03-04T15:33:55.796 INFO:teuthology.orchestra.run.smithi117.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-04T15:33:55.796 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:33:55 ntpd[17142]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-04T15:33:55.796 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:33:55 ntpd[17142]: restrict ::: KOD does nothing without LIMITED. 2024-03-04T15:33:55.796 INFO:teuthology.orchestra.run.smithi117.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-04T15:33:55.797 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:33:55 ntpd[17142]: Listen and drop on 0 v6wildcard [::]:123 2024-03-04T15:33:55.797 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:33:55 ntpd[17142]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-04T15:33:55.797 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:33:55 ntpd[17142]: Listen normally on 2 lo 127.0.0.1:123 2024-03-04T15:33:55.798 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:33:55 ntpd[17142]: Listen normally on 3 enp3s0f1 172.21.15.117:123 2024-03-04T15:33:55.798 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:33:55 ntpd[17142]: Listen normally on 4 lo [::1]:123 2024-03-04T15:33:55.798 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:33:55 ntpd[17142]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:4463%5]:123 2024-03-04T15:33:55.798 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:33:55 ntpd[17142]: Listening on routing socket on fd #22 for interface updates 2024-03-04T15:33:55.821 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:33:55 ntpd[17158]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2024-03-04T15:33:55.821 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:33:55 ntpd[17158]: Command line: ntpd -gq 2024-03-04T15:33:55.821 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:33:55 ntpd[17158]: ---------------------------------------------------- 2024-03-04T15:33:55.821 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:33:55 ntpd[17158]: ntp-4 is maintained by Network Time Foundation, 2024-03-04T15:33:55.821 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:33:55 ntpd[17158]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2024-03-04T15:33:55.821 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:33:55 ntpd[17158]: corporation. Support and training for ntp-4 are 2024-03-04T15:33:55.821 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:33:55 ntpd[17158]: available at https://www.nwtime.org/support 2024-03-04T15:33:55.822 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:33:55 ntpd[17158]: ---------------------------------------------------- 2024-03-04T15:33:55.822 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:33:55 ntpd[17158]: proto: precision = 0.063 usec (-24) 2024-03-04T15:33:55.822 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:33:55 ntpd[17158]: basedate set to 2022-02-04 2024-03-04T15:33:55.822 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:33:55 ntpd[17158]: gps base set to 2022-02-06 (week 2196) 2024-03-04T15:33:55.823 INFO:teuthology.orchestra.run.smithi151.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-04T15:33:55.823 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:33:55 ntpd[17158]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2024-03-04T15:33:55.823 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:33:55 ntpd[17158]: restrict ::: KOD does nothing without LIMITED. 2024-03-04T15:33:55.823 INFO:teuthology.orchestra.run.smithi151.stderr:restrict ::: KOD does nothing without LIMITED. 2024-03-04T15:33:55.824 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:33:55 ntpd[17158]: Listen and drop on 0 v6wildcard [::]:123 2024-03-04T15:33:55.824 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:33:55 ntpd[17158]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2024-03-04T15:33:55.825 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:33:55 ntpd[17158]: Listen normally on 2 lo 127.0.0.1:123 2024-03-04T15:33:55.825 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:33:55 ntpd[17158]: Listen normally on 3 enp3s0f1 172.21.15.151:123 2024-03-04T15:33:55.825 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:33:55 ntpd[17158]: Listen normally on 4 lo [::1]:123 2024-03-04T15:33:55.825 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:33:55 ntpd[17158]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe88:6f63%5]:123 2024-03-04T15:33:55.825 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:33:55 ntpd[17158]: Listening on routing socket on fd #22 for interface updates 2024-03-04T15:33:56.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:56 ntpd[17142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:33:56.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:56 ntpd[17142]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:33:56.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:56 ntpd[17142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:33:56.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:56 ntpd[17142]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:33:56.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:56 ntpd[17142]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T15:33:56.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:56 ntpd[17142]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T15:33:57.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:57 ntpd[17142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:33:57.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:57 ntpd[17142]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:33:57.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:57 ntpd[17142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:33:57.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:57 ntpd[17142]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:33:57.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:57 ntpd[17142]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T15:33:57.796 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:57 ntpd[17142]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T15:33:58.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:58 ntpd[17142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:33:58.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:58 ntpd[17142]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:33:58.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:58 ntpd[17142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:33:58.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:58 ntpd[17142]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:33:58.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:58 ntpd[17142]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T15:33:58.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:58 ntpd[17142]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T15:33:58.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:33:58 ntpd[17158]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:33:58.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:33:58 ntpd[17158]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:33:58.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:33:58 ntpd[17158]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:33:58.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:33:58 ntpd[17158]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:33:58.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:33:58 ntpd[17158]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T15:33:58.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:33:58 ntpd[17158]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T15:33:59.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:59 ntpd[17142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:33:59.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:59 ntpd[17142]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:33:59.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:59 ntpd[17142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:33:59.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:59 ntpd[17142]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:33:59.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:59 ntpd[17142]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T15:33:59.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:33:59 ntpd[17142]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T15:33:59.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:33:59 ntpd[17158]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:33:59.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:33:59 ntpd[17158]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:33:59.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:33:59 ntpd[17158]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:33:59.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:33:59 ntpd[17158]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:33:59.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:33:59 ntpd[17158]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:33:59.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:33:59 ntpd[17158]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:33:59.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:33:59 ntpd[17158]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T15:33:59.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:33:59 ntpd[17158]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T15:34:00.796 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:00 ntpd[17142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:00.796 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:00 ntpd[17142]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:00.796 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:00 ntpd[17142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:00.796 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:00 ntpd[17142]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:00.796 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:00 ntpd[17142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:00.796 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:00 ntpd[17142]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:00.796 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:00 ntpd[17142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:00.796 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:00 ntpd[17142]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:00.796 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:00 ntpd[17142]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T15:34:00.796 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:00 ntpd[17142]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T15:34:00.796 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:00 ntpd[17142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:00.797 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:00 ntpd[17142]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:00.797 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:00 ntpd[17142]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T15:34:00.797 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:00 ntpd[17142]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T15:34:00.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:00 ntpd[17158]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:00.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:00 ntpd[17158]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:00.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:00 ntpd[17158]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:00.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:00 ntpd[17158]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:00.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:00 ntpd[17158]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:00.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:00 ntpd[17158]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:00.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:00 ntpd[17158]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T15:34:00.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:00 ntpd[17158]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T15:34:00.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:00 ntpd[17158]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:00.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:00 ntpd[17158]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:00.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:00 ntpd[17158]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T15:34:00.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:00 ntpd[17158]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T15:34:01.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:01 ntpd[17142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:01.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:01 ntpd[17142]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:01.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:01 ntpd[17142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:01.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:01 ntpd[17142]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:01.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:01 ntpd[17142]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T15:34:01.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:01 ntpd[17142]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T15:34:01.821 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:01 ntpd[17158]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:01.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:01 ntpd[17158]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:01.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:01 ntpd[17158]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:01.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:01 ntpd[17158]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:01.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:01 ntpd[17158]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T15:34:01.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:01 ntpd[17158]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T15:34:02.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:02 ntpd[17142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:02.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:02 ntpd[17142]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:02.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:02 ntpd[17142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:02.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:02 ntpd[17142]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:02.795 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:02 ntpd[17142]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:02.796 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:02 ntpd[17142]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:02.796 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:02 ntpd[17142]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T15:34:02.796 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:02 ntpd[17142]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T15:34:02.796 INFO:teuthology.orchestra.run.smithi117.stdout: 4 Mar 15:34:02 ntpd[17142]: ntpd: time slew -0.006339 s 2024-03-04T15:34:02.796 INFO:teuthology.orchestra.run.smithi117.stdout:ntpd: time slew -0.006339s 2024-03-04T15:34:02.797 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:02 ntpd[17142]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-04T15:34:02.797 INFO:teuthology.orchestra.run.smithi117.stderr: 4 Mar 15:34:02 ntpd[17142]: can't open /var/log/ntpstats/loopstats.20240304: Permission denied 2024-03-04T15:34:02.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:02 ntpd[17158]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:02.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:02 ntpd[17158]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:02.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:02 ntpd[17158]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:02.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:02 ntpd[17158]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:02.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:02 ntpd[17158]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:02.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:02 ntpd[17158]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:02.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:02 ntpd[17158]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T15:34:02.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:02 ntpd[17158]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T15:34:02.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:02 ntpd[17158]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:02.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:02 ntpd[17158]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:02.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:02 ntpd[17158]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T15:34:02.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:02 ntpd[17158]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T15:34:02.846 INFO:teuthology.orchestra.run.smithi117.stdout: remote refid st t when poll reach delay offset jitter 2024-03-04T15:34:02.846 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================== 2024-03-04T15:34:02.846 INFO:teuthology.orchestra.run.smithi117.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T15:34:02.846 INFO:teuthology.orchestra.run.smithi117.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T15:34:02.846 INFO:teuthology.orchestra.run.smithi117.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T15:34:02.847 INFO:teuthology.orchestra.run.smithi117.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T15:34:03.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:03 ntpd[17158]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:03.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:03 ntpd[17158]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:03.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:03 ntpd[17158]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:03.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:03 ntpd[17158]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:03.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:03 ntpd[17158]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T15:34:03.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:03 ntpd[17158]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T15:34:04.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:04 ntpd[17158]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:04.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:04 ntpd[17158]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:04.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:04 ntpd[17158]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:04.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:04 ntpd[17158]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:04.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:04 ntpd[17158]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2024-03-04T15:34:04.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:04 ntpd[17158]: can't open /var/log/ntpstats/rawstats.20240304: Permission denied 2024-03-04T15:34:04.822 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:04 ntpd[17158]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2024-03-04T15:34:04.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:04 ntpd[17158]: can't open /var/log/ntpstats/peerstats.20240304: Permission denied 2024-03-04T15:34:04.823 INFO:teuthology.orchestra.run.smithi151.stdout: 4 Mar 15:34:04 ntpd[17158]: ntpd: time slew -0.001230 s 2024-03-04T15:34:04.823 INFO:teuthology.orchestra.run.smithi151.stdout:ntpd: time slew -0.001230s 2024-03-04T15:34:04.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:04 ntpd[17158]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2024-03-04T15:34:04.823 INFO:teuthology.orchestra.run.smithi151.stderr: 4 Mar 15:34:04 ntpd[17158]: can't open /var/log/ntpstats/loopstats.20240304: Permission denied 2024-03-04T15:34:04.879 INFO:teuthology.orchestra.run.smithi151.stdout: remote refid st t when poll reach delay offset jitter 2024-03-04T15:34:04.879 INFO:teuthology.orchestra.run.smithi151.stdout:============================================================================== 2024-03-04T15:34:04.879 INFO:teuthology.orchestra.run.smithi151.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T15:34:04.879 INFO:teuthology.orchestra.run.smithi151.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T15:34:04.879 INFO:teuthology.orchestra.run.smithi151.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T15:34:04.879 INFO:teuthology.orchestra.run.smithi151.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2024-03-04T15:34:04.880 INFO:teuthology.run_tasks:Running task install... 2024-03-04T15:34:04.888 DEBUG:teuthology.task.install:project ceph 2024-03-04T15:34:04.888 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '9ca0b5de35ed230e84387abdd8e619ed8e4f3282'}, 'extra_packages': ['cephadm']} 2024-03-04T15:34:04.888 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '9ca0b5de35ed230e84387abdd8e619ed8e4f3282'} 2024-03-04T15:34:04.889 INFO:teuthology.task.install:Using flavor: default 2024-03-04T15:34:04.894 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-03-04T15:34:04.894 INFO:teuthology.task.install:extra packages: [] 2024-03-04T15:34:04.895 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-key list | grep Ceph 2024-03-04T15:34:04.895 DEBUG:teuthology.orchestra.run.smithi151:> sudo apt-key list | grep Ceph 2024-03-04T15:34:04.958 INFO:teuthology.orchestra.run.smithi117.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-04T15:34:04.980 INFO:teuthology.orchestra.run.smithi151.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2024-03-04T15:34:04.987 INFO:teuthology.orchestra.run.smithi117.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-04T15:34:04.987 INFO:teuthology.orchestra.run.smithi117.stdout:uid [ unknown] Ceph.com (release key) 2024-03-04T15:34:04.988 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-03-04T15:34:04.988 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=9ca0b5de35ed230e84387abdd8e619ed8e4f3282 2024-03-04T15:34:05.009 INFO:teuthology.orchestra.run.smithi151.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2024-03-04T15:34:05.009 INFO:teuthology.orchestra.run.smithi151.stdout:uid [ unknown] Ceph.com (release key) 2024-03-04T15:34:05.010 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse on remote deb x86_64 2024-03-04T15:34:05.010 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=9ca0b5de35ed230e84387abdd8e619ed8e4f3282 2024-03-04T15:34:05.215 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default/ 2024-03-04T15:34:05.215 INFO:teuthology.task.install.deb:Package version is 19.0.0-1826-g9ca0b5de-1jammy 2024-03-04T15:34:05.233 INFO:teuthology.task.install.deb:Pulling from https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default/ 2024-03-04T15:34:05.233 INFO:teuthology.task.install.deb:Package version is 19.0.0-1826-g9ca0b5de-1jammy 2024-03-04T15:34:05.334 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-04T15:34:05.334 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-04T15:34:05.348 DEBUG:teuthology.orchestra.run.smithi117:> sudo apt-get update 2024-03-04T15:34:05.411 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-04T15:34:05.411 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2024-03-04T15:34:05.425 DEBUG:teuthology.orchestra.run.smithi151:> sudo apt-get update 2024-03-04T15:34:05.548 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-04T15:34:05.559 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-04T15:34:05.579 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-04T15:34:05.636 INFO:teuthology.orchestra.run.smithi117.stdout:Ign:4 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy InRelease 2024-03-04T15:34:05.666 INFO:teuthology.orchestra.run.smithi117.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-04T15:34:05.672 INFO:teuthology.orchestra.run.smithi117.stdout:Get:6 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-04T15:34:05.706 INFO:teuthology.orchestra.run.smithi151.stdout:Ign:1 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy InRelease 2024-03-04T15:34:05.709 INFO:teuthology.orchestra.run.smithi117.stdout:Ign:7 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-04T15:34:05.721 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2024-03-04T15:34:05.721 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2024-03-04T15:34:05.742 INFO:teuthology.orchestra.run.smithi151.stdout:Get:4 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2024-03-04T15:34:05.745 INFO:teuthology.orchestra.run.smithi117.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,409 B] 2024-03-04T15:34:05.779 INFO:teuthology.orchestra.run.smithi151.stdout:Ign:5 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy Release.gpg 2024-03-04T15:34:05.786 INFO:teuthology.orchestra.run.smithi117.stdout:Get:9 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-03-04T15:34:05.793 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2024-03-04T15:34:05.816 INFO:teuthology.orchestra.run.smithi151.stdout:Get:7 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.8 kB] 2024-03-04T15:34:05.859 INFO:teuthology.orchestra.run.smithi151.stdout:Get:8 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,409 B] 2024-03-04T15:34:05.875 INFO:teuthology.orchestra.run.smithi151.stdout:Hit:9 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2024-03-04T15:34:05.898 INFO:teuthology.orchestra.run.smithi117.stdout:Fetched 28.9 kB in 0s (77.8 kB/s) 2024-03-04T15:34:06.075 INFO:teuthology.orchestra.run.smithi151.stdout:Fetched 28.9 kB in 0s (61.1 kB/s) 2024-03-04T15:34:06.810 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-03-04T15:34:06.825 DEBUG:teuthology.orchestra.run.smithi117:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1826-g9ca0b5de-1jammy cephadm=19.0.0-1826-g9ca0b5de-1jammy ceph-mds=19.0.0-1826-g9ca0b5de-1jammy ceph-mgr=19.0.0-1826-g9ca0b5de-1jammy ceph-common=19.0.0-1826-g9ca0b5de-1jammy ceph-fuse=19.0.0-1826-g9ca0b5de-1jammy ceph-test=19.0.0-1826-g9ca0b5de-1jammy ceph-volume=19.0.0-1826-g9ca0b5de-1jammy radosgw=19.0.0-1826-g9ca0b5de-1jammy python3-rados=19.0.0-1826-g9ca0b5de-1jammy python3-rgw=19.0.0-1826-g9ca0b5de-1jammy python3-cephfs=19.0.0-1826-g9ca0b5de-1jammy python3-rbd=19.0.0-1826-g9ca0b5de-1jammy libcephfs2=19.0.0-1826-g9ca0b5de-1jammy libcephfs-dev=19.0.0-1826-g9ca0b5de-1jammy librados2=19.0.0-1826-g9ca0b5de-1jammy librbd1=19.0.0-1826-g9ca0b5de-1jammy rbd-fuse=19.0.0-1826-g9ca0b5de-1jammy 2024-03-04T15:34:06.876 INFO:teuthology.orchestra.run.smithi117.stdout:Reading package lists... 2024-03-04T15:34:06.958 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-03-04T15:34:06.974 DEBUG:teuthology.orchestra.run.smithi151:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=19.0.0-1826-g9ca0b5de-1jammy cephadm=19.0.0-1826-g9ca0b5de-1jammy ceph-mds=19.0.0-1826-g9ca0b5de-1jammy ceph-mgr=19.0.0-1826-g9ca0b5de-1jammy ceph-common=19.0.0-1826-g9ca0b5de-1jammy ceph-fuse=19.0.0-1826-g9ca0b5de-1jammy ceph-test=19.0.0-1826-g9ca0b5de-1jammy ceph-volume=19.0.0-1826-g9ca0b5de-1jammy radosgw=19.0.0-1826-g9ca0b5de-1jammy python3-rados=19.0.0-1826-g9ca0b5de-1jammy python3-rgw=19.0.0-1826-g9ca0b5de-1jammy python3-cephfs=19.0.0-1826-g9ca0b5de-1jammy python3-rbd=19.0.0-1826-g9ca0b5de-1jammy libcephfs2=19.0.0-1826-g9ca0b5de-1jammy libcephfs-dev=19.0.0-1826-g9ca0b5de-1jammy librados2=19.0.0-1826-g9ca0b5de-1jammy librbd1=19.0.0-1826-g9ca0b5de-1jammy rbd-fuse=19.0.0-1826-g9ca0b5de-1jammy 2024-03-04T15:34:07.035 INFO:teuthology.orchestra.run.smithi151.stdout:Reading package lists... 2024-03-04T15:34:07.053 INFO:teuthology.orchestra.run.smithi117.stdout:Building dependency tree... 2024-03-04T15:34:07.054 INFO:teuthology.orchestra.run.smithi117.stdout:Reading state information... 2024-03-04T15:34:07.213 INFO:teuthology.orchestra.run.smithi117.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T15:34:07.213 INFO:teuthology.orchestra.run.smithi117.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T15:34:07.213 INFO:teuthology.orchestra.run.smithi117.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T15:34:07.214 INFO:teuthology.orchestra.run.smithi117.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T15:34:07.214 INFO:teuthology.orchestra.run.smithi117.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-04T15:34:07.214 INFO:teuthology.orchestra.run.smithi117.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-04T15:34:07.214 INFO:teuthology.orchestra.run.smithi117.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T15:34:07.215 INFO:teuthology.orchestra.run.smithi117.stdout:The following additional packages will be installed: 2024-03-04T15:34:07.215 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-04T15:34:07.215 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-04T15:34:07.215 INFO:teuthology.orchestra.run.smithi117.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-04T15:34:07.215 INFO:teuthology.orchestra.run.smithi117.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-04T15:34:07.216 INFO:teuthology.orchestra.run.smithi117.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-04T15:34:07.216 INFO:teuthology.orchestra.run.smithi117.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-04T15:34:07.216 INFO:teuthology.orchestra.run.smithi117.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-04T15:34:07.216 INFO:teuthology.orchestra.run.smithi117.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-04T15:34:07.216 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-04T15:34:07.216 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-03-04T15:34:07.216 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-04T15:34:07.216 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-04T15:34:07.216 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-04T15:34:07.216 INFO:teuthology.orchestra.run.smithi117.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-04T15:34:07.216 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-04T15:34:07.216 INFO:teuthology.orchestra.run.smithi117.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-04T15:34:07.217 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-03-04T15:34:07.217 INFO:teuthology.orchestra.run.smithi117.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-03-04T15:34:07.217 INFO:teuthology.orchestra.run.smithi117.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-04T15:34:07.217 INFO:teuthology.orchestra.run.smithi117.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-04T15:34:07.217 INFO:teuthology.orchestra.run.smithi117.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-04T15:34:07.217 INFO:teuthology.orchestra.run.smithi117.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-04T15:34:07.217 INFO:teuthology.orchestra.run.smithi117.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-03-04T15:34:07.217 INFO:teuthology.orchestra.run.smithi117.stdout: xmlstarlet zip 2024-03-04T15:34:07.217 INFO:teuthology.orchestra.run.smithi117.stdout:Suggested packages: 2024-03-04T15:34:07.217 INFO:teuthology.orchestra.run.smithi117.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-04T15:34:07.217 INFO:teuthology.orchestra.run.smithi117.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-04T15:34:07.217 INFO:teuthology.orchestra.run.smithi117.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-04T15:34:07.218 INFO:teuthology.orchestra.run.smithi117.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-04T15:34:07.218 INFO:teuthology.orchestra.run.smithi117.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-04T15:34:07.218 INFO:teuthology.orchestra.run.smithi117.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-04T15:34:07.218 INFO:teuthology.orchestra.run.smithi117.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-04T15:34:07.218 INFO:teuthology.orchestra.run.smithi117.stdout:Recommended packages: 2024-03-04T15:34:07.218 INFO:teuthology.orchestra.run.smithi117.stdout: btrfs-tools 2024-03-04T15:34:07.227 INFO:teuthology.orchestra.run.smithi151.stdout:Building dependency tree... 2024-03-04T15:34:07.228 INFO:teuthology.orchestra.run.smithi151.stdout:Reading state information... 2024-03-04T15:34:07.317 INFO:teuthology.orchestra.run.smithi117.stdout:The following NEW packages will be installed: 2024-03-04T15:34:07.317 INFO:teuthology.orchestra.run.smithi117.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-04T15:34:07.317 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-04T15:34:07.317 INFO:teuthology.orchestra.run.smithi117.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-03-04T15:34:07.317 INFO:teuthology.orchestra.run.smithi117.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-03-04T15:34:07.317 INFO:teuthology.orchestra.run.smithi117.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-03-04T15:34:07.317 INFO:teuthology.orchestra.run.smithi117.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-03-04T15:34:07.318 INFO:teuthology.orchestra.run.smithi117.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-03-04T15:34:07.318 INFO:teuthology.orchestra.run.smithi117.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-03-04T15:34:07.319 INFO:teuthology.orchestra.run.smithi117.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-03-04T15:34:07.319 INFO:teuthology.orchestra.run.smithi117.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-03-04T15:34:07.319 INFO:teuthology.orchestra.run.smithi117.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-03-04T15:34:07.319 INFO:teuthology.orchestra.run.smithi117.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-03-04T15:34:07.319 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-04T15:34:07.319 INFO:teuthology.orchestra.run.smithi117.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-04T15:34:07.319 INFO:teuthology.orchestra.run.smithi117.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-04T15:34:07.319 INFO:teuthology.orchestra.run.smithi117.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-04T15:34:07.319 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-04T15:34:07.319 INFO:teuthology.orchestra.run.smithi117.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-04T15:34:07.319 INFO:teuthology.orchestra.run.smithi117.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-03-04T15:34:07.319 INFO:teuthology.orchestra.run.smithi117.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-03-04T15:34:07.319 INFO:teuthology.orchestra.run.smithi117.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-03-04T15:34:07.319 INFO:teuthology.orchestra.run.smithi117.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-03-04T15:34:07.319 INFO:teuthology.orchestra.run.smithi117.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-03-04T15:34:07.319 INFO:teuthology.orchestra.run.smithi117.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-03-04T15:34:07.319 INFO:teuthology.orchestra.run.smithi117.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-03-04T15:34:07.320 INFO:teuthology.orchestra.run.smithi117.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-03-04T15:34:07.383 INFO:teuthology.orchestra.run.smithi117.stdout:0 upgraded, 118 newly installed, 0 to remove and 275 not upgraded. 2024-03-04T15:34:07.383 INFO:teuthology.orchestra.run.smithi117.stdout:Need to get 176 MB of archives. 2024-03-04T15:34:07.383 INFO:teuthology.orchestra.run.smithi117.stdout:After this operation, 730 MB of additional disk space will be used. 2024-03-04T15:34:07.383 INFO:teuthology.orchestra.run.smithi117.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-04T15:34:07.392 INFO:teuthology.orchestra.run.smithi151.stdout:The following packages were automatically installed and are no longer required: 2024-03-04T15:34:07.392 INFO:teuthology.orchestra.run.smithi151.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2024-03-04T15:34:07.392 INFO:teuthology.orchestra.run.smithi151.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2024-03-04T15:34:07.392 INFO:teuthology.orchestra.run.smithi151.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2024-03-04T15:34:07.393 INFO:teuthology.orchestra.run.smithi151.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2024-03-04T15:34:07.393 INFO:teuthology.orchestra.run.smithi151.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2024-03-04T15:34:07.393 INFO:teuthology.orchestra.run.smithi151.stdout:Use 'sudo apt autoremove' to remove them. 2024-03-04T15:34:07.394 INFO:teuthology.orchestra.run.smithi151.stdout:The following additional packages will be installed: 2024-03-04T15:34:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2024-03-04T15:34:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2024-03-04T15:34:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: libdouble-conversion3 libjq1 liblttng-ust1 liblua5.3-dev liboath0 libonig5 2024-03-04T15:34:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2024-03-04T15:34:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: librdkafka1 libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 2024-03-04T15:34:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: lua-any lua-sec lua-socket lua5.1 luarocks nvme-cli pkg-config 2024-03-04T15:34:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2024-03-04T15:34:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2024-03-04T15:34:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2024-03-04T15:34:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-03-04T15:34:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-04T15:34:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-04T15:34:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-04T15:34:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-04T15:34:07.395 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-04T15:34:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-04T15:34:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pygments python3-pyinotify python3-pytest python3-repoze.lru 2024-03-04T15:34:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: python3-requests python3-requests-oauthlib python3-routes python3-rsa 2024-03-04T15:34:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2024-03-04T15:34:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2024-03-04T15:34:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2024-03-04T15:34:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2024-03-04T15:34:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n socat unzip 2024-03-04T15:34:07.396 INFO:teuthology.orchestra.run.smithi151.stdout: xmlstarlet zip 2024-03-04T15:34:07.396 INFO:teuthology.orchestra.run.smithi151.stdout:Suggested packages: 2024-03-04T15:34:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: python3-influxdb readline-doc python-jinja2-doc python3-beaker 2024-03-04T15:34:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: python-mako-doc python-natsort-doc python-openssl-doc python3-openssl-dbg 2024-03-04T15:34:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: httpd-wsgi libapache2-mod-python libapache2-mod-scgi libjs-mochikit 2024-03-04T15:34:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: python-pecan-doc python-psutil-doc subversion python-pygments-doc 2024-03-04T15:34:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: python-pyinotify-doc python3-socks python-requests-doc python3-dap 2024-03-04T15:34:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: python-sklearn-doc ipython3 python-waitress-doc python-webob-doc 2024-03-04T15:34:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: python-webtest-doc python-werkzeug-doc python3-watchdog 2024-03-04T15:34:07.397 INFO:teuthology.orchestra.run.smithi151.stdout:Recommended packages: 2024-03-04T15:34:07.397 INFO:teuthology.orchestra.run.smithi151.stdout: btrfs-tools 2024-03-04T15:34:07.485 INFO:teuthology.orchestra.run.smithi117.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-04T15:34:07.488 INFO:teuthology.orchestra.run.smithi117.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-04T15:34:07.493 INFO:teuthology.orchestra.run.smithi151.stdout:The following NEW packages will be installed: 2024-03-04T15:34:07.493 INFO:teuthology.orchestra.run.smithi151.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2024-03-04T15:34:07.494 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2024-03-04T15:34:07.494 INFO:teuthology.orchestra.run.smithi151.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm jq 2024-03-04T15:34:07.494 INFO:teuthology.orchestra.run.smithi151.stdout: libcephfs-dev libcephfs2 libdouble-conversion3 libjq1 liblttng-ust1 2024-03-04T15:34:07.494 INFO:teuthology.orchestra.run.smithi151.stdout: liblua5.3-dev liboath0 libonig5 libpcre2-16-0 libqt5core5a libqt5dbus5 2024-03-04T15:34:07.494 INFO:teuthology.orchestra.run.smithi151.stdout: libqt5network5 librados2 libradosstriper1 librbd1 librdkafka1 2024-03-04T15:34:07.494 INFO:teuthology.orchestra.run.smithi151.stdout: libreadline-dev librgw2 libsqlite3-mod-ceph libthrift-0.16.0 lua-any lua-sec 2024-03-04T15:34:07.495 INFO:teuthology.orchestra.run.smithi151.stdout: lua-socket lua5.1 luarocks nvme-cli pkg-config python-asyncssh-doc 2024-03-04T15:34:07.495 INFO:teuthology.orchestra.run.smithi151.stdout: python-babel-localedata python-pastedeploy-tpl python3-asyncssh 2024-03-04T15:34:07.495 INFO:teuthology.orchestra.run.smithi151.stdout: python3-babel python3-bcrypt python3-cachetools python3-ceph-argparse 2024-03-04T15:34:07.495 INFO:teuthology.orchestra.run.smithi151.stdout: python3-ceph-common python3-cephfs python3-certifi python3-cheroot 2024-03-04T15:34:07.495 INFO:teuthology.orchestra.run.smithi151.stdout: python3-cherrypy3 python3-google-auth python3-idna python3-iniconfig 2024-03-04T15:34:07.495 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2024-03-04T15:34:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2024-03-04T15:34:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2024-03-04T15:34:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2024-03-04T15:34:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2024-03-04T15:34:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2024-03-04T15:34:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: python3-pygments python3-pyinotify python3-pytest python3-rados python3-rbd 2024-03-04T15:34:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: python3-repoze.lru python3-requests python3-requests-oauthlib python3-rgw 2024-03-04T15:34:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: python3-routes python3-rsa python3-simplegeneric python3-simplejson 2024-03-04T15:34:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: python3-singledispatch python3-sklearn python3-sklearn-lib python3-tempita 2024-03-04T15:34:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: python3-tempora python3-threadpoolctl python3-toml python3-urllib3 2024-03-04T15:34:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: python3-waitress python3-wcwidth python3-webob python3-websocket 2024-03-04T15:34:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: python3-webtest python3-werkzeug python3-zc.lockfile qttranslations5-l10n 2024-03-04T15:34:07.496 INFO:teuthology.orchestra.run.smithi151.stdout: radosgw rbd-fuse socat unzip xmlstarlet zip 2024-03-04T15:34:07.507 INFO:teuthology.orchestra.run.smithi117.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-04T15:34:07.539 INFO:teuthology.orchestra.run.smithi117.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1826-g9ca0b5de-1jammy [3,546 kB] 2024-03-04T15:34:07.560 INFO:teuthology.orchestra.run.smithi151.stdout:0 upgraded, 118 newly installed, 0 to remove and 275 not upgraded. 2024-03-04T15:34:07.560 INFO:teuthology.orchestra.run.smithi151.stdout:Need to get 176 MB of archives. 2024-03-04T15:34:07.560 INFO:teuthology.orchestra.run.smithi151.stdout:After this operation, 730 MB of additional disk space will be used. 2024-03-04T15:34:07.560 INFO:teuthology.orchestra.run.smithi151.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2024-03-04T15:34:07.570 INFO:teuthology.orchestra.run.smithi117.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-04T15:34:07.575 INFO:teuthology.orchestra.run.smithi117.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-04T15:34:07.587 INFO:teuthology.orchestra.run.smithi117.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-04T15:34:07.591 INFO:teuthology.orchestra.run.smithi117.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-04T15:34:07.591 INFO:teuthology.orchestra.run.smithi117.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-04T15:34:07.592 INFO:teuthology.orchestra.run.smithi117.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-04T15:34:07.601 INFO:teuthology.orchestra.run.smithi117.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-04T15:34:07.603 INFO:teuthology.orchestra.run.smithi117.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-04T15:34:07.605 INFO:teuthology.orchestra.run.smithi117.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-04T15:34:07.607 INFO:teuthology.orchestra.run.smithi117.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-04T15:34:07.613 INFO:teuthology.orchestra.run.smithi117.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-04T15:34:07.616 INFO:teuthology.orchestra.run.smithi117.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-04T15:34:07.618 INFO:teuthology.orchestra.run.smithi117.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-04T15:34:07.633 INFO:teuthology.orchestra.run.smithi117.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-04T15:34:07.634 INFO:teuthology.orchestra.run.smithi117.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-04T15:34:07.634 INFO:teuthology.orchestra.run.smithi117.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-04T15:34:07.635 INFO:teuthology.orchestra.run.smithi117.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-04T15:34:07.636 INFO:teuthology.orchestra.run.smithi117.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-04T15:34:07.636 INFO:teuthology.orchestra.run.smithi117.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-04T15:34:07.636 INFO:teuthology.orchestra.run.smithi117.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-04T15:34:07.652 INFO:teuthology.orchestra.run.smithi117.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-04T15:34:07.653 INFO:teuthology.orchestra.run.smithi117.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-04T15:34:07.653 INFO:teuthology.orchestra.run.smithi117.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-04T15:34:07.672 INFO:teuthology.orchestra.run.smithi117.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-04T15:34:07.676 INFO:teuthology.orchestra.run.smithi117.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-04T15:34:07.676 INFO:teuthology.orchestra.run.smithi117.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-04T15:34:07.676 INFO:teuthology.orchestra.run.smithi117.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-04T15:34:07.676 INFO:teuthology.orchestra.run.smithi117.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-04T15:34:07.676 INFO:teuthology.orchestra.run.smithi117.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-04T15:34:07.676 INFO:teuthology.orchestra.run.smithi117.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-04T15:34:07.676 INFO:teuthology.orchestra.run.smithi151.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2024-03-04T15:34:07.677 INFO:teuthology.orchestra.run.smithi151.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2024-03-04T15:34:07.691 INFO:teuthology.orchestra.run.smithi117.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-04T15:34:07.692 INFO:teuthology.orchestra.run.smithi117.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-04T15:34:07.692 INFO:teuthology.orchestra.run.smithi117.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-04T15:34:07.698 INFO:teuthology.orchestra.run.smithi151.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2024-03-04T15:34:07.711 INFO:teuthology.orchestra.run.smithi151.stdout:Get:5 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 19.0.0-1826-g9ca0b5de-1jammy [3,546 kB] 2024-03-04T15:34:07.711 INFO:teuthology.orchestra.run.smithi117.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-04T15:34:07.711 INFO:teuthology.orchestra.run.smithi117.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-04T15:34:07.712 INFO:teuthology.orchestra.run.smithi117.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-04T15:34:07.713 INFO:teuthology.orchestra.run.smithi117.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-04T15:34:07.714 INFO:teuthology.orchestra.run.smithi117.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-04T15:34:07.716 INFO:teuthology.orchestra.run.smithi117.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-04T15:34:07.716 INFO:teuthology.orchestra.run.smithi117.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-04T15:34:07.730 INFO:teuthology.orchestra.run.smithi117.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-04T15:34:07.731 INFO:teuthology.orchestra.run.smithi117.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-04T15:34:07.733 INFO:teuthology.orchestra.run.smithi117.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-04T15:34:07.750 INFO:teuthology.orchestra.run.smithi117.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-04T15:34:07.751 INFO:teuthology.orchestra.run.smithi117.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-04T15:34:07.753 INFO:teuthology.orchestra.run.smithi117.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-04T15:34:07.784 INFO:teuthology.orchestra.run.smithi151.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2024-03-04T15:34:07.789 INFO:teuthology.orchestra.run.smithi151.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2024-03-04T15:34:07.805 INFO:teuthology.orchestra.run.smithi151.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2024-03-04T15:34:07.811 INFO:teuthology.orchestra.run.smithi151.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2024-03-04T15:34:07.811 INFO:teuthology.orchestra.run.smithi151.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2024-03-04T15:34:07.812 INFO:teuthology.orchestra.run.smithi151.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2024-03-04T15:34:07.829 INFO:teuthology.orchestra.run.smithi117.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-04T15:34:07.829 INFO:teuthology.orchestra.run.smithi151.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libreadline-dev amd64 8.1.2-1 [166 kB] 2024-03-04T15:34:07.830 INFO:teuthology.orchestra.run.smithi117.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-04T15:34:07.832 INFO:teuthology.orchestra.run.smithi117.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-04T15:34:07.832 INFO:teuthology.orchestra.run.smithi151.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblua5.3-dev amd64 5.3.6-1build1 [167 kB] 2024-03-04T15:34:07.832 INFO:teuthology.orchestra.run.smithi117.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-04T15:34:07.834 INFO:teuthology.orchestra.run.smithi117.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-04T15:34:07.841 INFO:teuthology.orchestra.run.smithi151.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua5.1 amd64 5.1.5-8.1build4 [94.6 kB] 2024-03-04T15:34:07.843 INFO:teuthology.orchestra.run.smithi151.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-any all 27ubuntu1 [5,034 B] 2024-03-04T15:34:07.858 INFO:teuthology.orchestra.run.smithi117.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-04T15:34:07.860 INFO:teuthology.orchestra.run.smithi117.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-04T15:34:07.860 INFO:teuthology.orchestra.run.smithi117.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-04T15:34:07.875 INFO:teuthology.orchestra.run.smithi151.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 zip amd64 3.0-12build2 [176 kB] 2024-03-04T15:34:07.878 INFO:teuthology.orchestra.run.smithi151.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 unzip amd64 6.0-26ubuntu3.2 [175 kB] 2024-03-04T15:34:07.881 INFO:teuthology.orchestra.run.smithi117.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-04T15:34:07.881 INFO:teuthology.orchestra.run.smithi151.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/universe amd64 luarocks all 3.8.0+dfsg1-1 [140 kB] 2024-03-04T15:34:07.882 INFO:teuthology.orchestra.run.smithi117.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-04T15:34:07.882 INFO:teuthology.orchestra.run.smithi117.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-04T15:34:07.883 INFO:teuthology.orchestra.run.smithi117.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-04T15:34:07.884 INFO:teuthology.orchestra.run.smithi151.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2024-03-04T15:34:07.885 INFO:teuthology.orchestra.run.smithi117.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-04T15:34:07.885 INFO:teuthology.orchestra.run.smithi151.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2024-03-04T15:34:07.885 INFO:teuthology.orchestra.run.smithi151.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2024-03-04T15:34:07.885 INFO:teuthology.orchestra.run.smithi117.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-04T15:34:07.886 INFO:teuthology.orchestra.run.smithi151.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2024-03-04T15:34:07.894 INFO:teuthology.orchestra.run.smithi117.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-04T15:34:07.894 INFO:teuthology.orchestra.run.smithi117.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-04T15:34:07.897 INFO:teuthology.orchestra.run.smithi117.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-04T15:34:07.898 INFO:teuthology.orchestra.run.smithi151.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2024-03-04T15:34:07.899 INFO:teuthology.orchestra.run.smithi151.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2024-03-04T15:34:07.899 INFO:teuthology.orchestra.run.smithi151.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2024-03-04T15:34:07.913 INFO:teuthology.orchestra.run.smithi117.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-04T15:34:07.914 INFO:teuthology.orchestra.run.smithi117.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-04T15:34:07.915 INFO:teuthology.orchestra.run.smithi117.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-04T15:34:07.918 INFO:teuthology.orchestra.run.smithi117.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-04T15:34:07.921 INFO:teuthology.orchestra.run.smithi117.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-04T15:34:07.922 INFO:teuthology.orchestra.run.smithi117.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-04T15:34:07.922 INFO:teuthology.orchestra.run.smithi117.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-04T15:34:07.926 INFO:teuthology.orchestra.run.smithi151.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2024-03-04T15:34:07.927 INFO:teuthology.orchestra.run.smithi151.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2024-03-04T15:34:07.927 INFO:teuthology.orchestra.run.smithi151.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2024-03-04T15:34:07.933 INFO:teuthology.orchestra.run.smithi117.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-04T15:34:07.933 INFO:teuthology.orchestra.run.smithi151.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2024-03-04T15:34:07.933 INFO:teuthology.orchestra.run.smithi117.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-04T15:34:07.934 INFO:teuthology.orchestra.run.smithi151.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2024-03-04T15:34:07.935 INFO:teuthology.orchestra.run.smithi151.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2024-03-04T15:34:07.936 INFO:teuthology.orchestra.run.smithi151.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2024-03-04T15:34:07.936 INFO:teuthology.orchestra.run.smithi117.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-04T15:34:07.947 INFO:teuthology.orchestra.run.smithi151.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2024-03-04T15:34:07.947 INFO:teuthology.orchestra.run.smithi151.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2024-03-04T15:34:07.948 INFO:teuthology.orchestra.run.smithi151.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2024-03-04T15:34:07.952 INFO:teuthology.orchestra.run.smithi117.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-04T15:34:07.953 INFO:teuthology.orchestra.run.smithi117.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-04T15:34:07.953 INFO:teuthology.orchestra.run.smithi117.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-04T15:34:07.955 INFO:teuthology.orchestra.run.smithi117.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-04T15:34:07.955 INFO:teuthology.orchestra.run.smithi117.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-04T15:34:07.962 INFO:teuthology.orchestra.run.smithi117.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-04T15:34:07.962 INFO:teuthology.orchestra.run.smithi117.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-04T15:34:07.968 INFO:teuthology.orchestra.run.smithi151.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2024-03-04T15:34:07.969 INFO:teuthology.orchestra.run.smithi151.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2024-03-04T15:34:07.970 INFO:teuthology.orchestra.run.smithi151.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2024-03-04T15:34:07.973 INFO:teuthology.orchestra.run.smithi117.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-04T15:34:07.976 INFO:teuthology.orchestra.run.smithi117.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-04T15:34:07.977 INFO:teuthology.orchestra.run.smithi117.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-04T15:34:07.981 INFO:teuthology.orchestra.run.smithi151.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2024-03-04T15:34:07.981 INFO:teuthology.orchestra.run.smithi151.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2024-03-04T15:34:07.982 INFO:teuthology.orchestra.run.smithi151.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2024-03-04T15:34:07.983 INFO:teuthology.orchestra.run.smithi151.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2024-03-04T15:34:07.999 INFO:teuthology.orchestra.run.smithi151.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2024-03-04T15:34:08.003 INFO:teuthology.orchestra.run.smithi151.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2024-03-04T15:34:08.004 INFO:teuthology.orchestra.run.smithi151.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2024-03-04T15:34:08.020 INFO:teuthology.orchestra.run.smithi151.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2024-03-04T15:34:08.021 INFO:teuthology.orchestra.run.smithi151.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2024-03-04T15:34:08.026 INFO:teuthology.orchestra.run.smithi151.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2024-03-04T15:34:08.027 INFO:teuthology.orchestra.run.smithi151.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2024-03-04T15:34:08.028 INFO:teuthology.orchestra.run.smithi151.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2024-03-04T15:34:08.033 INFO:teuthology.orchestra.run.smithi151.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2024-03-04T15:34:08.105 INFO:teuthology.orchestra.run.smithi151.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2024-03-04T15:34:08.106 INFO:teuthology.orchestra.run.smithi151.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-jinja2 all 3.0.3-1ubuntu0.1 [108 kB] 2024-03-04T15:34:08.107 INFO:teuthology.orchestra.run.smithi151.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2024-03-04T15:34:08.107 INFO:teuthology.orchestra.run.smithi151.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2024-03-04T15:34:08.108 INFO:teuthology.orchestra.run.smithi151.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2024-03-04T15:34:08.126 INFO:teuthology.orchestra.run.smithi151.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2024-03-04T15:34:08.128 INFO:teuthology.orchestra.run.smithi151.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2024-03-04T15:34:08.128 INFO:teuthology.orchestra.run.smithi151.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2024-03-04T15:34:08.142 INFO:teuthology.orchestra.run.smithi151.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2024-03-04T15:34:08.143 INFO:teuthology.orchestra.run.smithi151.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2024-03-04T15:34:08.153 INFO:teuthology.orchestra.run.smithi151.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2024-03-04T15:34:08.153 INFO:teuthology.orchestra.run.smithi151.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2024-03-04T15:34:08.153 INFO:teuthology.orchestra.run.smithi151.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2024-03-04T15:34:08.154 INFO:teuthology.orchestra.run.smithi151.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2024-03-04T15:34:08.154 INFO:teuthology.orchestra.run.smithi151.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2024-03-04T15:34:08.155 INFO:teuthology.orchestra.run.smithi151.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2024-03-04T15:34:08.159 INFO:teuthology.orchestra.run.smithi151.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2024-03-04T15:34:08.173 INFO:teuthology.orchestra.run.smithi151.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2024-03-04T15:34:08.175 INFO:teuthology.orchestra.run.smithi151.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2024-03-04T15:34:08.175 INFO:teuthology.orchestra.run.smithi151.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2024-03-04T15:34:08.194 INFO:teuthology.orchestra.run.smithi151.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2024-03-04T15:34:08.197 INFO:teuthology.orchestra.run.smithi151.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-socket amd64 3.0~rc1+git+ac3201d-6 [78.9 kB] 2024-03-04T15:34:08.198 INFO:teuthology.orchestra.run.smithi151.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 lua-sec amd64 1.0.2-1 [37.6 kB] 2024-03-04T15:34:08.199 INFO:teuthology.orchestra.run.smithi151.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2024-03-04T15:34:08.205 INFO:teuthology.orchestra.run.smithi151.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 pkg-config amd64 0.29.2-1ubuntu3 [48.2 kB] 2024-03-04T15:34:08.205 INFO:teuthology.orchestra.run.smithi151.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2024-03-04T15:34:08.209 INFO:teuthology.orchestra.run.smithi151.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2024-03-04T15:34:08.214 INFO:teuthology.orchestra.run.smithi151.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2024-03-04T15:34:08.215 INFO:teuthology.orchestra.run.smithi151.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2024-03-04T15:34:08.215 INFO:teuthology.orchestra.run.smithi151.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2024-03-04T15:34:08.235 INFO:teuthology.orchestra.run.smithi151.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2024-03-04T15:34:08.236 INFO:teuthology.orchestra.run.smithi151.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2024-03-04T15:34:08.254 INFO:teuthology.orchestra.run.smithi151.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2024-03-04T15:34:08.255 INFO:teuthology.orchestra.run.smithi151.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2024-03-04T15:34:08.255 INFO:teuthology.orchestra.run.smithi151.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2024-03-04T15:34:08.258 INFO:teuthology.orchestra.run.smithi151.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2024-03-04T15:34:08.259 INFO:teuthology.orchestra.run.smithi151.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2024-03-04T15:34:08.524 INFO:teuthology.orchestra.run.smithi117.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1826-g9ca0b5de-1jammy [3,220 kB] 2024-03-04T15:34:08.677 INFO:teuthology.orchestra.run.smithi151.stdout:Get:89 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 19.0.0-1826-g9ca0b5de-1jammy [3,220 kB] 2024-03-04T15:34:08.853 INFO:teuthology.orchestra.run.smithi117.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1826-g9ca0b5de-1jammy [709 kB] 2024-03-04T15:34:08.922 INFO:teuthology.orchestra.run.smithi117.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1826-g9ca0b5de-1jammy [356 kB] 2024-03-04T15:34:08.956 INFO:teuthology.orchestra.run.smithi117.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1826-g9ca0b5de-1jammy [32.8 kB] 2024-03-04T15:34:08.957 INFO:teuthology.orchestra.run.smithi117.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1826-g9ca0b5de-1jammy [180 kB] 2024-03-04T15:34:08.961 INFO:teuthology.orchestra.run.smithi117.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1826-g9ca0b5de-1jammy [64.5 kB] 2024-03-04T15:34:08.963 INFO:teuthology.orchestra.run.smithi117.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1826-g9ca0b5de-1jammy [330 kB] 2024-03-04T15:34:08.995 INFO:teuthology.orchestra.run.smithi117.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1826-g9ca0b5de-1jammy [6,572 kB] 2024-03-04T15:34:09.009 INFO:teuthology.orchestra.run.smithi151.stdout:Get:90 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 19.0.0-1826-g9ca0b5de-1jammy [709 kB] 2024-03-04T15:34:09.076 INFO:teuthology.orchestra.run.smithi151.stdout:Get:91 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 19.0.0-1826-g9ca0b5de-1jammy [356 kB] 2024-03-04T15:34:09.109 INFO:teuthology.orchestra.run.smithi151.stdout:Get:92 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 19.0.0-1826-g9ca0b5de-1jammy [32.8 kB] 2024-03-04T15:34:09.111 INFO:teuthology.orchestra.run.smithi151.stdout:Get:93 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 19.0.0-1826-g9ca0b5de-1jammy [180 kB] 2024-03-04T15:34:09.116 INFO:teuthology.orchestra.run.smithi151.stdout:Get:94 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 19.0.0-1826-g9ca0b5de-1jammy [64.5 kB] 2024-03-04T15:34:09.139 INFO:teuthology.orchestra.run.smithi151.stdout:Get:95 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 19.0.0-1826-g9ca0b5de-1jammy [330 kB] 2024-03-04T15:34:09.150 INFO:teuthology.orchestra.run.smithi151.stdout:Get:96 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 19.0.0-1826-g9ca0b5de-1jammy [6,572 kB] 2024-03-04T15:34:09.389 INFO:teuthology.orchestra.run.smithi117.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1826-g9ca0b5de-1jammy [112 kB] 2024-03-04T15:34:09.414 INFO:teuthology.orchestra.run.smithi117.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1826-g9ca0b5de-1jammy [467 kB] 2024-03-04T15:34:09.427 INFO:teuthology.orchestra.run.smithi117.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1826-g9ca0b5de-1jammy [24.2 MB] 2024-03-04T15:34:09.654 INFO:teuthology.orchestra.run.smithi151.stdout:Get:97 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 19.0.0-1826-g9ca0b5de-1jammy [112 kB] 2024-03-04T15:34:09.660 INFO:teuthology.orchestra.run.smithi151.stdout:Get:98 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 19.0.0-1826-g9ca0b5de-1jammy [467 kB] 2024-03-04T15:34:09.674 INFO:teuthology.orchestra.run.smithi151.stdout:Get:99 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 19.0.0-1826-g9ca0b5de-1jammy [24.2 MB] 2024-03-04T15:34:10.992 INFO:teuthology.orchestra.run.smithi117.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1826-g9ca0b5de-1jammy [5,081 kB] 2024-03-04T15:34:11.215 INFO:teuthology.orchestra.run.smithi117.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1826-g9ca0b5de-1jammy [243 kB] 2024-03-04T15:34:11.231 INFO:teuthology.orchestra.run.smithi117.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1826-g9ca0b5de-1jammy [124 kB] 2024-03-04T15:34:11.235 INFO:teuthology.orchestra.run.smithi117.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1826-g9ca0b5de-1jammy [1,521 kB] 2024-03-04T15:34:11.326 INFO:teuthology.orchestra.run.smithi117.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1826-g9ca0b5de-1jammy [6,247 kB] 2024-03-04T15:34:11.683 INFO:teuthology.orchestra.run.smithi117.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1826-g9ca0b5de-1jammy [22.8 MB] 2024-03-04T15:34:11.727 INFO:teuthology.orchestra.run.smithi151.stdout:Get:100 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 19.0.0-1826-g9ca0b5de-1jammy [5,081 kB] 2024-03-04T15:34:12.032 INFO:teuthology.orchestra.run.smithi151.stdout:Get:101 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 19.0.0-1826-g9ca0b5de-1jammy [243 kB] 2024-03-04T15:34:12.062 INFO:teuthology.orchestra.run.smithi151.stdout:Get:102 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 19.0.0-1826-g9ca0b5de-1jammy [124 kB] 2024-03-04T15:34:12.066 INFO:teuthology.orchestra.run.smithi151.stdout:Get:103 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 19.0.0-1826-g9ca0b5de-1jammy [1,521 kB] 2024-03-04T15:34:12.178 INFO:teuthology.orchestra.run.smithi151.stdout:Get:104 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 19.0.0-1826-g9ca0b5de-1jammy [6,247 kB] 2024-03-04T15:34:12.663 INFO:teuthology.orchestra.run.smithi151.stdout:Get:105 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 19.0.0-1826-g9ca0b5de-1jammy [22.8 MB] 2024-03-04T15:34:13.085 INFO:teuthology.orchestra.run.smithi117.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1826-g9ca0b5de-1jammy [14.0 kB] 2024-03-04T15:34:13.086 INFO:teuthology.orchestra.run.smithi117.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1826-g9ca0b5de-1jammy [867 kB] 2024-03-04T15:34:13.103 INFO:teuthology.orchestra.run.smithi117.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1826-g9ca0b5de-1jammy [2,190 kB] 2024-03-04T15:34:13.182 INFO:teuthology.orchestra.run.smithi117.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1826-g9ca0b5de-1jammy [777 kB] 2024-03-04T15:34:13.230 INFO:teuthology.orchestra.run.smithi117.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1826-g9ca0b5de-1jammy [144 kB] 2024-03-04T15:34:13.235 INFO:teuthology.orchestra.run.smithi117.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1826-g9ca0b5de-1jammy [1,922 kB] 2024-03-04T15:34:13.343 INFO:teuthology.orchestra.run.smithi117.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1826-g9ca0b5de-1jammy [8,625 kB] 2024-03-04T15:34:13.856 INFO:teuthology.orchestra.run.smithi117.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1826-g9ca0b5de-1jammy [14.2 kB] 2024-03-04T15:34:13.856 INFO:teuthology.orchestra.run.smithi117.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1826-g9ca0b5de-1jammy [49.9 MB] 2024-03-04T15:34:14.468 INFO:teuthology.orchestra.run.smithi151.stdout:Get:106 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 19.0.0-1826-g9ca0b5de-1jammy [14.0 kB] 2024-03-04T15:34:14.468 INFO:teuthology.orchestra.run.smithi151.stdout:Get:107 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 19.0.0-1826-g9ca0b5de-1jammy [867 kB] 2024-03-04T15:34:14.485 INFO:teuthology.orchestra.run.smithi151.stdout:Get:108 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 19.0.0-1826-g9ca0b5de-1jammy [2,190 kB] 2024-03-04T15:34:14.614 INFO:teuthology.orchestra.run.smithi151.stdout:Get:109 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 19.0.0-1826-g9ca0b5de-1jammy [777 kB] 2024-03-04T15:34:14.670 INFO:teuthology.orchestra.run.smithi151.stdout:Get:110 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 19.0.0-1826-g9ca0b5de-1jammy [144 kB] 2024-03-04T15:34:14.674 INFO:teuthology.orchestra.run.smithi151.stdout:Get:111 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 19.0.0-1826-g9ca0b5de-1jammy [1,922 kB] 2024-03-04T15:34:14.830 INFO:teuthology.orchestra.run.smithi151.stdout:Get:112 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 19.0.0-1826-g9ca0b5de-1jammy [8,625 kB] 2024-03-04T15:34:15.484 INFO:teuthology.orchestra.run.smithi151.stdout:Get:113 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 19.0.0-1826-g9ca0b5de-1jammy [14.2 kB] 2024-03-04T15:34:15.485 INFO:teuthology.orchestra.run.smithi151.stdout:Get:114 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 19.0.0-1826-g9ca0b5de-1jammy [49.9 MB] 2024-03-04T15:34:16.960 INFO:teuthology.orchestra.run.smithi117.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1826-g9ca0b5de-1jammy [121 kB] 2024-03-04T15:34:16.963 INFO:teuthology.orchestra.run.smithi117.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1826-g9ca0b5de-1jammy [38.4 kB] 2024-03-04T15:34:16.964 INFO:teuthology.orchestra.run.smithi117.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1826-g9ca0b5de-1jammy [13.0 MB] 2024-03-04T15:34:17.597 INFO:teuthology.orchestra.run.smithi117.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1826-g9ca0b5de-1jammy [91.3 kB] 2024-03-04T15:34:18.151 INFO:teuthology.orchestra.run.smithi117.stdout:Fetched 176 MB in 10s (17.1 MB/s) 2024-03-04T15:34:18.315 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-04T15:34:18.414 INFO:teuthology.orchestra.run.smithi117.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165994 files and directories currently installed.) 2024-03-04T15:34:18.420 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-04T15:34:18.504 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-04T15:34:18.850 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-04T15:34:18.870 INFO:teuthology.orchestra.run.smithi151.stdout:Get:115 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 19.0.0-1826-g9ca0b5de-1jammy [121 kB] 2024-03-04T15:34:18.873 INFO:teuthology.orchestra.run.smithi151.stdout:Get:116 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 19.0.0-1826-g9ca0b5de-1jammy [38.4 kB] 2024-03-04T15:34:18.874 INFO:teuthology.orchestra.run.smithi151.stdout:Get:117 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 19.0.0-1826-g9ca0b5de-1jammy [13.0 MB] 2024-03-04T15:34:18.876 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-04T15:34:18.916 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-04T15:34:19.426 INFO:teuthology.orchestra.run.smithi151.stdout:Get:118 https://4.chacra.ceph.com/r/ceph/wip-asm0deuz-testing-2024-03-04-1211/9ca0b5de35ed230e84387abdd8e619ed8e4f3282/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 19.0.0-1826-g9ca0b5de-1jammy [91.3 kB] 2024-03-04T15:34:19.444 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-04T15:34:19.471 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-04T15:34:19.511 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-04T15:34:19.948 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-04T15:34:19.974 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T15:34:19.988 INFO:teuthology.orchestra.run.smithi151.stdout:Fetched 176 MB in 12s (14.8 MB/s) 2024-03-04T15:34:20.022 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T15:34:20.157 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2024-03-04T15:34:20.257 INFO:teuthology.orchestra.run.smithi151.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165994 files and directories currently installed.) 2024-03-04T15:34:20.262 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../000-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2024-03-04T15:34:20.347 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-04T15:34:20.376 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-04T15:34:20.402 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T15:34:20.434 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T15:34:20.643 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2024-03-04T15:34:20.658 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../001-libdouble-conversion3_3.1.7-4_amd64.deb ... 2024-03-04T15:34:20.693 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-04T15:34:20.770 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-04T15:34:20.797 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T15:34:20.837 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T15:34:21.055 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2024-03-04T15:34:21.081 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../002-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2024-03-04T15:34:21.121 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-04T15:34:21.156 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-04T15:34:21.183 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-04T15:34:21.223 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-04T15:34:21.520 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package librados2. 2024-03-04T15:34:21.534 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libqt5core5a:amd64. 2024-03-04T15:34:21.546 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../007-librados2_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:21.561 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../003-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T15:34:21.586 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librados2 (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:21.603 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T15:34:21.955 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2024-03-04T15:34:21.981 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../004-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T15:34:22.013 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T15:34:22.038 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package librbd1. 2024-03-04T15:34:22.054 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../008-librbd1_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:22.088 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librbd1 (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:22.359 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libqt5network5:amd64. 2024-03-04T15:34:22.386 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../005-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2024-03-04T15:34:22.425 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T15:34:22.491 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libcephfs2. 2024-03-04T15:34:22.506 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:22.541 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libcephfs2 (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:22.746 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2024-03-04T15:34:22.768 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-rados. 2024-03-04T15:34:22.772 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../006-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2024-03-04T15:34:22.784 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../010-python3-rados_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:22.804 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-04T15:34:22.827 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-rados (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:23.121 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-04T15:34:23.147 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-1826-g9ca0b5de-1jammy_all.deb ... 2024-03-04T15:34:23.188 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-ceph-argparse (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:23.333 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package librados2. 2024-03-04T15:34:23.359 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../007-librados2_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:23.399 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking librados2 (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:23.423 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cephfs. 2024-03-04T15:34:23.449 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:23.490 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cephfs (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:23.801 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-04T15:34:23.827 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-1826-g9ca0b5de-1jammy_all.deb ... 2024-03-04T15:34:23.844 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package librbd1. 2024-03-04T15:34:23.867 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-ceph-common (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:23.871 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../008-librbd1_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:23.903 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking librbd1 (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:24.276 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-04T15:34:24.303 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-04T15:34:24.331 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libcephfs2. 2024-03-04T15:34:24.357 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../009-libcephfs2_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:24.398 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libcephfs2 (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:24.474 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-04T15:34:24.634 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-rados. 2024-03-04T15:34:24.649 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../010-python3-rados_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:24.684 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-rados (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:24.782 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-prettytable. 2024-03-04T15:34:24.808 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-03-04T15:34:24.848 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-04T15:34:25.101 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-rbd. 2024-03-04T15:34:25.127 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:25.179 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-ceph-argparse. 2024-03-04T15:34:25.193 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-rbd (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:25.205 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../011-python3-ceph-argparse_19.0.0-1826-g9ca0b5de-1jammy_all.deb ... 2024-03-04T15:34:25.237 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-ceph-argparse (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:25.449 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-cephfs. 2024-03-04T15:34:25.464 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../012-python3-cephfs_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:25.507 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-cephfs (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:25.521 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-04T15:34:25.547 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-04T15:34:25.587 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-04T15:34:25.836 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-ceph-common. 2024-03-04T15:34:25.848 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-04T15:34:25.862 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../013-python3-ceph-common_19.0.0-1826-g9ca0b5de-1jammy_all.deb ... 2024-03-04T15:34:25.863 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-04T15:34:25.898 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-04T15:34:25.904 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-ceph-common (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:26.201 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-04T15:34:26.227 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-04T15:34:26.239 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-wcwidth. 2024-03-04T15:34:26.265 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../014-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2024-03-04T15:34:26.267 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-04T15:34:26.306 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-04T15:34:26.596 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package lua5.1. 2024-03-04T15:34:26.618 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-prettytable. 2024-03-04T15:34:26.623 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-04T15:34:26.644 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../015-python3-prettytable_2.5.0-2_all.deb ... 2024-03-04T15:34:26.662 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-04T15:34:26.685 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2024-03-04T15:34:26.946 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-rbd. 2024-03-04T15:34:26.965 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package lua-any. 2024-03-04T15:34:26.972 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../016-python3-rbd_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:26.980 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-03-04T15:34:27.013 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-rbd (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:27.015 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-04T15:34:27.217 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package zip. 2024-03-04T15:34:27.232 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-03-04T15:34:27.275 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking zip (3.0-12build2) ... 2024-03-04T15:34:27.375 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package librdkafka1:amd64. 2024-03-04T15:34:27.401 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../017-librdkafka1_1.8.0-1build1_amd64.deb ... 2024-03-04T15:34:27.441 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-04T15:34:27.579 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package unzip. 2024-03-04T15:34:27.605 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-04T15:34:27.645 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-04T15:34:27.694 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libreadline-dev:amd64. 2024-03-04T15:34:27.710 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../018-libreadline-dev_8.1.2-1_amd64.deb ... 2024-03-04T15:34:27.745 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libreadline-dev:amd64 (8.1.2-1) ... 2024-03-04T15:34:28.016 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package luarocks. 2024-03-04T15:34:28.042 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-04T15:34:28.048 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package liblua5.3-dev:amd64. 2024-03-04T15:34:28.063 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../019-liblua5.3-dev_5.3.6-1build1_amd64.deb ... 2024-03-04T15:34:28.082 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-04T15:34:28.106 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-04T15:34:28.440 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package librgw2. 2024-03-04T15:34:28.461 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package lua5.1. 2024-03-04T15:34:28.466 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../025-librgw2_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:28.487 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../020-lua5.1_5.1.5-8.1build4_amd64.deb ... 2024-03-04T15:34:28.506 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking librgw2 (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:28.527 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking lua5.1 (5.1.5-8.1build4) ... 2024-03-04T15:34:28.856 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package lua-any. 2024-03-04T15:34:28.871 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../021-lua-any_27ubuntu1_all.deb ... 2024-03-04T15:34:28.914 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking lua-any (27ubuntu1) ... 2024-03-04T15:34:28.992 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-rgw. 2024-03-04T15:34:29.007 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:29.042 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-rgw (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:29.142 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package zip. 2024-03-04T15:34:29.157 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../022-zip_3.0-12build2_amd64.deb ... 2024-03-04T15:34:29.209 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking zip (3.0-12build2) ... 2024-03-04T15:34:29.538 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package unzip. 2024-03-04T15:34:29.562 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-04T15:34:29.564 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../023-unzip_6.0-26ubuntu3.2_amd64.deb ... 2024-03-04T15:34:29.588 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-04T15:34:29.605 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking unzip (6.0-26ubuntu3.2) ... 2024-03-04T15:34:29.628 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-04T15:34:29.931 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libradosstriper1. 2024-03-04T15:34:29.957 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:29.997 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libradosstriper1 (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:30.151 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package luarocks. 2024-03-04T15:34:30.177 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../024-luarocks_3.8.0+dfsg1-1_all.deb ... 2024-03-04T15:34:30.218 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking luarocks (3.8.0+dfsg1-1) ... 2024-03-04T15:34:30.343 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-common. 2024-03-04T15:34:30.369 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../029-ceph-common_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:30.400 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-common (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:30.578 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package librgw2. 2024-03-04T15:34:30.604 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../025-librgw2_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:30.644 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking librgw2 (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:31.197 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-rgw. 2024-03-04T15:34:31.212 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../026-python3-rgw_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:31.247 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-rgw (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:31.550 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package liboath0:amd64. 2024-03-04T15:34:31.576 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../027-liboath0_2.6.7-3build1_amd64.deb ... 2024-03-04T15:34:31.606 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-base. 2024-03-04T15:34:31.617 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2024-03-04T15:34:31.632 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../030-ceph-base_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:31.673 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-base (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:31.938 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libradosstriper1. 2024-03-04T15:34:31.964 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../028-libradosstriper1_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:32.004 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libradosstriper1 (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:32.185 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-04T15:34:32.212 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-04T15:34:32.260 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-04T15:34:32.366 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-common. 2024-03-04T15:34:32.393 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../029-ceph-common_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:32.424 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-common (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:32.563 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cheroot. 2024-03-04T15:34:32.589 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-04T15:34:32.629 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-04T15:34:32.916 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-04T15:34:32.942 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-04T15:34:32.982 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-04T15:34:33.285 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-04T15:34:33.311 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-04T15:34:33.368 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-04T15:34:33.688 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-04T15:34:33.715 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-04T15:34:33.754 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-04T15:34:33.873 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-base. 2024-03-04T15:34:33.899 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../030-ceph-base_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:33.949 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-base (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:34.015 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-tempora. 2024-03-04T15:34:34.042 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-03-04T15:34:34.074 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-04T15:34:34.343 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-portend. 2024-03-04T15:34:34.370 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-03-04T15:34:34.409 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-04T15:34:34.528 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-jaraco.functools. 2024-03-04T15:34:34.555 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../031-python3-jaraco.functools_3.4.0-2_all.deb ... 2024-03-04T15:34:34.595 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2024-03-04T15:34:34.721 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-04T15:34:34.748 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-04T15:34:34.787 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-04T15:34:34.890 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-cheroot. 2024-03-04T15:34:34.916 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../032-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2024-03-04T15:34:34.957 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-04T15:34:35.065 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-04T15:34:35.092 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-04T15:34:35.132 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-04T15:34:35.393 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-jaraco.classes. 2024-03-04T15:34:35.419 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../033-python3-jaraco.classes_3.2.1-3_all.deb ... 2024-03-04T15:34:35.478 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-natsort. 2024-03-04T15:34:35.502 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2024-03-04T15:34:35.504 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-03-04T15:34:35.544 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-04T15:34:35.831 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-jaraco.text. 2024-03-04T15:34:35.855 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-logutils. 2024-03-04T15:34:35.857 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../034-python3-jaraco.text_3.6.0-2_all.deb ... 2024-03-04T15:34:35.882 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-03-04T15:34:35.898 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2024-03-04T15:34:35.922 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-04T15:34:36.176 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-jaraco.collections. 2024-03-04T15:34:36.202 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../035-python3-jaraco.collections_3.4.0-2_all.deb ... 2024-03-04T15:34:36.208 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-04T15:34:36.234 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-04T15:34:36.243 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2024-03-04T15:34:36.274 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-04T15:34:36.544 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-mako. 2024-03-04T15:34:36.571 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-04T15:34:36.571 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-tempora. 2024-03-04T15:34:36.587 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../036-python3-tempora_4.1.2-1_all.deb ... 2024-03-04T15:34:36.602 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-04T15:34:36.630 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-tempora (4.1.2-1) ... 2024-03-04T15:34:36.891 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-portend. 2024-03-04T15:34:36.905 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-04T15:34:36.914 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../037-python3-portend_3.0.0-1_all.deb ... 2024-03-04T15:34:36.932 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-04T15:34:36.950 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-portend (3.0.0-1) ... 2024-03-04T15:34:36.980 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-04T15:34:37.245 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-zc.lockfile. 2024-03-04T15:34:37.272 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../038-python3-zc.lockfile_2.0-1_all.deb ... 2024-03-04T15:34:37.308 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-04T15:34:37.311 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2024-03-04T15:34:37.335 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-04T15:34:37.374 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-04T15:34:37.615 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-cherrypy3. 2024-03-04T15:34:37.641 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../039-python3-cherrypy3_18.6.1-4_all.deb ... 2024-03-04T15:34:37.682 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2024-03-04T15:34:37.685 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-webob. 2024-03-04T15:34:37.712 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-04T15:34:37.752 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-04T15:34:38.045 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-natsort. 2024-03-04T15:34:38.071 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../040-python3-natsort_8.0.2-1_all.deb ... 2024-03-04T15:34:38.088 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-waitress. 2024-03-04T15:34:38.111 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-natsort (8.0.2-1) ... 2024-03-04T15:34:38.115 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-04T15:34:38.158 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-04T15:34:38.423 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-logutils. 2024-03-04T15:34:38.441 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-tempita. 2024-03-04T15:34:38.450 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../041-python3-logutils_0.3.3-8_all.deb ... 2024-03-04T15:34:38.457 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-04T15:34:38.490 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-logutils (0.3.3-8) ... 2024-03-04T15:34:38.491 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-04T15:34:38.760 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-markupsafe. 2024-03-04T15:34:38.775 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../042-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2024-03-04T15:34:38.810 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2024-03-04T15:34:39.105 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-mako. 2024-03-04T15:34:39.132 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../043-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2024-03-04T15:34:39.172 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-04T15:34:39.311 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-paste. 2024-03-04T15:34:39.338 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-04T15:34:39.501 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-simplegeneric. 2024-03-04T15:34:39.528 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../044-python3-simplegeneric_0.8.1-3_all.deb ... 2024-03-04T15:34:39.553 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-04T15:34:39.567 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2024-03-04T15:34:39.839 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-04T15:34:39.855 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-04T15:34:39.871 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-singledispatch. 2024-03-04T15:34:39.897 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-04T15:34:39.898 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../045-python3-singledispatch_3.4.0.3-3_all.deb ... 2024-03-04T15:34:39.929 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2024-03-04T15:34:40.200 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-04T15:34:40.224 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-webob. 2024-03-04T15:34:40.227 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-04T15:34:40.251 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../046-python3-webob_1%3a1.8.6-1.1_all.deb ... 2024-03-04T15:34:40.267 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-04T15:34:40.291 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2024-03-04T15:34:40.562 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-webtest. 2024-03-04T15:34:40.588 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-03-04T15:34:40.620 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-04T15:34:40.845 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-waitress. 2024-03-04T15:34:40.872 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../047-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2024-03-04T15:34:40.906 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pecan. 2024-03-04T15:34:40.932 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-04T15:34:40.972 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-04T15:34:41.274 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-04T15:34:41.344 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-certifi. 2024-03-04T15:34:41.371 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-03-04T15:34:41.410 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-04T15:34:41.730 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-idna. 2024-03-04T15:34:41.757 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-03-04T15:34:41.796 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-04T15:34:41.841 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-tempita. 2024-03-04T15:34:41.868 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../048-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2024-03-04T15:34:41.907 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2024-03-04T15:34:42.074 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-urllib3. 2024-03-04T15:34:42.101 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-04T15:34:42.140 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-04T15:34:42.161 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-paste. 2024-03-04T15:34:42.188 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../049-python3-paste_3.5.0+dfsg1-1_all.deb ... 2024-03-04T15:34:42.227 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2024-03-04T15:34:42.444 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-requests. 2024-03-04T15:34:42.471 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-04T15:34:42.510 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-04T15:34:42.548 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2024-03-04T15:34:42.564 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../050-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2024-03-04T15:34:42.606 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2024-03-04T15:34:42.822 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-04T15:34:42.848 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-04T15:34:42.868 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pastedeploy. 2024-03-04T15:34:42.883 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../051-python3-pastedeploy_2.1.1-1_all.deb ... 2024-03-04T15:34:42.888 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-04T15:34:42.926 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2024-03-04T15:34:43.166 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-04T15:34:43.190 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-1826-g9ca0b5de-1jammy_all.deb ... 2024-03-04T15:34:43.224 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:43.247 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-webtest. 2024-03-04T15:34:43.274 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../052-python3-webtest_2.0.35-1_all.deb ... 2024-03-04T15:34:43.313 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-webtest (2.0.35-1) ... 2024-03-04T15:34:43.579 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-04T15:34:43.606 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:43.625 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pecan. 2024-03-04T15:34:43.645 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:43.652 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../053-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2024-03-04T15:34:43.692 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2024-03-04T15:34:43.931 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-04T15:34:43.958 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-04T15:34:43.998 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-04T15:34:44.098 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-certifi. 2024-03-04T15:34:44.125 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../054-python3-certifi_2020.6.20-1_all.deb ... 2024-03-04T15:34:44.164 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2024-03-04T15:34:44.267 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-openssl. 2024-03-04T15:34:44.294 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-03-04T15:34:44.325 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-04T15:34:44.451 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-idna. 2024-03-04T15:34:44.467 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../055-python3-idna_3.3-1_all.deb ... 2024-03-04T15:34:44.510 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-idna (3.3-1) ... 2024-03-04T15:34:44.830 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-urllib3. 2024-03-04T15:34:44.857 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../056-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2024-03-04T15:34:44.896 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-04T15:34:44.920 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr. 2024-03-04T15:34:44.947 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:44.986 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:45.217 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-requests. 2024-03-04T15:34:45.243 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../057-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2024-03-04T15:34:45.292 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-04T15:34:45.373 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mon. 2024-03-04T15:34:45.400 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:45.439 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mon (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:45.612 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-werkzeug. 2024-03-04T15:34:45.639 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../058-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2024-03-04T15:34:45.679 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-04T15:34:46.018 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-osd. 2024-03-04T15:34:46.045 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:46.076 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-osd (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:46.182 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2024-03-04T15:34:46.198 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../059-ceph-mgr-modules-core_19.0.0-1826-g9ca0b5de-1jammy_all.deb ... 2024-03-04T15:34:46.241 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr-modules-core (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:46.589 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2024-03-04T15:34:46.615 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../060-libsqlite3-mod-ceph_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:46.655 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libsqlite3-mod-ceph (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:46.951 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-bcrypt. 2024-03-04T15:34:46.977 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../061-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2024-03-04T15:34:47.017 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2024-03-04T15:34:47.121 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph. 2024-03-04T15:34:47.136 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../066-ceph_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:47.179 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:47.321 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-openssl. 2024-03-04T15:34:47.347 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../062-python3-openssl_21.0.0-1_all.deb ... 2024-03-04T15:34:47.387 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-openssl (21.0.0-1) ... 2024-03-04T15:34:47.465 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-fuse. 2024-03-04T15:34:47.480 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:47.515 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-fuse (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:47.691 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr. 2024-03-04T15:34:47.706 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../063-ceph-mgr_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:47.741 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:47.852 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mds. 2024-03-04T15:34:47.878 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:47.918 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mds (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:48.136 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mon. 2024-03-04T15:34:48.163 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../064-ceph-mon_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:48.203 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mon (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:48.304 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package cephadm. 2024-03-04T15:34:48.331 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../069-cephadm_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:48.370 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking cephadm (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:48.682 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-04T15:34:48.708 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-04T15:34:48.756 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-04T15:34:48.799 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-osd. 2024-03-04T15:34:48.826 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../065-ceph-osd_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:48.865 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-osd (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:49.059 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-04T15:34:49.086 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-04T15:34:49.125 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-04T15:34:49.936 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph. 2024-03-04T15:34:49.951 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../066-ceph_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:50.003 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:50.139 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-babel. 2024-03-04T15:34:50.155 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-04T15:34:50.189 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-04T15:34:50.332 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-fuse. 2024-03-04T15:34:50.358 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../067-ceph-fuse_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:50.398 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-fuse (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:50.475 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-jinja2. 2024-03-04T15:34:50.502 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-04T15:34:50.533 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-04T15:34:50.760 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mds. 2024-03-04T15:34:50.786 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../068-ceph-mds_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:50.826 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mds (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:50.887 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-04T15:34:50.913 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-1826-g9ca0b5de-1jammy_all.deb ... 2024-03-04T15:34:50.944 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:51.230 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package cephadm. 2024-03-04T15:34:51.257 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../069-cephadm_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:34:51.273 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-04T15:34:51.288 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-03-04T15:34:51.297 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking cephadm (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:51.322 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-04T15:34:51.584 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-routes. 2024-03-04T15:34:51.610 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-04T15:34:51.617 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-asyncssh. 2024-03-04T15:34:51.643 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../070-python3-asyncssh_2.5.0-1_all.deb ... 2024-03-04T15:34:51.650 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-04T15:34:51.767 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2024-03-04T15:34:51.970 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-04T15:34:51.997 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-1826-g9ca0b5de-1jammy_all.deb ... 2024-03-04T15:34:52.036 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:52.087 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python-babel-localedata. 2024-03-04T15:34:52.114 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../071-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2024-03-04T15:34:52.153 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-04T15:34:52.473 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-04T15:34:52.500 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-04T15:34:52.559 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-04T15:34:53.043 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-joblib. 2024-03-04T15:34:53.070 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-04T15:34:53.109 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-04T15:34:53.193 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-babel. 2024-03-04T15:34:53.220 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../072-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2024-03-04T15:34:53.259 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2024-03-04T15:34:53.463 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-04T15:34:53.490 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-04T15:34:53.529 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-04T15:34:53.597 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-jinja2. 2024-03-04T15:34:53.624 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../073-python3-jinja2_3.0.3-1ubuntu0.1_all.deb ... 2024-03-04T15:34:53.663 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-04T15:34:53.840 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-sklearn. 2024-03-04T15:34:53.868 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-04T15:34:53.906 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-04T15:34:54.025 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2024-03-04T15:34:54.052 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../074-ceph-mgr-cephadm_19.0.0-1826-g9ca0b5de-1jammy_all.deb ... 2024-03-04T15:34:54.092 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr-cephadm (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:54.438 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-repoze.lru. 2024-03-04T15:34:54.464 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../075-python3-repoze.lru_0.7-2_all.deb ... 2024-03-04T15:34:54.504 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2024-03-04T15:34:54.557 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-04T15:34:54.585 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-1826-g9ca0b5de-1jammy_all.deb ... 2024-03-04T15:34:54.709 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:54.807 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-routes. 2024-03-04T15:34:54.834 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../076-python3-routes_2.5.1-1ubuntu1_all.deb ... 2024-03-04T15:34:54.874 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2024-03-04T15:34:55.211 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2024-03-04T15:34:55.227 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../077-ceph-mgr-dashboard_19.0.0-1826-g9ca0b5de-1jammy_all.deb ... 2024-03-04T15:34:55.269 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr-dashboard (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:55.684 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2024-03-04T15:34:55.711 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../078-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2024-03-04T15:34:55.751 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-04T15:34:55.862 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-cachetools. 2024-03-04T15:34:55.890 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-03-04T15:34:55.929 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-04T15:34:56.199 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-04T15:34:56.225 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-04T15:34:56.246 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-joblib. 2024-03-04T15:34:56.256 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-04T15:34:56.274 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../079-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2024-03-04T15:34:56.471 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2024-03-04T15:34:56.551 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-04T15:34:56.578 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-04T15:34:56.610 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-04T15:34:56.826 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-threadpoolctl. 2024-03-04T15:34:56.853 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../080-python3-threadpoolctl_3.1.0-1_all.deb ... 2024-03-04T15:34:56.884 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2024-03-04T15:34:56.929 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-rsa. 2024-03-04T15:34:56.956 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-03-04T15:34:56.996 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-04T15:34:57.171 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-sklearn. 2024-03-04T15:34:57.199 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../081-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2024-03-04T15:34:57.237 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-04T15:34:57.308 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-google-auth. 2024-03-04T15:34:57.335 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-03-04T15:34:57.366 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-04T15:34:57.661 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-04T15:34:57.688 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-04T15:34:57.727 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-04T15:34:57.990 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2024-03-04T15:34:58.017 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../082-ceph-mgr-diskprediction-local_19.0.0-1826-g9ca0b5de-1jammy_all.deb ... 2024-03-04T15:34:58.038 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-websocket. 2024-03-04T15:34:58.066 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-03-04T15:34:58.105 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-04T15:34:58.142 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr-diskprediction-local (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:58.417 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-04T15:34:58.444 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-04T15:34:58.483 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-04T15:34:59.097 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-04T15:34:59.113 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-1826-g9ca0b5de-1jammy_all.deb ... 2024-03-04T15:34:59.130 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-cachetools. 2024-03-04T15:34:59.146 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:34:59.158 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../083-python3-cachetools_5.0.0-1_all.deb ... 2024-03-04T15:34:59.196 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2024-03-04T15:34:59.450 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-04T15:34:59.476 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-04T15:34:59.516 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-04T15:34:59.517 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pyasn1. 2024-03-04T15:34:59.544 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../084-python3-pyasn1_0.4.8-1_all.deb ... 2024-03-04T15:34:59.583 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2024-03-04T15:34:59.855 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-04T15:34:59.881 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-04T15:34:59.904 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pyasn1-modules. 2024-03-04T15:34:59.920 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-04T15:34:59.931 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../085-python3-pyasn1-modules_0.2.1-1_all.deb ... 2024-03-04T15:34:59.971 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2024-03-04T15:35:00.148 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package jq. 2024-03-04T15:35:00.164 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-04T15:35:00.206 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-04T15:35:00.283 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-rsa. 2024-03-04T15:35:00.310 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../086-python3-rsa_4.8-1_all.deb ... 2024-03-04T15:35:00.349 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-rsa (4.8-1) ... 2024-03-04T15:35:00.634 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package socat. 2024-03-04T15:35:00.657 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-04T15:35:00.696 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-google-auth. 2024-03-04T15:35:00.700 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-04T15:35:00.723 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../087-python3-google-auth_1.5.1-3_all.deb ... 2024-03-04T15:35:00.754 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2024-03-04T15:35:01.004 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package xmlstarlet. 2024-03-04T15:35:01.031 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-04T15:35:01.070 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-04T15:35:01.083 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-requests-oauthlib. 2024-03-04T15:35:01.110 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../088-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2024-03-04T15:35:01.150 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-04T15:35:01.420 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-websocket. 2024-03-04T15:35:01.424 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-test. 2024-03-04T15:35:01.437 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../089-python3-websocket_1.2.3-1_all.deb ... 2024-03-04T15:35:01.440 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../097-ceph-test_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:35:01.478 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-websocket (1.2.3-1) ... 2024-03-04T15:35:01.482 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-test (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:01.799 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-kubernetes. 2024-03-04T15:35:01.827 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../090-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2024-03-04T15:35:01.865 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-04T15:35:02.589 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2024-03-04T15:35:02.605 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../091-ceph-mgr-k8sevents_19.0.0-1826-g9ca0b5de-1jammy_all.deb ... 2024-03-04T15:35:02.647 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-mgr-k8sevents (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:02.984 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libonig5:amd64. 2024-03-04T15:35:03.012 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../092-libonig5_6.9.7.1-2build1_amd64.deb ... 2024-03-04T15:35:03.051 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-04T15:35:03.371 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libjq1:amd64. 2024-03-04T15:35:03.399 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../093-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2024-03-04T15:35:03.429 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-04T15:35:03.636 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package ceph-volume. 2024-03-04T15:35:03.652 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-1826-g9ca0b5de-1jammy_all.deb ... 2024-03-04T15:35:03.691 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package jq. 2024-03-04T15:35:03.695 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking ceph-volume (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:03.707 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../094-jq_1.6-2.1ubuntu3_amd64.deb ... 2024-03-04T15:35:03.749 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2024-03-04T15:35:03.965 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-04T15:35:03.981 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:35:04.002 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package socat. 2024-03-04T15:35:04.015 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking libcephfs-dev (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:04.018 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../095-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2024-03-04T15:35:04.061 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2024-03-04T15:35:04.301 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-04T15:35:04.328 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-04T15:35:04.340 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package xmlstarlet. 2024-03-04T15:35:04.356 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../096-xmlstarlet_1.6.1-2.1_amd64.deb ... 2024-03-04T15:35:04.367 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-04T15:35:04.389 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2024-03-04T15:35:04.705 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-04T15:35:04.733 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-03-04T15:35:04.737 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-test. 2024-03-04T15:35:04.764 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../097-ceph-test_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:35:04.771 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-04T15:35:04.795 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-test (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:05.117 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package nvme-cli. 2024-03-04T15:35:05.144 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-04T15:35:05.183 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-04T15:35:05.589 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package pkg-config. 2024-03-04T15:35:05.616 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-04T15:35:05.655 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-04T15:35:05.899 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-04T15:35:05.926 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-04T15:35:05.957 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-04T15:35:06.377 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-04T15:35:06.404 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-04T15:35:06.443 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-04T15:35:06.721 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pastescript. 2024-03-04T15:35:06.749 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-03-04T15:35:06.787 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-04T15:35:07.108 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pluggy. 2024-03-04T15:35:07.135 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-04T15:35:07.174 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-04T15:35:07.317 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package ceph-volume. 2024-03-04T15:35:07.344 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../098-ceph-volume_19.0.0-1826-g9ca0b5de-1jammy_all.deb ... 2024-03-04T15:35:07.384 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking ceph-volume (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:07.485 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-psutil. 2024-03-04T15:35:07.512 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-04T15:35:07.551 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-04T15:35:07.680 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package libcephfs-dev. 2024-03-04T15:35:07.695 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../099-libcephfs-dev_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:35:07.738 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking libcephfs-dev (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:07.880 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-py. 2024-03-04T15:35:07.907 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-03-04T15:35:07.954 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-04T15:35:08.033 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package lua-socket:amd64. 2024-03-04T15:35:08.060 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../100-lua-socket_3.0~rc1+git+ac3201d-6_amd64.deb ... 2024-03-04T15:35:08.100 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-04T15:35:08.299 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pygments. 2024-03-04T15:35:08.327 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-04T15:35:08.365 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-04T15:35:08.422 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package lua-sec:amd64. 2024-03-04T15:35:08.449 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../101-lua-sec_1.0.2-1_amd64.deb ... 2024-03-04T15:35:08.488 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking lua-sec:amd64 (1.0.2-1) ... 2024-03-04T15:35:08.802 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-04T15:35:08.818 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-04T15:35:08.843 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package nvme-cli. 2024-03-04T15:35:08.860 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-04T15:35:08.871 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../102-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2024-03-04T15:35:08.910 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2024-03-04T15:35:09.114 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-toml. 2024-03-04T15:35:09.141 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-03-04T15:35:09.171 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-04T15:35:09.333 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package pkg-config. 2024-03-04T15:35:09.360 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../103-pkg-config_0.29.2-1ubuntu3_amd64.deb ... 2024-03-04T15:35:09.399 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking pkg-config (0.29.2-1ubuntu3) ... 2024-03-04T15:35:09.458 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-pytest. 2024-03-04T15:35:09.487 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-04T15:35:09.524 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-04T15:35:09.645 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python-asyncssh-doc. 2024-03-04T15:35:09.660 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../104-python-asyncssh-doc_2.5.0-1_all.deb ... 2024-03-04T15:35:09.703 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2024-03-04T15:35:09.853 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package python3-simplejson. 2024-03-04T15:35:09.880 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-04T15:35:09.919 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-04T15:35:10.140 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-iniconfig. 2024-03-04T15:35:10.167 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../105-python3-iniconfig_1.1.1-2_all.deb ... 2024-03-04T15:35:10.180 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-04T15:35:10.206 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2024-03-04T15:35:10.208 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-04T15:35:10.238 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-04T15:35:10.519 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pastescript. 2024-03-04T15:35:10.546 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../106-python3-pastescript_2.0.2-4_all.deb ... 2024-03-04T15:35:10.585 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2024-03-04T15:35:10.906 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pluggy. 2024-03-04T15:35:10.933 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../107-python3-pluggy_0.13.0-7.1_all.deb ... 2024-03-04T15:35:10.972 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2024-03-04T15:35:11.234 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package radosgw. 2024-03-04T15:35:11.259 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-psutil. 2024-03-04T15:35:11.261 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../116-radosgw_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:35:11.287 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2024-03-04T15:35:11.300 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking radosgw (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:11.327 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2024-03-04T15:35:11.629 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-py. 2024-03-04T15:35:11.645 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../109-python3-py_1.10.0-1_all.deb ... 2024-03-04T15:35:11.679 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-py (1.10.0-1) ... 2024-03-04T15:35:12.000 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pygments. 2024-03-04T15:35:12.028 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../110-python3-pygments_2.11.2+dfsg-2_all.deb ... 2024-03-04T15:35:12.061 INFO:teuthology.orchestra.run.smithi117.stdout:Selecting previously unselected package rbd-fuse. 2024-03-04T15:35:12.066 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2024-03-04T15:35:12.077 INFO:teuthology.orchestra.run.smithi117.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:35:12.119 INFO:teuthology.orchestra.run.smithi117.stdout:Unpacking rbd-fuse (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:12.406 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-04T15:35:12.505 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pyinotify. 2024-03-04T15:35:12.532 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../111-python3-pyinotify_0.9.6-1.3_all.deb ... 2024-03-04T15:35:12.563 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2024-03-04T15:35:12.641 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-04T15:35:12.767 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-04T15:35:12.875 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-toml. 2024-03-04T15:35:12.902 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../112-python3-toml_0.10.2-1_all.deb ... 2024-03-04T15:35:12.941 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-toml (0.10.2-1) ... 2024-03-04T15:35:13.025 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-04T15:35:13.245 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-pytest. 2024-03-04T15:35:13.272 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../113-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2024-03-04T15:35:13.311 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2024-03-04T15:35:13.367 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-04T15:35:13.607 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package python3-simplejson. 2024-03-04T15:35:13.623 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../114-python3-simplejson_3.17.6-1build1_amd64.deb ... 2024-03-04T15:35:13.665 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2024-03-04T15:35:13.788 INFO:teuthology.orchestra.run.smithi117.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-04T15:35:13.860 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up cephadm (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:13.919 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package qttranslations5-l10n. 2024-03-04T15:35:13.935 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../115-qttranslations5-l10n_5.15.3-1_all.deb ... 2024-03-04T15:35:13.977 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2024-03-04T15:35:14.368 INFO:teuthology.orchestra.run.smithi117.stdout:Adding system user cephadm....done 2024-03-04T15:35:14.516 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-04T15:35:14.690 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package radosgw. 2024-03-04T15:35:14.718 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../116-radosgw_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:35:14.765 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking radosgw (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:14.800 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-04T15:35:15.060 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-04T15:35:15.186 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-04T15:35:15.437 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-04T15:35:15.535 INFO:teuthology.orchestra.run.smithi151.stdout:Selecting previously unselected package rbd-fuse. 2024-03-04T15:35:15.559 INFO:teuthology.orchestra.run.smithi151.stdout:Preparing to unpack .../117-rbd-fuse_19.0.0-1826-g9ca0b5de-1jammy_amd64.deb ... 2024-03-04T15:35:15.602 INFO:teuthology.orchestra.run.smithi151.stdout:Unpacking rbd-fuse (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:15.680 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-04T15:35:15.789 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-04T15:35:15.907 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2024-03-04T15:35:16.116 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-04T15:35:16.167 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2024-03-04T15:35:16.293 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2024-03-04T15:35:16.459 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-04T15:35:16.562 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2024-03-04T15:35:16.727 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-04T15:35:16.870 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2024-03-04T15:35:17.003 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-04T15:35:17.255 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-04T15:35:17.267 INFO:teuthology.orchestra.run.smithi151.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2024-03-04T15:35:17.506 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-04T15:35:17.570 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up cephadm (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:17.765 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-04T15:35:18.016 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-ceph-argparse (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:18.071 INFO:teuthology.orchestra.run.smithi151.stdout:Adding system user cephadm....done 2024-03-04T15:35:18.211 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2024-03-04T15:35:18.284 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-04T15:35:18.409 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-04T15:35:18.479 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2024-03-04T15:35:18.535 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-04T15:35:18.653 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-04T15:35:18.772 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2024-03-04T15:35:18.899 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2024-03-04T15:35:18.917 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-04T15:35:19.150 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-repoze.lru (0.7-2) ... 2024-03-04T15:35:19.300 INFO:teuthology.orchestra.run.smithi117.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-04T15:35:19.335 INFO:teuthology.orchestra.run.smithi117.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-04T15:35:19.399 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-04T15:35:19.401 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2024-03-04T15:35:19.528 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-py (1.10.0-1) ... 2024-03-04T15:35:19.550 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-04T15:35:19.830 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2024-03-04T15:35:19.909 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-04T15:35:20.052 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-04T15:35:20.183 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-cachetools (5.0.0-1) ... 2024-03-04T15:35:20.311 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-04T15:35:20.451 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-openssl (21.0.0-1) ... 2024-03-04T15:35:20.546 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-04T15:35:20.728 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up unzip (6.0-26ubuntu3.2) ... 2024-03-04T15:35:20.847 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up zip (3.0-12build2) ... 2024-03-04T15:35:20.964 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-04T15:35:20.971 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2024-03-04T15:35:21.231 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2024-03-04T15:35:21.490 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2024-03-04T15:35:21.666 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-04T15:35:21.742 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-ceph-argparse (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:21.925 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-04T15:35:22.001 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2024-03-04T15:35:22.051 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-04T15:35:22.120 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up lua-socket:amd64 (3.0~rc1+git+ac3201d-6) ... 2024-03-04T15:35:22.177 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-04T15:35:22.229 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libreadline-dev:amd64 (8.1.2-1) ... 2024-03-04T15:35:22.338 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2024-03-04T15:35:22.453 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-04T15:35:22.623 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up lua5.1 (5.1.5-8.1build4) ... 2024-03-04T15:35:22.813 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-04T15:35:23.054 INFO:teuthology.orchestra.run.smithi151.stdout:update-alternatives: using /usr/bin/lua5.1 to provide /usr/bin/lua (lua-interpreter) in auto mode 2024-03-04T15:35:23.055 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-04T15:35:23.088 INFO:teuthology.orchestra.run.smithi151.stdout:update-alternatives: using /usr/bin/luac5.1 to provide /usr/bin/luac (lua-compiler) in auto mode 2024-03-04T15:35:23.152 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2024-03-04T15:35:23.278 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2024-03-04T15:35:23.440 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-04T15:35:23.630 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2024-03-04T15:35:23.750 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-04T15:35:23.756 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-natsort (8.0.2-1) ... 2024-03-04T15:35:24.024 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2024-03-04T15:35:24.093 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-04T15:35:24.267 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2024-03-04T15:35:24.352 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-04T15:35:24.462 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-ceph-common (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:24.569 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up zip (3.0-12build2) ... 2024-03-04T15:35:24.695 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2024-03-04T15:35:24.729 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-idna (3.3-1) ... 2024-03-04T15:35:25.039 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-04T15:35:25.331 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2024-03-04T15:35:25.590 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2024-03-04T15:35:25.700 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2024-03-04T15:35:25.810 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2024-03-04T15:35:26.120 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2024-03-04T15:35:26.144 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-04T15:35:26.455 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T15:35:26.529 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-certifi (2020.6.20-1) ... 2024-03-04T15:35:26.763 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-04T15:35:26.781 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2024-03-04T15:35:27.031 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-04T15:35:27.166 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2024-03-04T15:35:27.182 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-04T15:35:27.308 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-04T15:35:27.459 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2024-03-04T15:35:27.601 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-04T15:35:27.811 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2024-03-04T15:35:27.869 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-04T15:35:28.046 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2024-03-04T15:35:28.120 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T15:35:28.172 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-ceph-common (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:28.237 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-04T15:35:28.474 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-idna (3.3-1) ... 2024-03-04T15:35:28.530 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-04T15:35:28.781 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-04T15:35:28.783 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2024-03-04T15:35:29.049 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-04T15:35:29.300 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-04T15:35:29.551 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-04T15:35:29.810 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-04T15:35:29.920 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-04T15:35:29.940 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up pkg-config (0.29.2-1ubuntu3) ... 2024-03-04T15:35:30.179 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-04T15:35:30.209 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T15:35:30.305 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-04T15:35:30.344 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-toml (0.10.2-1) ... 2024-03-04T15:35:30.598 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-04T15:35:30.620 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2024-03-04T15:35:30.747 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2024-03-04T15:35:30.873 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2024-03-04T15:35:30.898 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-04T15:35:31.150 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2024-03-04T15:35:31.183 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-04T15:35:31.309 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-04T15:35:31.393 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2024-03-04T15:35:31.435 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-04T15:35:31.524 INFO:teuthology.orchestra.run.smithi117.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-04T15:35:31.636 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T15:35:31.745 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-04T15:35:31.762 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2024-03-04T15:35:31.863 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-04T15:35:32.064 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2024-03-04T15:35:32.247 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-04T15:35:32.323 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-logutils (0.3.3-8) ... 2024-03-04T15:35:32.507 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-04T15:35:32.558 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-tempora (4.1.2-1) ... 2024-03-04T15:35:32.633 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-04T15:35:32.892 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T15:35:33.026 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-04T15:35:33.068 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2024-03-04T15:35:33.294 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-prettytable (2.5.0-2) ... 2024-03-04T15:35:33.344 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-04T15:35:33.546 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2024-03-04T15:35:33.620 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-04T15:35:33.672 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-websocket (1.2.3-1) ... 2024-03-04T15:35:33.879 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-04T15:35:33.948 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2024-03-04T15:35:34.005 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-04T15:35:34.083 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2024-03-04T15:35:34.306 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-04T15:35:34.368 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2024-03-04T15:35:34.574 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-04T15:35:34.677 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2024-03-04T15:35:34.910 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-04T15:35:34.920 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up liblua5.3-dev:amd64 (5.3.6-1build1) ... 2024-03-04T15:35:35.047 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up lua-sec:amd64 (1.0.2-1) ... 2024-03-04T15:35:35.173 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2024-03-04T15:35:35.245 INFO:teuthology.orchestra.run.smithi151.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2024-03-04T15:35:35.270 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-04T15:35:35.475 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2024-03-04T15:35:35.555 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-04T15:35:35.577 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2024-03-04T15:35:35.899 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-04T15:35:35.945 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2024-03-04T15:35:36.025 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librados2 (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:36.160 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:36.213 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up lua-any (27ubuntu1) ... 2024-03-04T15:35:36.260 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-04T15:35:36.339 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-portend (3.0.0-1) ... 2024-03-04T15:35:36.563 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libcephfs2 (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:36.574 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2024-03-04T15:35:36.686 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-jinja2 (3.0.3-1ubuntu0.1) ... 2024-03-04T15:35:36.689 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libradosstriper1 (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:36.815 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-04T15:35:36.987 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2024-03-04T15:35:37.100 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librbd1 (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:37.226 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-modules-core (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:37.280 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2024-03-04T15:35:37.327 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-fuse (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:37.530 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-04T15:35:37.530 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-04T15:35:37.547 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2024-03-04T15:35:37.673 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-rsa (4.8-1) ... 2024-03-04T15:35:37.941 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-webtest (2.0.35-1) ... 2024-03-04T15:35:37.995 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up libcephfs-dev (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:38.121 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-rados (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:38.209 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2024-03-04T15:35:38.247 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up librgw2 (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:38.373 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-04T15:35:38.554 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2024-03-04T15:35:38.930 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pastescript (2.0.2-4) ... 2024-03-04T15:35:39.215 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2024-03-04T15:35:39.542 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2024-03-04T15:35:39.584 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-rbd (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:39.668 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up librados2 (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:39.685 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up rbd-fuse (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:39.787 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-rgw (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:39.802 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libsqlite3-mod-ceph (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:39.904 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up python3-cephfs (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:39.929 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up luarocks (3.8.0+dfsg1-1) ... 2024-03-04T15:35:40.030 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-common (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:40.265 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libcephfs2 (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:40.382 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libradosstriper1 (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:40.491 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-google-auth (1.5.1-3) ... 2024-03-04T15:35:40.573 INFO:teuthology.orchestra.run.smithi117.stdout:Adding group ceph....done 2024-03-04T15:35:40.785 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up librbd1 (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:40.899 INFO:teuthology.orchestra.run.smithi117.stdout:Adding system user ceph....done 2024-03-04T15:35:40.911 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr-modules-core (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:41.037 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-fuse (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:41.068 INFO:teuthology.orchestra.run.smithi117.stdout:Setting system user ceph properties....done 2024-03-04T15:35:41.079 INFO:teuthology.orchestra.run.smithi117.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-04T15:35:41.215 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-04T15:35:41.248 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-04T15:35:41.248 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2024-03-04T15:35:41.528 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-04T15:35:41.721 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up libcephfs-dev (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:41.847 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-rados (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:41.973 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up librgw2 (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:42.014 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-test (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:42.091 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2024-03-04T15:35:42.140 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up radosgw (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:42.591 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-04T15:35:42.591 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-04T15:35:43.092 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-base (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:43.328 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-rbd (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:43.436 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-04T15:35:43.453 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up rbd-fuse (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:43.579 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-rgw (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:43.705 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up python3-cephfs (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:43.831 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-common (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:43.877 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mds (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:44.112 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-04T15:35:44.112 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-04T15:35:44.358 INFO:teuthology.orchestra.run.smithi151.stdout:Adding group ceph....done 2024-03-04T15:35:44.611 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:44.742 INFO:teuthology.orchestra.run.smithi151.stdout:Adding system user ceph....done 2024-03-04T15:35:44.827 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-04T15:35:44.827 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-04T15:35:44.911 INFO:teuthology.orchestra.run.smithi151.stdout:Setting system user ceph properties....done 2024-03-04T15:35:44.922 INFO:teuthology.orchestra.run.smithi151.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2024-03-04T15:35:45.059 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2024-03-04T15:35:45.296 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-osd (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:45.358 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2024-03-04T15:35:45.557 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-04T15:35:45.557 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-04T15:35:45.798 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-test (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:45.925 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up radosgw (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:46.047 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:46.173 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:46.358 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mon (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:46.362 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-04T15:35:46.362 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2024-03-04T15:35:46.579 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-04T15:35:46.579 INFO:teuthology.orchestra.run.smithi117.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-04T15:35:46.785 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-base (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:47.042 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-cephadm (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:47.132 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2024-03-04T15:35:47.346 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:47.464 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-mgr-dashboard (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:47.606 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mds (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:47.615 INFO:teuthology.orchestra.run.smithi117.stdout:Setting up ceph-volume (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:47.960 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-04T15:35:47.960 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2024-03-04T15:35:47.967 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-04T15:35:48.060 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-04T15:35:48.212 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-04T15:35:48.440 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:49.039 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-04T15:35:49.040 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2024-03-04T15:35:49.346 INFO:teuthology.orchestra.run.smithi117.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-04T15:35:49.494 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-osd (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:49.765 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-04T15:35:49.765 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2024-03-04T15:35:49.976 INFO:teuthology.orchestra.run.smithi117.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-04T15:35:49.982 DEBUG:teuthology.parallel:result is None 2024-03-04T15:35:50.221 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr-k8sevents (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:50.347 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr-diskprediction-local (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:50.497 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mon (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:50.719 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-04T15:35:50.719 INFO:teuthology.orchestra.run.smithi151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2024-03-04T15:35:51.215 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr-cephadm (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:51.350 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:51.459 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-mgr-dashboard (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:51.650 INFO:teuthology.orchestra.run.smithi151.stdout:Setting up ceph-volume (19.0.0-1826-g9ca0b5de-1jammy) ... 2024-03-04T15:35:52.063 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for mailcap (3.70+nmu1ubuntu1) ... 2024-03-04T15:35:52.180 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2024-03-04T15:35:52.333 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for man-db (2.10.2-1) ... 2024-03-04T15:35:53.132 INFO:teuthology.orchestra.run.smithi151.stdout:Processing triggers for install-info (6.8-4build1) ... 2024-03-04T15:35:54.332 INFO:teuthology.orchestra.run.smithi151.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2024-03-04T15:35:54.337 DEBUG:teuthology.parallel:result is None 2024-03-04T15:35:54.337 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=9ca0b5de35ed230e84387abdd8e619ed8e4f3282 2024-03-04T15:35:54.565 DEBUG:teuthology.orchestra.run.smithi117:> dpkg-query -W -f '${Version}' ceph 2024-03-04T15:35:54.584 INFO:teuthology.orchestra.run.smithi117.stdout:19.0.0-1826-g9ca0b5de-1jammy 2024-03-04T15:35:54.584 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1826-g9ca0b5de-1jammy 2024-03-04T15:35:54.584 INFO:teuthology.task.install:The correct ceph version 19.0.0-1826-g9ca0b5de-1jammy is installed. 2024-03-04T15:35:54.586 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=9ca0b5de35ed230e84387abdd8e619ed8e4f3282 2024-03-04T15:35:54.811 DEBUG:teuthology.orchestra.run.smithi151:> dpkg-query -W -f '${Version}' ceph 2024-03-04T15:35:54.830 INFO:teuthology.orchestra.run.smithi151.stdout:19.0.0-1826-g9ca0b5de-1jammy 2024-03-04T15:35:54.831 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-1826-g9ca0b5de-1jammy 2024-03-04T15:35:54.831 INFO:teuthology.task.install:The correct ceph version 19.0.0-1826-g9ca0b5de-1jammy is installed. 2024-03-04T15:35:54.832 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-03-04T15:35:54.833 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-04T15:35:54.833 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-04T15:35:54.846 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-04T15:35:54.846 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-03-04T15:35:54.887 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-03-04T15:35:54.887 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-04T15:35:54.887 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/daemon-helper 2024-03-04T15:35:54.900 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-04T15:35:54.956 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-04T15:35:54.956 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/daemon-helper 2024-03-04T15:35:54.969 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-03-04T15:35:55.022 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-03-04T15:35:55.023 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-04T15:35:55.023 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-04T15:35:55.036 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-04T15:35:55.091 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-04T15:35:55.091 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/adjust-ulimits 2024-03-04T15:35:55.104 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-03-04T15:35:55.158 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-03-04T15:35:55.158 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-04T15:35:55.158 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/usr/bin/stdin-killer 2024-03-04T15:35:55.172 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-04T15:35:55.229 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-04T15:35:55.229 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/usr/bin/stdin-killer 2024-03-04T15:35:55.241 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-03-04T15:35:55.294 INFO:teuthology.run_tasks:Running task cephadm... 2024-03-04T15:35:55.389 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 1, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '9ca0b5de35ed230e84387abdd8e619ed8e4f3282', 'cephadm_mode': 'cephadm-package'} 2024-03-04T15:35:55.390 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 2024-03-04T15:35:55.390 INFO:tasks.cephadm:Cluster fsid is e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T15:35:55.390 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-03-04T15:35:55.390 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.117', 'mon.c': '[v2:172.21.15.117:3301,v1:172.21.15.117:6790]', 'mon.b': '172.21.15.151'} 2024-03-04T15:35:55.391 INFO:tasks.cephadm:First mon is mon.a on smithi117 2024-03-04T15:35:55.391 INFO:tasks.cephadm:First mgr is y 2024-03-04T15:35:55.391 INFO:tasks.cephadm:Normalizing hostnames... 2024-03-04T15:35:55.391 DEBUG:teuthology.orchestra.run.smithi117:> sudo hostname $(hostname -s) 2024-03-04T15:35:55.405 DEBUG:teuthology.orchestra.run.smithi151:> sudo hostname $(hostname -s) 2024-03-04T15:35:55.428 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 on all hosts... 2024-03-04T15:35:55.428 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 pull 2024-03-04T15:35:55.450 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 pull 2024-03-04T15:35:55.612 INFO:teuthology.orchestra.run.smithi117.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282... 2024-03-04T15:35:55.629 INFO:teuthology.orchestra.run.smithi151.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282... 2024-03-04T15:38:54.601 INFO:teuthology.orchestra.run.smithi151.stdout:{ 2024-03-04T15:38:54.601 INFO:teuthology.orchestra.run.smithi151.stdout: "ceph_version": "ceph version 19.0.0-1826-g9ca0b5de (9ca0b5de35ed230e84387abdd8e619ed8e4f3282) squid (dev)", 2024-03-04T15:38:54.602 INFO:teuthology.orchestra.run.smithi151.stdout: "image_id": "0d1497d8ab5e9dc8a566513caa4bcfff51798ecc87de6dcca9e525d8d416d692", 2024-03-04T15:38:54.602 INFO:teuthology.orchestra.run.smithi151.stdout: "repo_digests": [ 2024-03-04T15:38:54.602 INFO:teuthology.orchestra.run.smithi151.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6d920225840c1daadea71eb62fa66076c3ca6088306e8e7302cea6cb04aef977" 2024-03-04T15:38:54.602 INFO:teuthology.orchestra.run.smithi151.stdout: ] 2024-03-04T15:38:54.602 INFO:teuthology.orchestra.run.smithi151.stdout:} 2024-03-04T15:39:03.650 INFO:teuthology.orchestra.run.smithi117.stdout:{ 2024-03-04T15:39:03.651 INFO:teuthology.orchestra.run.smithi117.stdout: "ceph_version": "ceph version 19.0.0-1826-g9ca0b5de (9ca0b5de35ed230e84387abdd8e619ed8e4f3282) squid (dev)", 2024-03-04T15:39:03.651 INFO:teuthology.orchestra.run.smithi117.stdout: "image_id": "0d1497d8ab5e9dc8a566513caa4bcfff51798ecc87de6dcca9e525d8d416d692", 2024-03-04T15:39:03.651 INFO:teuthology.orchestra.run.smithi117.stdout: "repo_digests": [ 2024-03-04T15:39:03.651 INFO:teuthology.orchestra.run.smithi117.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:6d920225840c1daadea71eb62fa66076c3ca6088306e8e7302cea6cb04aef977" 2024-03-04T15:39:03.651 INFO:teuthology.orchestra.run.smithi117.stdout: ] 2024-03-04T15:39:03.651 INFO:teuthology.orchestra.run.smithi117.stdout:} 2024-03-04T15:39:03.683 DEBUG:teuthology.orchestra.run.smithi117:> sudo mkdir -p /etc/ceph 2024-03-04T15:39:03.697 DEBUG:teuthology.orchestra.run.smithi151:> sudo mkdir -p /etc/ceph 2024-03-04T15:39:03.712 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 777 /etc/ceph 2024-03-04T15:39:03.750 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 777 /etc/ceph 2024-03-04T15:39:03.766 INFO:tasks.cephadm:Writing seed config... 2024-03-04T15:39:03.768 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-03-04T15:39:03.768 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-03-04T15:39:03.768 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-03-04T15:39:03.768 INFO:tasks.cephadm: override: [global] mon election default strategy = 1 2024-03-04T15:39:03.768 INFO:tasks.cephadm: override: [global] ms type = async 2024-03-04T15:39:03.768 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-03-04T15:39:03.768 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-03-04T15:39:03.768 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-03-04T15:39:03.768 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-03-04T15:39:03.768 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-03-04T15:39:03.769 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-03-04T15:39:03.769 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-03-04T15:39:03.769 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-03-04T15:39:03.769 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-03-04T15:39:03.769 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-04T15:39:03.769 DEBUG:teuthology.orchestra.run.smithi117:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-03-04T15:39:03.796 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = e40b5504-da3c-11ee-95c6-87774f69a715 mon election default strategy = 1 ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-03-04T15:39:03.797 DEBUG:teuthology.orchestra.run.smithi117:mon.a> sudo journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mon.a.service 2024-03-04T15:39:03.841 DEBUG:teuthology.orchestra.run.smithi117:mgr.y> sudo journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mgr.y.service 2024-03-04T15:39:03.884 INFO:tasks.cephadm:Bootstrapping... 2024-03-04T15:39:03.885 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 -v bootstrap --fsid e40b5504-da3c-11ee-95c6-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.117 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-03-04T15:39:04.084 INFO:teuthology.orchestra.run.smithi117.stdout:-------------------------------------------------------------------------------- 2024-03-04T15:39:04.084 INFO:teuthology.orchestra.run.smithi117.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282', '-v', 'bootstrap', '--fsid', 'e40b5504-da3c-11ee-95c6-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.117', '--skip-admin-label'] 2024-03-04T15:39:04.085 INFO:teuthology.orchestra.run.smithi117.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-03-04T15:39:04.085 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying podman|docker is present... 2024-03-04T15:39:04.085 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying lvm2 is present... 2024-03-04T15:39:04.085 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying time synchronization is in place... 2024-03-04T15:39:04.090 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-04T15:39:04.090 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-04T15:39:04.094 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-04T15:39:04.094 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-03-04T15:39:04.097 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled chronyd.service 2024-03-04T15:39:04.097 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Failed to get unit file state for chronyd.service: No such file or directory 2024-03-04T15:39:04.101 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active chronyd.service 2024-03-04T15:39:04.101 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-03-04T15:39:04.104 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled systemd-timesyncd.service 2024-03-04T15:39:04.104 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout masked 2024-03-04T15:39:04.107 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active systemd-timesyncd.service 2024-03-04T15:39:04.108 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-03-04T15:39:04.111 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled ntpd.service 2024-03-04T15:39:04.111 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Failed to get unit file state for ntpd.service: No such file or directory 2024-03-04T15:39:04.114 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active ntpd.service 2024-03-04T15:39:04.115 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-03-04T15:39:04.118 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout enabled 2024-03-04T15:39:04.122 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout active 2024-03-04T15:39:04.122 INFO:teuthology.orchestra.run.smithi117.stdout:Unit ntp.service is enabled and running 2024-03-04T15:39:04.122 INFO:teuthology.orchestra.run.smithi117.stdout:Repeating the final host check... 2024-03-04T15:39:04.122 INFO:teuthology.orchestra.run.smithi117.stdout:docker (/usr/bin/docker) is present 2024-03-04T15:39:04.122 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl is present 2024-03-04T15:39:04.122 INFO:teuthology.orchestra.run.smithi117.stdout:lvcreate is present 2024-03-04T15:39:04.125 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-03-04T15:39:04.125 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-03-04T15:39:04.128 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-03-04T15:39:04.128 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-03-04T15:39:04.131 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled chronyd.service 2024-03-04T15:39:04.131 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Failed to get unit file state for chronyd.service: No such file or directory 2024-03-04T15:39:04.134 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active chronyd.service 2024-03-04T15:39:04.135 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-03-04T15:39:04.138 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled systemd-timesyncd.service 2024-03-04T15:39:04.138 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout masked 2024-03-04T15:39:04.140 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active systemd-timesyncd.service 2024-03-04T15:39:04.141 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-03-04T15:39:04.144 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl is-enabled ntpd.service 2024-03-04T15:39:04.144 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Failed to get unit file state for ntpd.service: No such file or directory 2024-03-04T15:39:04.147 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 3 from systemctl is-active ntpd.service 2024-03-04T15:39:04.147 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout inactive 2024-03-04T15:39:04.150 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout enabled 2024-03-04T15:39:04.154 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stdout active 2024-03-04T15:39:04.154 INFO:teuthology.orchestra.run.smithi117.stdout:Unit ntp.service is enabled and running 2024-03-04T15:39:04.154 INFO:teuthology.orchestra.run.smithi117.stdout:Host looks OK 2024-03-04T15:39:04.154 INFO:teuthology.orchestra.run.smithi117.stdout:Cluster fsid: e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T15:39:04.154 INFO:teuthology.orchestra.run.smithi117.stdout:Acquiring lock 140642575757232 on /run/cephadm/e40b5504-da3c-11ee-95c6-87774f69a715.lock 2024-03-04T15:39:04.154 INFO:teuthology.orchestra.run.smithi117.stdout:Lock 140642575757232 acquired on /run/cephadm/e40b5504-da3c-11ee-95c6-87774f69a715.lock 2024-03-04T15:39:04.154 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying IP 172.21.15.117 port 3300 ... 2024-03-04T15:39:04.155 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying IP 172.21.15.117 port 6789 ... 2024-03-04T15:39:04.155 INFO:teuthology.orchestra.run.smithi117.stdout:Base mon IP(s) is [172.21.15.117:3300, 172.21.15.117:6789], mon addrv is [v2:172.21.15.117:3300,v1:172.21.15.117:6789] 2024-03-04T15:39:04.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 2024-03-04T15:39:04.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.117 metric 100 2024-03-04T15:39:04.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/sbin/ip: stdout 172.17.0.0/16 dev docker0 proto kernel scope link src 172.17.0.1 linkdown 2024-03-04T15:39:04.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.117 metric 100 2024-03-04T15:39:04.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/sbin/ip: stdout 172.21.0.1 dev enp3s0f1 proto dhcp scope link src 172.21.15.117 metric 100 2024-03-04T15:39:04.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/sbin/ip: stdout 172.21.0.2 dev enp3s0f1 proto dhcp scope link src 172.21.15.117 metric 100 2024-03-04T15:39:04.157 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/sbin/ip: stdout 172.21.15.254 dev enp3s0f1 proto dhcp scope link src 172.21.15.117 metric 100 2024-03-04T15:39:04.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-03-04T15:39:04.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 256 pref medium 2024-03-04T15:39:04.159 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 expires 1757sec pref medium 2024-03-04T15:39:04.160 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-03-04T15:39:04.161 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/sbin/ip: stdout inet6 ::1/128 scope host 2024-03-04T15:39:04.161 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-04T15:39:04.161 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-03-04T15:39:04.161 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:4463/64 scope link 2024-03-04T15:39:04.161 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-03-04T15:39:04.162 INFO:teuthology.orchestra.run.smithi117.stdout:Mon IP `172.21.15.117` is in CIDR network `172.21.0.0/20` 2024-03-04T15:39:04.162 INFO:teuthology.orchestra.run.smithi117.stdout:Mon IP `172.21.15.117` is in CIDR network `172.21.0.0/20` 2024-03-04T15:39:04.162 INFO:teuthology.orchestra.run.smithi117.stdout:Mon IP `172.21.15.117` is in CIDR network `172.21.0.1/32` 2024-03-04T15:39:04.162 INFO:teuthology.orchestra.run.smithi117.stdout:Mon IP `172.21.15.117` is in CIDR network `172.21.0.1/32` 2024-03-04T15:39:04.162 INFO:teuthology.orchestra.run.smithi117.stdout:Mon IP `172.21.15.117` is in CIDR network `172.21.0.2/32` 2024-03-04T15:39:04.162 INFO:teuthology.orchestra.run.smithi117.stdout:Mon IP `172.21.15.117` is in CIDR network `172.21.0.2/32` 2024-03-04T15:39:04.162 INFO:teuthology.orchestra.run.smithi117.stdout:Mon IP `172.21.15.117` is in CIDR network `172.21.15.254/32` 2024-03-04T15:39:04.162 INFO:teuthology.orchestra.run.smithi117.stdout:Mon IP `172.21.15.117` is in CIDR network `172.21.15.254/32` 2024-03-04T15:39:04.162 INFO:teuthology.orchestra.run.smithi117.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20', '172.21.0.1/32', '172.21.0.1/32', '172.21.0.2/32', '172.21.0.2/32', '172.21.15.254/32', '172.21.15.254/32'] 2024-03-04T15:39:04.163 INFO:teuthology.orchestra.run.smithi117.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-03-04T15:39:04.163 INFO:teuthology.orchestra.run.smithi117.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282... 2024-03-04T15:39:04.355 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/docker: stdout 9ca0b5de35ed230e84387abdd8e619ed8e4f3282: Pulling from ceph-ci/ceph 2024-03-04T15:39:04.355 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/docker: stdout Digest: sha256:6d920225840c1daadea71eb62fa66076c3ca6088306e8e7302cea6cb04aef977 2024-03-04T15:39:04.355 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/docker: stdout Status: Image is up to date for quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 2024-03-04T15:39:04.355 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/docker: stdout quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 2024-03-04T15:39:05.805 INFO:teuthology.orchestra.run.smithi117.stdout:ceph: stdout ceph version 19.0.0-1826-g9ca0b5de (9ca0b5de35ed230e84387abdd8e619ed8e4f3282) squid (dev) 2024-03-04T15:39:05.806 INFO:teuthology.orchestra.run.smithi117.stdout:Ceph version: ceph version 19.0.0-1826-g9ca0b5de (9ca0b5de35ed230e84387abdd8e619ed8e4f3282) squid (dev) 2024-03-04T15:39:05.806 INFO:teuthology.orchestra.run.smithi117.stdout:Extracting ceph user uid/gid from container image... 2024-03-04T15:39:07.034 INFO:teuthology.orchestra.run.smithi117.stdout:stat: stdout 167 167 2024-03-04T15:39:07.035 INFO:teuthology.orchestra.run.smithi117.stdout:Creating initial keys... 2024-03-04T15:39:08.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-authtool: stdout AQAb6+VlCI8bLRAAA1L7NkzDZV5Ct/M78LenoA== 2024-03-04T15:39:09.461 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-authtool: stdout AQAc6+VlXmVROBAAuN+2qXQtJHEOgT0cUNT9kQ== 2024-03-04T15:39:10.682 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-authtool: stdout AQAe6+VluSTECBAAxRmX8PWm9xRwv9B0r8FiIQ== 2024-03-04T15:39:10.683 INFO:teuthology.orchestra.run.smithi117.stdout:Creating initial monmap... 2024-03-04T15:39:11.954 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-04T15:39:11.954 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-03-04T15:39:11.954 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T15:39:11.954 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-04T15:39:11.954 INFO:teuthology.orchestra.run.smithi117.stdout:monmaptool for a [v2:172.21.15.117:3300,v1:172.21.15.117:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-03-04T15:39:11.955 INFO:teuthology.orchestra.run.smithi117.stdout:setting min_mon_release = quincy 2024-03-04T15:39:11.955 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: set fsid to e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T15:39:11.955 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-03-04T15:39:11.955 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-04T15:39:11.955 INFO:teuthology.orchestra.run.smithi117.stdout:Creating mon... 2024-03-04T15:39:13.296 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.696+0000 7fe8f306ac80 0 set uid:gid to 167:167 (ceph:ceph) 2024-03-04T15:39:13.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.696+0000 7fe8f306ac80 1 imported monmap: 2024-03-04T15:39:13.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr epoch 0 2024-03-04T15:39:13.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr fsid e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T15:39:13.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr last_changed 2024-03-04T15:39:11.403703+0000 2024-03-04T15:39:13.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr created 2024-03-04T15:39:11.403703+0000 2024-03-04T15:39:13.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr min_mon_release 17 (quincy) 2024-03-04T15:39:13.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr election_strategy: 1 2024-03-04T15:39:13.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 0: [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon.a 2024-03-04T15:39:13.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 2024-03-04T15:39:13.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.696+0000 7fe8f306ac80 0 /usr/bin/ceph-mon: set fsid to e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T15:39:13.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: RocksDB version: 7.9.2 2024-03-04T15:39:13.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 2024-03-04T15:39:13.297 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Git sha 0 2024-03-04T15:39:13.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Compile date 2024-03-04 11:37:16 2024-03-04T15:39:13.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: DB SUMMARY 2024-03-04T15:39:13.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 2024-03-04T15:39:13.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: DB Session ID: CYZ7QWN5BOMESYQ2RHTI 2024-03-04T15:39:13.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 2024-03-04T15:39:13.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 0, files: 2024-03-04T15:39:13.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 2024-03-04T15:39:13.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 2024-03-04T15:39:13.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 2024-03-04T15:39:13.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.error_if_exists: 0 2024-03-04T15:39:13.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.create_if_missing: 1 2024-03-04T15:39:13.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.paranoid_checks: 1 2024-03-04T15:39:13.298 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.flush_verify_memtable_count: 1 2024-03-04T15:39:13.299 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-04T15:39:13.299 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-04T15:39:13.299 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.env: 0x55d009200b20 2024-03-04T15:39:13.299 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.fs: PosixFileSystem 2024-03-04T15:39:13.299 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.info_log: 0x55d00ba0eb40 2024-03-04T15:39:13.299 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_file_opening_threads: 16 2024-03-04T15:39:13.299 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.statistics: (nil) 2024-03-04T15:39:13.299 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.use_fsync: 0 2024-03-04T15:39:13.299 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_log_file_size: 0 2024-03-04T15:39:13.299 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-04T15:39:13.299 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.log_file_time_to_roll: 0 2024-03-04T15:39:13.299 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.keep_log_file_num: 1000 2024-03-04T15:39:13.299 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.recycle_log_file_num: 0 2024-03-04T15:39:13.299 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.allow_fallocate: 1 2024-03-04T15:39:13.300 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.allow_mmap_reads: 0 2024-03-04T15:39:13.300 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.allow_mmap_writes: 0 2024-03-04T15:39:13.300 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.use_direct_reads: 0 2024-03-04T15:39:13.300 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-04T15:39:13.300 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.create_missing_column_families: 0 2024-03-04T15:39:13.300 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.db_log_dir: 2024-03-04T15:39:13.300 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.wal_dir: 2024-03-04T15:39:13.300 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.table_cache_numshardbits: 6 2024-03-04T15:39:13.300 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-03-04T15:39:13.300 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-03-04T15:39:13.300 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-04T15:39:13.300 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-04T15:39:13.300 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-03-04T15:39:13.300 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.advise_random_on_open: 1 2024-03-04T15:39:13.301 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.db_write_buffer_size: 0 2024-03-04T15:39:13.301 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.write_buffer_manager: 0x55d00b9e3360 2024-03-04T15:39:13.301 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-04T15:39:13.301 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-04T15:39:13.301 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.use_adaptive_mutex: 0 2024-03-04T15:39:13.301 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.rate_limiter: (nil) 2024-03-04T15:39:13.301 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-04T15:39:13.301 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.wal_recovery_mode: 2 2024-03-04T15:39:13.301 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.enable_thread_tracking: 0 2024-03-04T15:39:13.301 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.enable_pipelined_write: 0 2024-03-04T15:39:13.301 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.unordered_write: 0 2024-03-04T15:39:13.301 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-04T15:39:13.301 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-04T15:39:13.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-04T15:39:13.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-04T15:39:13.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.row_cache: None 2024-03-04T15:39:13.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.wal_filter: None 2024-03-04T15:39:13.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-04T15:39:13.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.allow_ingest_behind: 0 2024-03-04T15:39:13.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.two_write_queues: 0 2024-03-04T15:39:13.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.manual_wal_flush: 0 2024-03-04T15:39:13.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.wal_compression: 0 2024-03-04T15:39:13.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.atomic_flush: 0 2024-03-04T15:39:13.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-04T15:39:13.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.persist_stats_to_disk: 0 2024-03-04T15:39:13.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-03-04T15:39:13.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.log_readahead_size: 0 2024-03-04T15:39:13.302 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-04T15:39:13.303 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.best_efforts_recovery: 0 2024-03-04T15:39:13.303 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-04T15:39:13.303 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-04T15:39:13.303 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.allow_data_in_errors: 0 2024-03-04T15:39:13.303 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.db_host_id: __hostname__ 2024-03-04T15:39:13.303 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.enforce_single_del_contracts: true 2024-03-04T15:39:13.303 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_background_jobs: 2 2024-03-04T15:39:13.303 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_background_compactions: -1 2024-03-04T15:39:13.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_subcompactions: 1 2024-03-04T15:39:13.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-04T15:39:13.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-04T15:39:13.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.delayed_write_rate : 16777216 2024-03-04T15:39:13.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_total_wal_size: 0 2024-03-04T15:39:13.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-04T15:39:13.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.stats_dump_period_sec: 600 2024-03-04T15:39:13.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.stats_persist_period_sec: 600 2024-03-04T15:39:13.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-04T15:39:13.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_open_files: -1 2024-03-04T15:39:13.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.bytes_per_sync: 0 2024-03-04T15:39:13.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-03-04T15:39:13.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-03-04T15:39:13.304 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compaction_readahead_size: 0 2024-03-04T15:39:13.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_background_flushes: -1 2024-03-04T15:39:13.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Compression algorithms supported: 2024-03-04T15:39:13.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-04T15:39:13.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: kZSTD supported: 0 2024-03-04T15:39:13.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: kXpressCompression supported: 0 2024-03-04T15:39:13.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: kLZ4HCCompression supported: 1 2024-03-04T15:39:13.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: kZlibCompression supported: 1 2024-03-04T15:39:13.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: kSnappyCompression supported: 1 2024-03-04T15:39:13.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: kLZ4Compression supported: 1 2024-03-04T15:39:13.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: kBZip2Compression supported: 0 2024-03-04T15:39:13.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-03-04T15:39:13.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: DMutex implementation: pthread_mutex_t 2024-03-04T15:39:13.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: [db/db_impl/db_impl_open.cc:317] Creating manifest 1 2024-03-04T15:39:13.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 2024-03-04T15:39:13.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 2024-03-04T15:39:13.305 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 2024-03-04T15:39:13.306 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-04T15:39:13.306 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 2024-03-04T15:39:13.306 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-04T15:39:13.306 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.merge_operator: 2024-03-04T15:39:13.306 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compaction_filter: None 2024-03-04T15:39:13.306 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compaction_filter_factory: None 2024-03-04T15:39:13.306 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.sst_partitioner_factory: None 2024-03-04T15:39:13.306 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-03-04T15:39:13.306 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.table_factory: BlockBasedTable 2024-03-04T15:39:13.306 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d00ba0ec60) 2024-03-04T15:39:13.306 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr cache_index_and_filter_blocks: 1 2024-03-04T15:39:13.306 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr cache_index_and_filter_blocks_with_high_priority: 0 2024-03-04T15:39:13.306 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-04T15:39:13.306 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr pin_top_level_index_and_filter: 1 2024-03-04T15:39:13.306 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr index_type: 0 2024-03-04T15:39:13.307 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr data_block_index_type: 0 2024-03-04T15:39:13.307 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr index_shortening: 1 2024-03-04T15:39:13.307 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr data_block_hash_table_util_ratio: 0.750000 2024-03-04T15:39:13.307 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr checksum: 4 2024-03-04T15:39:13.307 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr no_block_cache: 0 2024-03-04T15:39:13.307 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr block_cache: 0x55d00ba15090 2024-03-04T15:39:13.307 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr block_cache_name: BinnedLRUCache 2024-03-04T15:39:13.307 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr block_cache_options: 2024-03-04T15:39:13.307 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr capacity : 536870912 2024-03-04T15:39:13.307 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr num_shard_bits : 4 2024-03-04T15:39:13.307 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr strict_capacity_limit : 0 2024-03-04T15:39:13.307 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr high_pri_pool_ratio: 0.000 2024-03-04T15:39:13.307 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr block_cache_compressed: (nil) 2024-03-04T15:39:13.307 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr persistent_cache: (nil) 2024-03-04T15:39:13.307 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr block_size: 4096 2024-03-04T15:39:13.307 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr block_size_deviation: 10 2024-03-04T15:39:13.308 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr block_restart_interval: 16 2024-03-04T15:39:13.308 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr index_block_restart_interval: 1 2024-03-04T15:39:13.308 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr metadata_block_size: 4096 2024-03-04T15:39:13.308 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr partition_filters: 0 2024-03-04T15:39:13.308 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr use_delta_encoding: 1 2024-03-04T15:39:13.308 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr filter_policy: bloomfilter 2024-03-04T15:39:13.308 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr whole_key_filtering: 1 2024-03-04T15:39:13.308 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr verify_compression: 0 2024-03-04T15:39:13.308 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr read_amp_bytes_per_bit: 0 2024-03-04T15:39:13.308 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr format_version: 5 2024-03-04T15:39:13.308 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr enable_index_compression: 1 2024-03-04T15:39:13.308 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr block_align: 0 2024-03-04T15:39:13.308 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr max_auto_readahead_size: 262144 2024-03-04T15:39:13.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr prepopulate_block_cache: 0 2024-03-04T15:39:13.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr initial_auto_readahead_size: 8192 2024-03-04T15:39:13.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr num_file_reads_for_auto_readahead: 2 2024-03-04T15:39:13.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 2024-03-04T15:39:13.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.write_buffer_size: 33554432 2024-03-04T15:39:13.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_write_buffer_number: 2 2024-03-04T15:39:13.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compression: NoCompression 2024-03-04T15:39:13.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.bottommost_compression: Disabled 2024-03-04T15:39:13.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.prefix_extractor: nullptr 2024-03-04T15:39:13.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-04T15:39:13.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.num_levels: 7 2024-03-04T15:39:13.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-04T15:39:13.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-04T15:39:13.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-04T15:39:13.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-04T15:39:13.309 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-04T15:39:13.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-04T15:39:13.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-04T15:39:13.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-04T15:39:13.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-04T15:39:13.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-04T15:39:13.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-04T15:39:13.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-04T15:39:13.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compression_opts.window_bits: -14 2024-03-04T15:39:13.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compression_opts.level: 32767 2024-03-04T15:39:13.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compression_opts.strategy: 0 2024-03-04T15:39:13.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-04T15:39:13.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-04T15:39:13.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-04T15:39:13.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-04T15:39:13.310 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compression_opts.enabled: false 2024-03-04T15:39:13.311 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-04T15:39:13.311 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-04T15:39:13.311 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-04T15:39:13.311 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-04T15:39:13.311 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.target_file_size_base: 67108864 2024-03-04T15:39:13.311 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.target_file_size_multiplier: 1 2024-03-04T15:39:13.311 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-04T15:39:13.311 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-04T15:39:13.311 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-04T15:39:13.311 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-04T15:39:13.311 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-04T15:39:13.311 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-04T15:39:13.311 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-04T15:39:13.311 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-04T15:39:13.312 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-04T15:39:13.312 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-04T15:39:13.312 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-04T15:39:13.312 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-04T15:39:13.312 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-04T15:39:13.312 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.arena_block_size: 1048576 2024-03-04T15:39:13.312 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-04T15:39:13.312 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-04T15:39:13.312 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.disable_auto_compactions: 0 2024-03-04T15:39:13.312 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-04T15:39:13.312 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-04T15:39:13.312 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-04T15:39:13.312 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-04T15:39:13.312 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-04T15:39:13.313 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-04T15:39:13.313 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-04T15:39:13.313 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-04T15:39:13.313 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-04T15:39:13.313 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-04T15:39:13.313 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.table_properties_collectors: 2024-03-04T15:39:13.313 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.inplace_update_support: 0 2024-03-04T15:39:13.313 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-03-04T15:39:13.313 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-04T15:39:13.313 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-04T15:39:13.313 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.memtable_huge_page_size: 0 2024-03-04T15:39:13.313 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.bloom_locality: 0 2024-03-04T15:39:13.313 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.max_successive_merges: 0 2024-03-04T15:39:13.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-03-04T15:39:13.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.paranoid_file_checks: 0 2024-03-04T15:39:13.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.force_consistency_checks: 1 2024-03-04T15:39:13.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.report_bg_io_stats: 0 2024-03-04T15:39:13.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.ttl: 2592000 2024-03-04T15:39:13.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-03-04T15:39:13.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-04T15:39:13.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-04T15:39:13.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.enable_blob_files: false 2024-03-04T15:39:13.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.min_blob_size: 0 2024-03-04T15:39:13.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.blob_file_size: 268435456 2024-03-04T15:39:13.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.blob_compression_type: NoCompression 2024-03-04T15:39:13.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.enable_blob_garbage_collection: false 2024-03-04T15:39:13.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-04T15:39:13.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-04T15:39:13.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-04T15:39:13.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.blob_file_starting_level: 0 2024-03-04T15:39:13.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-04T15:39:13.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000001 succeeded,manifest_file_number is 1, next_file_number is 3, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-03-04T15:39:13.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 2024-03-04T15:39:13.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-03-04T15:39:13.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 2024-03-04T15:39:13.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 98fb126d-5162-4c02-a3ff-f3cb2bfa7942 2024-03-04T15:39:13.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 2024-03-04T15:39:13.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: [db/version_set.cc:5047] Creating manifest 5 2024-03-04T15:39:13.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 2024-03-04T15:39:13.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d00bae2000 2024-03-04T15:39:13.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8f306ac80 4 rocksdb: DB pointer 0x55d00bad2000 2024-03-04T15:39:13.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8e31db700 4 rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-04T15:39:13.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.700+0000 7fe8e31db700 4 rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-04T15:39:13.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr ** DB Stats ** 2024-03-04T15:39:13.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr Uptime(secs): 0.0 total, 0.0 interval 2024-03-04T15:39:13.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-04T15:39:13.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-04T15:39:13.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-04T15:39:13.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-04T15:39:13.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-04T15:39:13.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-04T15:39:13.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 2024-03-04T15:39:13.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr ** Compaction Stats [default] ** 2024-03-04T15:39:13.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-04T15:39:13.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-04T15:39:13.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr Sum 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-04T15:39:13.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 0.0 0.0 2024-03-04T15:39:13.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 2024-03-04T15:39:13.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr ** Compaction Stats [default] ** 2024-03-04T15:39:13.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-04T15:39:13.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-04T15:39:13.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 2024-03-04T15:39:13.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-04T15:39:13.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 2024-03-04T15:39:13.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr Uptime(secs): 0.0 total, 0.0 interval 2024-03-04T15:39:13.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr Flush(GB): cumulative 0.000, interval 0.000 2024-03-04T15:39:13.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr AddFile(GB): cumulative 0.000, interval 0.000 2024-03-04T15:39:13.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr AddFile(Total Files): cumulative 0, interval 0 2024-03-04T15:39:13.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr AddFile(L0 Files): cumulative 0, interval 0 2024-03-04T15:39:13.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr AddFile(Keys): cumulative 0, interval 0 2024-03-04T15:39:13.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr Cumulative compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T15:39:13.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T15:39:13.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-04T15:39:13.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr Block cache BinnedLRUCache@0x55d00ba15090#7 capacity: 512.00 MB usage: 0.00 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.7e-05 secs_since: 0 2024-03-04T15:39:13.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr Block cache entry stats(count,size,portion): Misc(1,0.00 KB,0%) 2024-03-04T15:39:13.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 2024-03-04T15:39:13.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr ** File Read Latency Histogram By Level [default] ** 2024-03-04T15:39:13.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr 2024-03-04T15:39:13.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.704+0000 7fe8f306ac80 4 rocksdb: [db/db_impl/db_impl.cc:496] Shutdown: canceling all background work 2024-03-04T15:39:13.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.704+0000 7fe8f306ac80 4 rocksdb: [db/db_impl/db_impl.cc:704] Shutdown complete 2024-03-04T15:39:13.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph-mon: stderr debug 2024-03-04T15:39:12.704+0000 7fe8f306ac80 0 /usr/bin/ceph-mon: created monfs at /var/lib/ceph/mon/ceph-a for mon.a 2024-03-04T15:39:13.318 INFO:teuthology.orchestra.run.smithi117.stdout:create mon.a on 2024-03-04T15:39:13.577 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2024-03-04T15:39:13.793 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /etc/systemd/system/ceph.target. 2024-03-04T15:39:14.023 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-e40b5504-da3c-11ee-95c6-87774f69a715.target -> /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715.target. 2024-03-04T15:39:14.023 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-e40b5504-da3c-11ee-95c6-87774f69a715.target -> /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715.target. 2024-03-04T15:39:14.285 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mon.a 2024-03-04T15:39:14.285 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mon.a.service: Unit ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mon.a.service not loaded. 2024-03-04T15:39:14.482 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715.target.wants/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mon.a.service -> /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service. 2024-03-04T15:39:14.492 INFO:teuthology.orchestra.run.smithi117.stdout:firewalld does not appear to be present 2024-03-04T15:39:14.493 INFO:teuthology.orchestra.run.smithi117.stdout:Not possible to enable service . firewalld.service is not available 2024-03-04T15:39:14.493 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mon to start... 2024-03-04T15:39:14.493 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mon... 2024-03-04T15:39:14.532 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:14 smithi117 systemd[1]: Started Ceph mon.a for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T15:39:15.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:15 smithi117 bash[20996]: cluster 2024-03-04T15:39:15.691785+0000 mon.a (mon.0) 0 : cluster 1 mkfs e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T15:39:15.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:15 smithi117 bash[20996]: cluster 2024-03-04T15:39:15.687689+0000 mon.a (mon.0) 1 : cluster 1 mon.a is new leader, mons a in quorum (ranks 0) 2024-03-04T15:39:16.451 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout cluster: 2024-03-04T15:39:16.451 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout id: e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T15:39:16.451 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-03-04T15:39:16.451 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-03-04T15:39:16.451 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout services: 2024-03-04T15:39:16.451 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.188586s) 2024-03-04T15:39:16.452 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-03-04T15:39:16.452 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-03-04T15:39:16.452 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-03-04T15:39:16.452 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout data: 2024-03-04T15:39:16.452 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-03-04T15:39:16.452 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-03-04T15:39:16.452 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-03-04T15:39:16.452 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout pgs: 2024-03-04T15:39:16.452 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-03-04T15:39:16.452 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.736+0000 7fa06517a700 1 Processor -- start 2024-03-04T15:39:16.452 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.736+0000 7fa06517a700 1 -- start start 2024-03-04T15:39:16.452 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.736+0000 7fa06517a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa060105c60 0x7fa060106080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:16.452 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.736+0000 7fa06517a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa060106650 con 0x7fa060105c60 2024-03-04T15:39:16.453 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.736+0000 7fa05ed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa060105c60 0x7fa060106080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:16.453 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.736+0000 7fa05ed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa060105c60 0x7fa060106080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40418/0 (socket says 172.21.15.117:40418) 2024-03-04T15:39:16.453 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa05ed9d700 1 -- 172.21.15.117:0/3800703932 learned_addr learned my addr 172.21.15.117:0/3800703932 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:16.453 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa05ed9d700 1 -- 172.21.15.117:0/3800703932 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa060106e60 con 0x7fa060105c60 2024-03-04T15:39:16.453 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa05ed9d700 1 --2- 172.21.15.117:0/3800703932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa060105c60 0x7fa060106080 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa058009e80 tx=0x7fa05800b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c48611b8e9d5e2e7 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:16.453 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa05dd9b700 1 -- 172.21.15.117:0/3800703932 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa058017070 con 0x7fa060105c60 2024-03-04T15:39:16.453 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa05dd9b700 1 -- 172.21.15.117:0/3800703932 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa058003a20 con 0x7fa060105c60 2024-03-04T15:39:16.453 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa05dd9b700 1 -- 172.21.15.117:0/3800703932 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa058003d20 con 0x7fa060105c60 2024-03-04T15:39:16.453 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa06517a700 1 -- 172.21.15.117:0/3800703932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa060105c60 msgr2=0x7fa060106080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:16.453 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa06517a700 1 --2- 172.21.15.117:0/3800703932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa060105c60 0x7fa060106080 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa058009e80 tx=0x7fa05800b330 comp rx=0 tx=0).stop 2024-03-04T15:39:16.453 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa06517a700 1 -- 172.21.15.117:0/3800703932 shutdown_connections 2024-03-04T15:39:16.453 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa06517a700 1 --2- 172.21.15.117:0/3800703932 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa060105c60 0x7fa060106080 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:16.453 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa06517a700 1 -- 172.21.15.117:0/3800703932 >> 172.21.15.117:0/3800703932 conn(0x7fa060101220 msgr2=0x7fa060103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:16.453 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa06517a700 1 -- 172.21.15.117:0/3800703932 shutdown_connections 2024-03-04T15:39:16.453 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa06517a700 1 -- 172.21.15.117:0/3800703932 wait complete. 2024-03-04T15:39:16.454 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa06517a700 1 Processor -- start 2024-03-04T15:39:16.454 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa06517a700 1 -- start start 2024-03-04T15:39:16.454 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa06517a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa060105c60 0x7fa06019b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:16.454 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa06517a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa060107390 con 0x7fa060105c60 2024-03-04T15:39:16.454 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa05ed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa060105c60 0x7fa06019b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:16.454 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa05ed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa060105c60 0x7fa06019b520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:40422/0 (socket says 172.21.15.117:40422) 2024-03-04T15:39:16.454 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa05ed9d700 1 -- 172.21.15.117:0/3806865371 learned_addr learned my addr 172.21.15.117:0/3806865371 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:16.454 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa05ed9d700 1 -- 172.21.15.117:0/3806865371 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06019ba60 con 0x7fa060105c60 2024-03-04T15:39:16.454 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa05ed9d700 1 --2- 172.21.15.117:0/3806865371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa060105c60 0x7fa06019b520 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa058004000 tx=0x7fa058004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:16.454 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa057fff700 1 -- 172.21.15.117:0/3806865371 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa058022020 con 0x7fa060105c60 2024-03-04T15:39:16.454 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa057fff700 1 -- 172.21.15.117:0/3806865371 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa058018040 con 0x7fa060105c60 2024-03-04T15:39:16.454 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa06517a700 1 -- 172.21.15.117:0/3806865371 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa06019bcf0 con 0x7fa060105c60 2024-03-04T15:39:16.454 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa057fff700 1 -- 172.21.15.117:0/3806865371 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa058017070 con 0x7fa060105c60 2024-03-04T15:39:16.454 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa06517a700 1 -- 172.21.15.117:0/3806865371 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa06019e9e0 con 0x7fa060105c60 2024-03-04T15:39:16.454 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa057fff700 1 -- 172.21.15.117:0/3806865371 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fa058025020 con 0x7fa060105c60 2024-03-04T15:39:16.454 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.740+0000 7fa057fff700 1 -- 172.21.15.117:0/3806865371 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa05800f050 con 0x7fa060105c60 2024-03-04T15:39:16.455 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.744+0000 7fa06517a700 1 -- 172.21.15.117:0/3806865371 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa040005150 con 0x7fa060105c60 2024-03-04T15:39:16.455 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.744+0000 7fa057fff700 1 -- 172.21.15.117:0/3806865371 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fa05802c030 con 0x7fa060105c60 2024-03-04T15:39:16.455 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.876+0000 7fa06517a700 1 -- 172.21.15.117:0/3806865371 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7fa040005710 con 0x7fa060105c60 2024-03-04T15:39:16.455 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.876+0000 7fa057fff700 1 -- 172.21.15.117:0/3806865371 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7fa05801e480 con 0x7fa060105c60 2024-03-04T15:39:16.455 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.876+0000 7fa06517a700 1 -- 172.21.15.117:0/3806865371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa060105c60 msgr2=0x7fa06019b520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:16.455 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.876+0000 7fa06517a700 1 --2- 172.21.15.117:0/3806865371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa060105c60 0x7fa06019b520 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa058004000 tx=0x7fa058004ad0 comp rx=0 tx=0).stop 2024-03-04T15:39:16.455 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.876+0000 7fa06517a700 1 -- 172.21.15.117:0/3806865371 shutdown_connections 2024-03-04T15:39:16.455 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.876+0000 7fa06517a700 1 --2- 172.21.15.117:0/3806865371 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa060105c60 0x7fa06019b520 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:16.455 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.876+0000 7fa06517a700 1 -- 172.21.15.117:0/3806865371 >> 172.21.15.117:0/3806865371 conn(0x7fa060101220 msgr2=0x7fa06007ab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:16.455 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.876+0000 7fa06517a700 1 -- 172.21.15.117:0/3806865371 shutdown_connections 2024-03-04T15:39:16.455 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:15.880+0000 7fa06517a700 1 -- 172.21.15.117:0/3806865371 wait complete. 2024-03-04T15:39:16.455 INFO:teuthology.orchestra.run.smithi117.stdout:mon is available 2024-03-04T15:39:16.455 INFO:teuthology.orchestra.run.smithi117.stdout:Assimilating anything we can from ceph.conf... 2024-03-04T15:39:17.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:16 smithi117 bash[20996]: cluster 2024-03-04T15:39:15.690915+0000 mon.a (mon.0) 2 : cluster 1 mon.a is new leader, mons a in quorum (ranks 0) 2024-03-04T15:39:17.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:16 smithi117 bash[20996]: cluster 2024-03-04T15:39:15.691091+0000 mon.a (mon.0) 3 : cluster 0 monmap e1: 1 mons at {a=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T15:39:17.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:16 smithi117 bash[20996]: cluster 2024-03-04T15:39:15.693073+0000 mon.a (mon.0) 4 : cluster 0 fsmap 2024-03-04T15:39:17.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:16 smithi117 bash[20996]: cluster 2024-03-04T15:39:15.695428+0000 mon.a (mon.0) 5 : cluster 0 osdmap e1: 0 total, 0 up, 0 in 2024-03-04T15:39:17.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:16 smithi117 bash[20996]: cluster 2024-03-04T15:39:15.695870+0000 mon.a (mon.0) 6 : cluster 0 mgrmap e1: no daemons active 2024-03-04T15:39:17.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:16 smithi117 bash[20996]: audit 2024-03-04T15:39:15.879467+0000 mon.a (mon.0) 7 : audit 0 from='client.? 172.21.15.117:0/3806865371' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-03-04T15:39:17.988 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-03-04T15:39:17.988 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [global] 2024-03-04T15:39:17.988 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout fsid = e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T15:39:17.988 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.117:3300,v1:172.21.15.117:6789] 2024-03-04T15:39:17.988 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-04T15:39:17.988 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-04T15:39:17.988 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-04T15:39:17.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-04T15:39:17.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-03-04T15:39:17.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-04T15:39:17.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-04T15:39:17.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-03-04T15:39:17.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [osd] 2024-03-04T15:39:17.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-04T15:39:17.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-04T15:39:17.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.264+0000 7f50eeef0700 1 Processor -- start 2024-03-04T15:39:17.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.264+0000 7f50eeef0700 1 -- start start 2024-03-04T15:39:17.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.264+0000 7f50eeef0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e8105c60 0x7f50e8106080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:17.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.264+0000 7f50eeef0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e8106650 con 0x7f50e8105c60 2024-03-04T15:39:17.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.264+0000 7f50ecc8c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e8105c60 0x7f50e8106080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:17.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.264+0000 7f50ecc8c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e8105c60 0x7f50e8106080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41316/0 (socket says 172.21.15.117:41316) 2024-03-04T15:39:17.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.264+0000 7f50ecc8c700 1 -- 172.21.15.117:0/2009903278 learned_addr learned my addr 172.21.15.117:0/2009903278 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:17.989 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.264+0000 7f50ecc8c700 1 -- 172.21.15.117:0/2009903278 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e8106e60 con 0x7f50e8105c60 2024-03-04T15:39:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.264+0000 7f50ecc8c700 1 --2- 172.21.15.117:0/2009903278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e8105c60 0x7f50e8106080 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f50d80098a0 tx=0x7f50d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e624eb056afd3cf server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.264+0000 7f50e77fe700 1 -- 172.21.15.117:0/2009903278 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50d8004160 con 0x7f50e8105c60 2024-03-04T15:39:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.264+0000 7f50e77fe700 1 -- 172.21.15.117:0/2009903278 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d8004300 con 0x7f50e8105c60 2024-03-04T15:39:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.264+0000 7f50e77fe700 1 -- 172.21.15.117:0/2009903278 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50d80036a0 con 0x7f50e8105c60 2024-03-04T15:39:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50eeef0700 1 -- 172.21.15.117:0/2009903278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e8105c60 msgr2=0x7f50e8106080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50eeef0700 1 --2- 172.21.15.117:0/2009903278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e8105c60 0x7f50e8106080 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f50d80098a0 tx=0x7f50d8009bb0 comp rx=0 tx=0).stop 2024-03-04T15:39:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50eeef0700 1 -- 172.21.15.117:0/2009903278 shutdown_connections 2024-03-04T15:39:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50eeef0700 1 --2- 172.21.15.117:0/2009903278 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e8105c60 0x7f50e8106080 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50eeef0700 1 -- 172.21.15.117:0/2009903278 >> 172.21.15.117:0/2009903278 conn(0x7f50e81011c0 msgr2=0x7f50e8103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50eeef0700 1 -- 172.21.15.117:0/2009903278 shutdown_connections 2024-03-04T15:39:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50eeef0700 1 -- 172.21.15.117:0/2009903278 wait complete. 2024-03-04T15:39:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50eeef0700 1 Processor -- start 2024-03-04T15:39:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50eeef0700 1 -- start start 2024-03-04T15:39:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50eeef0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e8105c60 0x7f50e807b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50eeef0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e8107390 con 0x7f50e8105c60 2024-03-04T15:39:17.990 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50ecc8c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e8105c60 0x7f50e807b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:17.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50ecc8c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e8105c60 0x7f50e807b3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41330/0 (socket says 172.21.15.117:41330) 2024-03-04T15:39:17.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50ecc8c700 1 -- 172.21.15.117:0/1379511070 learned_addr learned my addr 172.21.15.117:0/1379511070 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:17.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50ecc8c700 1 -- 172.21.15.117:0/1379511070 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e807b8e0 con 0x7f50e8105c60 2024-03-04T15:39:17.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50ecc8c700 1 --2- 172.21.15.117:0/1379511070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e8105c60 0x7f50e807b3a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f50d8000c00 tx=0x7f50d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:17.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50e5ffb700 1 -- 172.21.15.117:0/1379511070 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50d8003db0 con 0x7f50e8105c60 2024-03-04T15:39:17.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50e5ffb700 1 -- 172.21.15.117:0/1379511070 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d8011630 con 0x7f50e8105c60 2024-03-04T15:39:17.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50eeef0700 1 -- 172.21.15.117:0/1379511070 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e8079b20 con 0x7f50e8105c60 2024-03-04T15:39:17.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50e5ffb700 1 -- 172.21.15.117:0/1379511070 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f50d8011950 con 0x7f50e8105c60 2024-03-04T15:39:17.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50eeef0700 1 -- 172.21.15.117:0/1379511070 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e807a000 con 0x7f50e8105c60 2024-03-04T15:39:17.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50e5ffb700 1 -- 172.21.15.117:0/1379511070 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f50d801a460 con 0x7f50e8105c60 2024-03-04T15:39:17.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50e5ffb700 1 -- 172.21.15.117:0/1379511070 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f50d801ba50 con 0x7f50e8105c60 2024-03-04T15:39:17.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.268+0000 7f50eeef0700 1 -- 172.21.15.117:0/1379511070 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e8063410 con 0x7f50e8105c60 2024-03-04T15:39:17.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.272+0000 7f50e5ffb700 1 -- 172.21.15.117:0/1379511070 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f50d8031020 con 0x7f50e8105c60 2024-03-04T15:39:17.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.392+0000 7f50eeef0700 1 -- 172.21.15.117:0/1379511070 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f50e8192910 con 0x7f50e8105c60 2024-03-04T15:39:17.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.396+0000 7f50e5ffb700 1 -- 172.21.15.117:0/1379511070 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+340 (secure 0 0 0) 0x7f50d8004260 con 0x7f50e8105c60 2024-03-04T15:39:17.991 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.400+0000 7f50eeef0700 1 -- 172.21.15.117:0/1379511070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e8105c60 msgr2=0x7f50e807b3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:17.992 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.400+0000 7f50eeef0700 1 --2- 172.21.15.117:0/1379511070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e8105c60 0x7f50e807b3a0 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f50d8000c00 tx=0x7f50d8004750 comp rx=0 tx=0).stop 2024-03-04T15:39:17.992 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.400+0000 7f50eeef0700 1 -- 172.21.15.117:0/1379511070 shutdown_connections 2024-03-04T15:39:17.992 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.400+0000 7f50eeef0700 1 --2- 172.21.15.117:0/1379511070 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f50e8105c60 0x7f50e807b3a0 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:17.992 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.400+0000 7f50eeef0700 1 -- 172.21.15.117:0/1379511070 >> 172.21.15.117:0/1379511070 conn(0x7f50e81011c0 msgr2=0x7f50e818ff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:17.992 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.400+0000 7f50eeef0700 1 -- 172.21.15.117:0/1379511070 shutdown_connections 2024-03-04T15:39:17.992 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:17.400+0000 7f50eeef0700 1 -- 172.21.15.117:0/1379511070 wait complete. 2024-03-04T15:39:17.992 INFO:teuthology.orchestra.run.smithi117.stdout:Generating new minimal ceph.conf... 2024-03-04T15:39:18.008 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:17 smithi117 bash[20996]: audit 2024-03-04T15:39:17.396267+0000 mon.a (mon.0) 8 : audit 1 from='client.? 172.21.15.117:0/1379511070' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-04T15:39:18.008 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:17 smithi117 bash[20996]: audit 2024-03-04T15:39:17.400259+0000 mon.a (mon.0) 9 : audit 1 from='client.? 172.21.15.117:0/1379511070' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-03-04T15:39:19.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:19 smithi117 bash[20996]: audit 2024-03-04T15:39:18.984920+0000 mon.a (mon.0) 10 : audit 0 from='client.? 172.21.15.117:0/1929205358' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:39:19.558 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.852+0000 7fc88621c700 1 Processor -- start 2024-03-04T15:39:19.558 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.852+0000 7fc88621c700 1 -- start start 2024-03-04T15:39:19.558 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.852+0000 7fc88621c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc88007b760 0x7fc880079bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:19.558 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.852+0000 7fc88621c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc88007a190 con 0x7fc88007b760 2024-03-04T15:39:19.558 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc87f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc88007b760 0x7fc880079bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:19.558 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc87f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc88007b760 0x7fc880079bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41336/0 (socket says 172.21.15.117:41336) 2024-03-04T15:39:19.558 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc87f7fe700 1 -- 172.21.15.117:0/1144569259 learned_addr learned my addr 172.21.15.117:0/1144569259 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:19.558 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc87f7fe700 1 -- 172.21.15.117:0/1144569259 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88007a310 con 0x7fc88007b760 2024-03-04T15:39:19.558 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc87f7fe700 1 --2- 172.21.15.117:0/1144569259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc88007b760 0x7fc880079bc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc870009e80 tx=0x7fc87000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e38225ad5cea2682 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:19.558 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc87e7fc700 1 -- 172.21.15.117:0/1144569259 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc870017070 con 0x7fc88007b760 2024-03-04T15:39:19.558 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc87e7fc700 1 -- 172.21.15.117:0/1144569259 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fc87000b8c0 con 0x7fc88007b760 2024-03-04T15:39:19.558 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc87e7fc700 1 -- 172.21.15.117:0/1144569259 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc870004370 con 0x7fc88007b760 2024-03-04T15:39:19.559 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc88621c700 1 -- 172.21.15.117:0/1144569259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc88007b760 msgr2=0x7fc880079bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:19.559 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc88621c700 1 --2- 172.21.15.117:0/1144569259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc88007b760 0x7fc880079bc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc870009e80 tx=0x7fc87000b330 comp rx=0 tx=0).stop 2024-03-04T15:39:19.559 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc88621c700 1 -- 172.21.15.117:0/1144569259 shutdown_connections 2024-03-04T15:39:19.559 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc88621c700 1 --2- 172.21.15.117:0/1144569259 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc88007b760 0x7fc880079bc0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:19.559 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc88621c700 1 -- 172.21.15.117:0/1144569259 >> 172.21.15.117:0/1144569259 conn(0x7fc8801011f0 msgr2=0x7fc880103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:19.559 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc88621c700 1 -- 172.21.15.117:0/1144569259 shutdown_connections 2024-03-04T15:39:19.559 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc88621c700 1 -- 172.21.15.117:0/1144569259 wait complete. 2024-03-04T15:39:19.559 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc88621c700 1 Processor -- start 2024-03-04T15:39:19.559 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc88621c700 1 -- start start 2024-03-04T15:39:19.559 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc88621c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc88007b760 0x7fc88019ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:19.559 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc88621c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc880107c90 con 0x7fc88007b760 2024-03-04T15:39:19.559 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc87f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc88007b760 0x7fc88019ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:19.559 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc87f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc88007b760 0x7fc88019ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41348/0 (socket says 172.21.15.117:41348) 2024-03-04T15:39:19.559 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc87f7fe700 1 -- 172.21.15.117:0/1929205358 learned_addr learned my addr 172.21.15.117:0/1929205358 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:19.559 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc87f7fe700 1 -- 172.21.15.117:0/1929205358 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8801a0500 con 0x7fc88007b760 2024-03-04T15:39:19.559 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc87f7fe700 1 --2- 172.21.15.117:0/1929205358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc88007b760 0x7fc88019ffc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc870004000 tx=0x7fc870003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:19.560 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc87cff9700 1 -- 172.21.15.117:0/1929205358 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc870020020 con 0x7fc88007b760 2024-03-04T15:39:19.560 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc87cff9700 1 -- 172.21.15.117:0/1929205358 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fc870012890 con 0x7fc88007b760 2024-03-04T15:39:19.560 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.856+0000 7fc88621c700 1 -- 172.21.15.117:0/1929205358 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8801a0790 con 0x7fc88007b760 2024-03-04T15:39:19.560 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.860+0000 7fc87cff9700 1 -- 172.21.15.117:0/1929205358 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc870017040 con 0x7fc88007b760 2024-03-04T15:39:19.560 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.860+0000 7fc88621c700 1 -- 172.21.15.117:0/1929205358 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8801a3480 con 0x7fc88007b760 2024-03-04T15:39:19.560 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.860+0000 7fc87cff9700 1 -- 172.21.15.117:0/1929205358 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7fc870023020 con 0x7fc88007b760 2024-03-04T15:39:19.560 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.860+0000 7fc87cff9700 1 -- 172.21.15.117:0/1929205358 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc87000f050 con 0x7fc88007b760 2024-03-04T15:39:19.560 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.860+0000 7fc88621c700 1 -- 172.21.15.117:0/1929205358 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc880063410 con 0x7fc88007b760 2024-03-04T15:39:19.560 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.860+0000 7fc87cff9700 1 -- 172.21.15.117:0/1929205358 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7fc87002a020 con 0x7fc88007b760 2024-03-04T15:39:19.560 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.980+0000 7fc88621c700 1 -- 172.21.15.117:0/1929205358 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fc88002d8d0 con 0x7fc88007b760 2024-03-04T15:39:19.560 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.984+0000 7fc87cff9700 1 -- 172.21.15.117:0/1929205358 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7fc87001cdc0 con 0x7fc88007b760 2024-03-04T15:39:19.560 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.984+0000 7fc88621c700 1 -- 172.21.15.117:0/1929205358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc88007b760 msgr2=0x7fc88019ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:19.560 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.984+0000 7fc88621c700 1 --2- 172.21.15.117:0/1929205358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc88007b760 0x7fc88019ffc0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc870004000 tx=0x7fc870003c50 comp rx=0 tx=0).stop 2024-03-04T15:39:19.560 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.984+0000 7fc88621c700 1 -- 172.21.15.117:0/1929205358 shutdown_connections 2024-03-04T15:39:19.560 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.984+0000 7fc88621c700 1 --2- 172.21.15.117:0/1929205358 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc88007b760 0x7fc88019ffc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:19.560 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.984+0000 7fc88621c700 1 -- 172.21.15.117:0/1929205358 >> 172.21.15.117:0/1929205358 conn(0x7fc8801011f0 msgr2=0x7fc880196f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:19.561 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.984+0000 7fc88621c700 1 -- 172.21.15.117:0/1929205358 shutdown_connections 2024-03-04T15:39:19.561 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:18.984+0000 7fc88621c700 1 -- 172.21.15.117:0/1929205358 wait complete. 2024-03-04T15:39:19.561 INFO:teuthology.orchestra.run.smithi117.stdout:Restarting the monitor... 2024-03-04T15:39:19.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:19 smithi117 systemd[1]: Stopping Ceph mon.a for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T15:39:19.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:19 smithi117 bash[20996]: debug 2024-03-04T15:39:19.684+0000 7f1212633700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-03-04T15:39:19.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:19 smithi117 bash[20996]: debug 2024-03-04T15:39:19.684+0000 7f1212633700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-03-04T15:39:20.421 INFO:teuthology.orchestra.run.smithi117.stdout:Setting public_network to 172.21.0.1/32,172.21.0.2/32,172.21.0.0/20,172.21.15.254/32 in mon config section 2024-03-04T15:39:20.462 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:20 smithi117 bash[21381]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-mon-a 2024-03-04T15:39:20.462 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:20 smithi117 bash[21443]: Error response from daemon: No such container: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-mon-a 2024-03-04T15:39:20.462 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:20 smithi117 systemd[1]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-04T15:39:20.462 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:20 smithi117 systemd[1]: Stopped Ceph mon.a for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T15:39:20.462 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:20 smithi117 systemd[1]: Started Ceph mon.a for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T15:39:21.762 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.516+0000 7efe16117c80 0 set uid:gid to 167:167 (ceph:ceph) 2024-03-04T15:39:21.762 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.516+0000 7efe16117c80 0 ceph version 19.0.0-1826-g9ca0b5de (9ca0b5de35ed230e84387abdd8e619ed8e4f3282) squid (dev), process ceph-mon, pid 7 2024-03-04T15:39:21.762 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.516+0000 7efe16117c80 0 pidfile_write: ignore empty --pid-file 2024-03-04T15:39:21.762 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 0 load: jerasure load: lrc 2024-03-04T15:39:21.762 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: RocksDB version: 7.9.2 2024-03-04T15:39:21.762 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Git sha 0 2024-03-04T15:39:21.762 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Compile date 2024-03-04 11:37:16 2024-03-04T15:39:21.763 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: DB SUMMARY 2024-03-04T15:39:21.763 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: DB Session ID: 2C3U6B506RFJ1JY3D6YZ 2024-03-04T15:39:21.763 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: CURRENT file: CURRENT 2024-03-04T15:39:21.763 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: IDENTITY file: IDENTITY 2024-03-04T15:39:21.763 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-03-04T15:39:21.763 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-03-04T15:39:21.763 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 86835 ; 2024-03-04T15:39:21.763 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.error_if_exists: 0 2024-03-04T15:39:21.763 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.create_if_missing: 0 2024-03-04T15:39:21.763 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.paranoid_checks: 1 2024-03-04T15:39:21.763 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.flush_verify_memtable_count: 1 2024-03-04T15:39:21.763 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-03-04T15:39:21.763 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-03-04T15:39:21.763 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.env: 0x559c23d92b20 2024-03-04T15:39:21.763 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.fs: PosixFileSystem 2024-03-04T15:39:21.764 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.info_log: 0x559c262cf920 2024-03-04T15:39:21.764 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_file_opening_threads: 16 2024-03-04T15:39:21.764 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.statistics: (nil) 2024-03-04T15:39:21.764 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.use_fsync: 0 2024-03-04T15:39:21.764 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_log_file_size: 0 2024-03-04T15:39:21.764 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_manifest_file_size: 1073741824 2024-03-04T15:39:21.764 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.log_file_time_to_roll: 0 2024-03-04T15:39:21.764 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.keep_log_file_num: 1000 2024-03-04T15:39:21.764 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.recycle_log_file_num: 0 2024-03-04T15:39:21.764 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.allow_fallocate: 1 2024-03-04T15:39:21.764 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.allow_mmap_reads: 0 2024-03-04T15:39:21.764 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.allow_mmap_writes: 0 2024-03-04T15:39:21.764 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.use_direct_reads: 0 2024-03-04T15:39:21.764 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-03-04T15:39:21.765 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.create_missing_column_families: 0 2024-03-04T15:39:21.765 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.db_log_dir: 2024-03-04T15:39:21.765 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.wal_dir: 2024-03-04T15:39:21.765 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.table_cache_numshardbits: 6 2024-03-04T15:39:21.765 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.WAL_ttl_seconds: 0 2024-03-04T15:39:21.765 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.WAL_size_limit_MB: 0 2024-03-04T15:39:21.765 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-03-04T15:39:21.765 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.manifest_preallocation_size: 4194304 2024-03-04T15:39:21.765 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.is_fd_close_on_exec: 1 2024-03-04T15:39:21.765 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.advise_random_on_open: 1 2024-03-04T15:39:21.765 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.db_write_buffer_size: 0 2024-03-04T15:39:21.765 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.write_buffer_manager: 0x559c262a3400 2024-03-04T15:39:21.765 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.access_hint_on_compaction_start: 1 2024-03-04T15:39:21.765 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.random_access_max_buffer_size: 1048576 2024-03-04T15:39:21.765 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.use_adaptive_mutex: 0 2024-03-04T15:39:21.765 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.rate_limiter: (nil) 2024-03-04T15:39:21.766 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-03-04T15:39:21.766 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.wal_recovery_mode: 2 2024-03-04T15:39:21.767 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.enable_thread_tracking: 0 2024-03-04T15:39:21.767 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.enable_pipelined_write: 0 2024-03-04T15:39:21.767 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.unordered_write: 0 2024-03-04T15:39:21.767 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.allow_concurrent_memtable_write: 1 2024-03-04T15:39:21.767 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-03-04T15:39:21.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.write_thread_max_yield_usec: 100 2024-03-04T15:39:21.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.write_thread_slow_yield_usec: 3 2024-03-04T15:39:21.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.row_cache: None 2024-03-04T15:39:21.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.wal_filter: None 2024-03-04T15:39:21.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.avoid_flush_during_recovery: 0 2024-03-04T15:39:21.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.allow_ingest_behind: 0 2024-03-04T15:39:21.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.two_write_queues: 0 2024-03-04T15:39:21.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.manual_wal_flush: 0 2024-03-04T15:39:21.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.wal_compression: 0 2024-03-04T15:39:21.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.atomic_flush: 0 2024-03-04T15:39:21.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-03-04T15:39:21.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.persist_stats_to_disk: 0 2024-03-04T15:39:21.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.write_dbid_to_manifest: 0 2024-03-04T15:39:21.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.log_readahead_size: 0 2024-03-04T15:39:21.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.file_checksum_gen_factory: Unknown 2024-03-04T15:39:21.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.best_efforts_recovery: 0 2024-03-04T15:39:21.769 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-03-04T15:39:21.769 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-03-04T15:39:21.769 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.allow_data_in_errors: 0 2024-03-04T15:39:21.769 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.db_host_id: __hostname__ 2024-03-04T15:39:21.769 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.enforce_single_del_contracts: true 2024-03-04T15:39:21.769 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_background_jobs: 2 2024-03-04T15:39:21.769 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_background_compactions: -1 2024-03-04T15:39:21.769 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_subcompactions: 1 2024-03-04T15:39:21.769 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.avoid_flush_during_shutdown: 0 2024-03-04T15:39:21.769 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-03-04T15:39:21.769 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.delayed_write_rate : 16777216 2024-03-04T15:39:21.769 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_total_wal_size: 0 2024-03-04T15:39:21.769 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-03-04T15:39:21.769 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.stats_dump_period_sec: 600 2024-03-04T15:39:21.769 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.stats_persist_period_sec: 600 2024-03-04T15:39:21.769 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.stats_history_buffer_size: 1048576 2024-03-04T15:39:21.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_open_files: -1 2024-03-04T15:39:21.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.bytes_per_sync: 0 2024-03-04T15:39:21.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.wal_bytes_per_sync: 0 2024-03-04T15:39:21.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.strict_bytes_per_sync: 0 2024-03-04T15:39:21.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compaction_readahead_size: 0 2024-03-04T15:39:21.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_background_flushes: -1 2024-03-04T15:39:21.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Compression algorithms supported: 2024-03-04T15:39:21.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: kZSTDNotFinalCompression supported: 0 2024-03-04T15:39:21.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: kZSTD supported: 0 2024-03-04T15:39:21.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: kXpressCompression supported: 0 2024-03-04T15:39:21.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: kLZ4HCCompression supported: 1 2024-03-04T15:39:21.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: kZlibCompression supported: 1 2024-03-04T15:39:21.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: kSnappyCompression supported: 1 2024-03-04T15:39:21.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: kLZ4Compression supported: 1 2024-03-04T15:39:21.770 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: kBZip2Compression supported: 0 2024-03-04T15:39:21.771 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Fast CRC32 supported: Supported on x86 2024-03-04T15:39:21.771 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: DMutex implementation: pthread_mutex_t 2024-03-04T15:39:21.771 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-03-04T15:39:21.771 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-03-04T15:39:21.771 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-03-04T15:39:21.771 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.merge_operator: 2024-03-04T15:39:21.771 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compaction_filter: None 2024-03-04T15:39:21.771 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compaction_filter_factory: None 2024-03-04T15:39:21.771 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.sst_partitioner_factory: None 2024-03-04T15:39:21.771 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.memtable_factory: SkipListFactory 2024-03-04T15:39:21.771 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.table_factory: BlockBasedTable 2024-03-04T15:39:21.771 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x559c262cfa20) 2024-03-04T15:39:21.771 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: cache_index_and_filter_blocks: 1 2024-03-04T15:39:21.771 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: cache_index_and_filter_blocks_with_high_priority: 0 2024-03-04T15:39:21.771 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: pin_l0_filter_and_index_blocks_in_cache: 0 2024-03-04T15:39:21.772 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: pin_top_level_index_and_filter: 1 2024-03-04T15:39:21.772 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: index_type: 0 2024-03-04T15:39:21.772 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: data_block_index_type: 0 2024-03-04T15:39:21.772 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: index_shortening: 1 2024-03-04T15:39:21.772 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: data_block_hash_table_util_ratio: 0.750000 2024-03-04T15:39:21.772 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: checksum: 4 2024-03-04T15:39:21.772 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: no_block_cache: 0 2024-03-04T15:39:21.772 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: block_cache: 0x559c262d3090 2024-03-04T15:39:21.772 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: block_cache_name: BinnedLRUCache 2024-03-04T15:39:21.772 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: block_cache_options: 2024-03-04T15:39:21.772 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: capacity : 536870912 2024-03-04T15:39:21.772 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: num_shard_bits : 4 2024-03-04T15:39:21.772 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: strict_capacity_limit : 0 2024-03-04T15:39:21.772 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: high_pri_pool_ratio: 0.000 2024-03-04T15:39:21.772 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: block_cache_compressed: (nil) 2024-03-04T15:39:21.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: persistent_cache: (nil) 2024-03-04T15:39:21.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: block_size: 4096 2024-03-04T15:39:21.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: block_size_deviation: 10 2024-03-04T15:39:21.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: block_restart_interval: 16 2024-03-04T15:39:21.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: index_block_restart_interval: 1 2024-03-04T15:39:21.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: metadata_block_size: 4096 2024-03-04T15:39:21.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: partition_filters: 0 2024-03-04T15:39:21.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: use_delta_encoding: 1 2024-03-04T15:39:21.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: filter_policy: bloomfilter 2024-03-04T15:39:21.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: whole_key_filtering: 1 2024-03-04T15:39:21.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: verify_compression: 0 2024-03-04T15:39:21.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: read_amp_bytes_per_bit: 0 2024-03-04T15:39:21.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: format_version: 5 2024-03-04T15:39:21.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: enable_index_compression: 1 2024-03-04T15:39:21.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: block_align: 0 2024-03-04T15:39:21.773 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: max_auto_readahead_size: 262144 2024-03-04T15:39:21.774 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: prepopulate_block_cache: 0 2024-03-04T15:39:21.774 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: initial_auto_readahead_size: 8192 2024-03-04T15:39:21.774 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: num_file_reads_for_auto_readahead: 2 2024-03-04T15:39:21.774 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.write_buffer_size: 33554432 2024-03-04T15:39:21.774 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_write_buffer_number: 2 2024-03-04T15:39:21.774 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compression: NoCompression 2024-03-04T15:39:21.774 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.bottommost_compression: Disabled 2024-03-04T15:39:21.774 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.prefix_extractor: nullptr 2024-03-04T15:39:21.774 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-03-04T15:39:21.774 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.num_levels: 7 2024-03-04T15:39:21.774 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-03-04T15:39:21.774 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-03-04T15:39:21.774 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-03-04T15:39:21.774 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-03-04T15:39:21.774 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.bottommost_compression_opts.level: 32767 2024-03-04T15:39:21.774 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-03-04T15:39:21.775 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-03-04T15:39:21.775 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-03-04T15:39:21.775 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-03-04T15:39:21.775 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.bottommost_compression_opts.enabled: false 2024-03-04T15:39:21.775 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-03-04T15:39:21.775 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-03-04T15:39:21.775 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compression_opts.window_bits: -14 2024-03-04T15:39:21.775 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compression_opts.level: 32767 2024-03-04T15:39:21.775 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compression_opts.strategy: 0 2024-03-04T15:39:21.775 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-03-04T15:39:21.775 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-03-04T15:39:21.775 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-03-04T15:39:21.775 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compression_opts.parallel_threads: 1 2024-03-04T15:39:21.775 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compression_opts.enabled: false 2024-03-04T15:39:21.775 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-03-04T15:39:21.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-03-04T15:39:21.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-03-04T15:39:21.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.level0_stop_writes_trigger: 36 2024-03-04T15:39:21.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.target_file_size_base: 67108864 2024-03-04T15:39:21.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.target_file_size_multiplier: 1 2024-03-04T15:39:21.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_bytes_for_level_base: 268435456 2024-03-04T15:39:21.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-03-04T15:39:21.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-03-04T15:39:21.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-03-04T15:39:21.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-03-04T15:39:21.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-03-04T15:39:21.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-03-04T15:39:21.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-03-04T15:39:21.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-03-04T15:39:21.776 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-03-04T15:39:21.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-03-04T15:39:21.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_compaction_bytes: 1677721600 2024-03-04T15:39:21.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-03-04T15:39:21.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.arena_block_size: 1048576 2024-03-04T15:39:21.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-03-04T15:39:21.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-03-04T15:39:21.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.disable_auto_compactions: 0 2024-03-04T15:39:21.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-03-04T15:39:21.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-03-04T15:39:21.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-03-04T15:39:21.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-03-04T15:39:21.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-03-04T15:39:21.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-03-04T15:39:21.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-03-04T15:39:21.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-03-04T15:39:21.777 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-03-04T15:39:21.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-03-04T15:39:21.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.table_properties_collectors: 2024-03-04T15:39:21.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.inplace_update_support: 0 2024-03-04T15:39:21.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.inplace_update_num_locks: 10000 2024-03-04T15:39:21.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-03-04T15:39:21.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.memtable_whole_key_filtering: 0 2024-03-04T15:39:21.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.memtable_huge_page_size: 0 2024-03-04T15:39:21.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.bloom_locality: 0 2024-03-04T15:39:21.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.max_successive_merges: 0 2024-03-04T15:39:21.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.optimize_filters_for_hits: 0 2024-03-04T15:39:21.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.paranoid_file_checks: 0 2024-03-04T15:39:21.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.force_consistency_checks: 1 2024-03-04T15:39:21.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.report_bg_io_stats: 0 2024-03-04T15:39:21.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.ttl: 2592000 2024-03-04T15:39:21.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.periodic_compaction_seconds: 0 2024-03-04T15:39:21.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.preclude_last_level_data_seconds: 0 2024-03-04T15:39:21.778 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.preserve_internal_time_seconds: 0 2024-03-04T15:39:21.779 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.enable_blob_files: false 2024-03-04T15:39:21.779 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.min_blob_size: 0 2024-03-04T15:39:21.779 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.blob_file_size: 268435456 2024-03-04T15:39:21.779 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.blob_compression_type: NoCompression 2024-03-04T15:39:21.779 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.enable_blob_garbage_collection: false 2024-03-04T15:39:21.779 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-03-04T15:39:21.779 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-03-04T15:39:21.779 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.blob_compaction_readahead_size: 0 2024-03-04T15:39:21.779 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.blob_file_starting_level: 0 2024-03-04T15:39:21.779 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.520+0000 7efe16117c80 4 rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-03-04T15:39:21.779 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.524+0000 7efe16117c80 4 rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-03-04T15:39:21.779 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.524+0000 7efe16117c80 4 rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-03-04T15:39:21.779 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.524+0000 7efe16117c80 4 rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 98fb126d-5162-4c02-a3ff-f3cb2bfa7942 2024-03-04T15:39:21.779 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.524+0000 7efe16117c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1709566761525855, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-03-04T15:39:21.779 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.524+0000 7efe16117c80 4 rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-03-04T15:39:21.780 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.524+0000 7efe16117c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1709566761527329, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 83257, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 262, "table_properties": {"data_size": 81429, "index_size": 238, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10592, "raw_average_key_size": 47, "raw_value_size": 75441, "raw_average_value_size": 341, "num_data_blocks": 10, "num_entries": 221, "num_filter_entries": 221, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1709566761, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "98fb126d-5162-4c02-a3ff-f3cb2bfa7942", "db_session_id": "2C3U6B506RFJ1JY3D6YZ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-03-04T15:39:21.780 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.524+0000 7efe16117c80 4 rocksdb: EVENT_LOG_v1 {"time_micros": 1709566761527451, "job": 1, "event": "recovery_finished"} 2024-03-04T15:39:21.780 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.524+0000 7efe16117c80 4 rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-03-04T15:39:21.780 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.528+0000 7efe16117c80 4 rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-03-04T15:39:21.780 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.528+0000 7efe16117c80 4 rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x559c263ca000 2024-03-04T15:39:21.780 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.528+0000 7efe16117c80 4 rocksdb: DB pointer 0x559c263b4000 2024-03-04T15:39:21.780 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.528+0000 7efe04880700 4 rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-03-04T15:39:21.780 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.528+0000 7efe04880700 4 rocksdb: [db/db_impl/db_impl.cc:1111] 2024-03-04T15:39:21.780 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: ** DB Stats ** 2024-03-04T15:39:21.780 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: Uptime(secs): 0.0 total, 0.0 interval 2024-03-04T15:39:21.780 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-03-04T15:39:21.780 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-04T15:39:21.780 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-04T15:39:21.780 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-03-04T15:39:21.780 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-03-04T15:39:21.781 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-03-04T15:39:21.781 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: ** Compaction Stats [default] ** 2024-03-04T15:39:21.781 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-04T15:39:21.781 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-03-04T15:39:21.781 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: L0 2/0 83.14 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 90.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-04T15:39:21.781 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: Sum 2/0 83.14 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 90.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-04T15:39:21.781 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 90.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-04T15:39:21.781 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: ** Compaction Stats [default] ** 2024-03-04T15:39:21.781 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-03-04T15:39:21.781 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-03-04T15:39:21.781 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 90.7 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-03-04T15:39:21.781 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-03-04T15:39:21.781 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: Uptime(secs): 0.0 total, 0.0 interval 2024-03-04T15:39:21.781 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: Flush(GB): cumulative 0.000, interval 0.000 2024-03-04T15:39:21.781 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: AddFile(GB): cumulative 0.000, interval 0.000 2024-03-04T15:39:21.781 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: AddFile(Total Files): cumulative 0, interval 0 2024-03-04T15:39:21.781 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: AddFile(L0 Files): cumulative 0, interval 0 2024-03-04T15:39:21.782 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: AddFile(Keys): cumulative 0, interval 0 2024-03-04T15:39:21.782 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: Cumulative compaction: 0.00 GB write, 15.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T15:39:21.782 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: Interval compaction: 0.00 GB write, 15.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-03-04T15:39:21.782 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-03-04T15:39:21.782 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: Block cache BinnedLRUCache@0x559c262d3090#7 capacity: 512.00 MB usage: 1.19 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.4e-05 secs_since: 0 2024-03-04T15:39:21.782 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: Block cache entry stats(count,size,portion): FilterBlock(2,0.77 KB,0.000146031%) IndexBlock(2,0.42 KB,8.04663e-05%) Misc(1,0.00 KB,0%) 2024-03-04T15:39:21.782 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: ** File Read Latency Histogram By Level [default] ** 2024-03-04T15:39:21.782 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.528+0000 7efe16117c80 0 starting mon.a rank 0 at public addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] at bind addrs [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T15:39:21.782 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.528+0000 7efe16117c80 1 mon.a@-1(???) e1 preinit fsid e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T15:39:21.782 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.528+0000 7efe16117c80 0 mon.a@-1(???).mds e1 new map 2024-03-04T15:39:21.782 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.528+0000 7efe16117c80 0 mon.a@-1(???).mds e1 print_map 2024-03-04T15:39:21.782 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: e1 2024-03-04T15:39:21.782 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: enable_multiple, ever_enabled_multiple: 1,1 2024-03-04T15:39:21.782 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments} 2024-03-04T15:39:21.783 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: legacy client fscid: -1 2024-03-04T15:39:21.783 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: 2024-03-04T15:39:21.783 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: No filesystems configured 2024-03-04T15:39:21.783 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.528+0000 7efe16117c80 0 mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-03-04T15:39:21.783 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.528+0000 7efe16117c80 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-04T15:39:21.783 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.528+0000 7efe16117c80 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-04T15:39:21.783 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.528+0000 7efe16117c80 0 mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-03-04T15:39:21.783 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: debug 2024-03-04T15:39:21.528+0000 7efe16117c80 1 mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-03-04T15:39:21.783 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: cluster 2024-03-04T15:39:21.538091+0000 mon.a (mon.0) 1 : cluster 1 mon.a is new leader, mons a in quorum (ranks 0) 2024-03-04T15:39:21.783 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: cluster 2024-03-04T15:39:21.538182+0000 mon.a (mon.0) 2 : cluster 0 monmap e1: 1 mons at {a=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T15:39:21.783 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: cluster 2024-03-04T15:39:21.557585+0000 mon.a (mon.0) 3 : cluster 0 fsmap 2024-03-04T15:39:21.783 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: cluster 2024-03-04T15:39:21.557620+0000 mon.a (mon.0) 4 : cluster 0 osdmap e1: 0 total, 0 up, 0 in 2024-03-04T15:39:21.783 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:21 smithi117 bash[21480]: cluster 2024-03-04T15:39:21.557997+0000 mon.a (mon.0) 5 : cluster 0 mgrmap e1: no daemons active 2024-03-04T15:39:22.466 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.604+0000 7f3daa5ab700 1 Processor -- start 2024-03-04T15:39:22.466 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.604+0000 7f3daa5ab700 1 -- start start 2024-03-04T15:39:22.466 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3daa5ab700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3da41044d0 0x7f3da41048f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:22.466 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3daa5ab700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da40796f0 con 0x7f3da41044d0 2024-03-04T15:39:22.466 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3da3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3da41044d0 0x7f3da41048f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:22.466 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3da3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3da41044d0 0x7f3da41048f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41364/0 (socket says 172.21.15.117:41364) 2024-03-04T15:39:22.466 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3da3fff700 1 -- 172.21.15.117:0/892754400 learned_addr learned my addr 172.21.15.117:0/892754400 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:22.466 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3da3fff700 1 -- 172.21.15.117:0/892754400 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da4104e30 con 0x7f3da41044d0 2024-03-04T15:39:22.467 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3da3fff700 1 --2- 172.21.15.117:0/892754400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3da41044d0 0x7f3da41048f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009e80 tx=0x7f3d9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f8107562229f9d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:22.467 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3da2ffd700 1 -- 172.21.15.117:0/892754400 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d94017070 con 0x7f3da41044d0 2024-03-04T15:39:22.467 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3da2ffd700 1 -- 172.21.15.117:0/892754400 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f3d9400b8c0 con 0x7f3da41044d0 2024-03-04T15:39:22.467 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3da2ffd700 1 -- 172.21.15.117:0/892754400 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d94004370 con 0x7f3da41044d0 2024-03-04T15:39:22.467 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3daa5ab700 1 -- 172.21.15.117:0/892754400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3da41044d0 msgr2=0x7f3da41048f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:22.467 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3daa5ab700 1 --2- 172.21.15.117:0/892754400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3da41044d0 0x7f3da41048f0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f3d94009e80 tx=0x7f3d9400b330 comp rx=0 tx=0).stop 2024-03-04T15:39:22.467 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3daa5ab700 1 -- 172.21.15.117:0/892754400 shutdown_connections 2024-03-04T15:39:22.470 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3daa5ab700 1 --2- 172.21.15.117:0/892754400 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3da41044d0 0x7f3da41048f0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:22.470 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3daa5ab700 1 -- 172.21.15.117:0/892754400 >> 172.21.15.117:0/892754400 conn(0x7f3da41000b0 msgr2=0x7f3da4102510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:22.470 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3daa5ab700 1 -- 172.21.15.117:0/892754400 shutdown_connections 2024-03-04T15:39:22.470 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3daa5ab700 1 -- 172.21.15.117:0/892754400 wait complete. 2024-03-04T15:39:22.470 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3daa5ab700 1 Processor -- start 2024-03-04T15:39:22.470 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3daa5ab700 1 -- start start 2024-03-04T15:39:22.470 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3daa5ab700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3da41044d0 0x7f3da419bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:22.470 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3daa5ab700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3da4105cc0 con 0x7f3da41044d0 2024-03-04T15:39:22.470 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3da3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3da41044d0 0x7f3da419bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:22.471 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3da3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3da41044d0 0x7f3da419bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41378/0 (socket says 172.21.15.117:41378) 2024-03-04T15:39:22.471 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3da3fff700 1 -- 172.21.15.117:0/2689907254 learned_addr learned my addr 172.21.15.117:0/2689907254 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:22.471 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3da3fff700 1 -- 172.21.15.117:0/2689907254 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3da419c0b0 con 0x7f3da41044d0 2024-03-04T15:39:22.471 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.608+0000 7f3da3fff700 1 --2- 172.21.15.117:0/2689907254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3da41044d0 0x7f3da419bb70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3d94004000 tx=0x7f3d94003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:22.471 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.612+0000 7f3da17fa700 1 -- 172.21.15.117:0/2689907254 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d94021020 con 0x7f3da41044d0 2024-03-04T15:39:22.471 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.612+0000 7f3da17fa700 1 -- 172.21.15.117:0/2689907254 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f3d9400bde0 con 0x7f3da41044d0 2024-03-04T15:39:22.471 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.612+0000 7f3da17fa700 1 -- 172.21.15.117:0/2689907254 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3d94017040 con 0x7f3da41044d0 2024-03-04T15:39:22.471 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.612+0000 7f3daa5ab700 1 -- 172.21.15.117:0/2689907254 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3da419c340 con 0x7f3da41044d0 2024-03-04T15:39:22.471 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.612+0000 7f3daa5ab700 1 -- 172.21.15.117:0/2689907254 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3da419f030 con 0x7f3da41044d0 2024-03-04T15:39:22.471 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.612+0000 7f3da17fa700 1 -- 172.21.15.117:0/2689907254 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f3d94024020 con 0x7f3da41044d0 2024-03-04T15:39:22.471 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.612+0000 7f3da17fa700 1 -- 172.21.15.117:0/2689907254 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f3d9400f050 con 0x7f3da41044d0 2024-03-04T15:39:22.471 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.616+0000 7f3daa5ab700 1 -- 172.21.15.117:0/2689907254 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3da4063410 con 0x7f3da41044d0 2024-03-04T15:39:22.471 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.616+0000 7f3da17fa700 1 -- 172.21.15.117:0/2689907254 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f3d9402b020 con 0x7f3da41044d0 2024-03-04T15:39:22.471 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.740+0000 7f3daa5ab700 1 -- 172.21.15.117:0/2689907254 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f3da402d8d0 con 0x7f3da41044d0 2024-03-04T15:39:22.471 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.748+0000 7f3da17fa700 1 -- 172.21.15.117:0/2689907254 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3d9401dad0 con 0x7f3da41044d0 2024-03-04T15:39:22.471 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.748+0000 7f3daa5ab700 1 -- 172.21.15.117:0/2689907254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3da41044d0 msgr2=0x7f3da419bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:22.472 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.748+0000 7f3daa5ab700 1 --2- 172.21.15.117:0/2689907254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3da41044d0 0x7f3da419bb70 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f3d94004000 tx=0x7f3d94003c50 comp rx=0 tx=0).stop 2024-03-04T15:39:22.472 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.748+0000 7f3daa5ab700 1 -- 172.21.15.117:0/2689907254 shutdown_connections 2024-03-04T15:39:22.472 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.748+0000 7f3daa5ab700 1 --2- 172.21.15.117:0/2689907254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3da41044d0 0x7f3da419bb70 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:22.472 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.748+0000 7f3daa5ab700 1 -- 172.21.15.117:0/2689907254 >> 172.21.15.117:0/2689907254 conn(0x7f3da41000b0 msgr2=0x7f3da418ff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:22.472 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.748+0000 7f3daa5ab700 1 -- 172.21.15.117:0/2689907254 shutdown_connections 2024-03-04T15:39:22.472 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:21.748+0000 7f3daa5ab700 1 -- 172.21.15.117:0/2689907254 wait complete. 2024-03-04T15:39:22.472 INFO:teuthology.orchestra.run.smithi117.stdout:Wrote config to /etc/ceph/ceph.conf 2024-03-04T15:39:22.498 INFO:teuthology.orchestra.run.smithi117.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-03-04T15:39:22.498 INFO:teuthology.orchestra.run.smithi117.stdout:Creating mgr... 2024-03-04T15:39:22.499 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying port 0.0.0.0:9283 ... 2024-03-04T15:39:22.499 INFO:teuthology.orchestra.run.smithi117.stdout:Verifying port 0.0.0.0:8765 ... 2024-03-04T15:39:22.818 INFO:teuthology.orchestra.run.smithi117.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mgr.y 2024-03-04T15:39:22.819 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Failed to reset failed state of unit ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mgr.y.service: Unit ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mgr.y.service not loaded. 2024-03-04T15:39:22.942 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:22 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:39:22.942 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:22 smithi117 bash[21480]: audit 2024-03-04T15:39:21.749884+0000 mon.a (mon.0) 6 : audit 1 from='client.? 172.21.15.117:0/2689907254' entity='client.admin' 2024-03-04T15:39:22.942 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:22 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:39:23.029 INFO:teuthology.orchestra.run.smithi117.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715.target.wants/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mgr.y.service -> /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service. 2024-03-04T15:39:23.038 INFO:teuthology.orchestra.run.smithi117.stdout:firewalld does not appear to be present 2024-03-04T15:39:23.038 INFO:teuthology.orchestra.run.smithi117.stdout:Not possible to enable service . firewalld.service is not available 2024-03-04T15:39:23.038 INFO:teuthology.orchestra.run.smithi117.stdout:firewalld does not appear to be present 2024-03-04T15:39:23.038 INFO:teuthology.orchestra.run.smithi117.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-03-04T15:39:23.038 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mgr to start... 2024-03-04T15:39:23.038 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mgr... 2024-03-04T15:39:24.608 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:24 smithi117 bash[21480]: audit 2024-03-04T15:39:24.381775+0000 mon.a (mon.0) 7 : audit 0 from='client.? 172.21.15.117:0/718659848' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-04T15:39:24.608 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:24 smithi117 bash[21734]: debug 2024-03-04T15:39:24.328+0000 7f1127db0200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T15:39:25.040 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-03-04T15:39:25.041 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-03-04T15:39:25.041 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsid": "e40b5504-da3c-11ee-95c6-87774f69a715", 2024-03-04T15:39:25.041 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "health": { 2024-03-04T15:39:25.041 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-04T15:39:25.041 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-04T15:39:25.041 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-04T15:39:25.041 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:25.041 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-04T15:39:25.041 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-04T15:39:25.041 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 0 2024-03-04T15:39:25.041 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-03-04T15:39:25.041 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-04T15:39:25.041 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "a" 2024-03-04T15:39:25.041 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-03-04T15:39:25.042 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-03-04T15:39:25.042 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-04T15:39:25.042 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T15:39:25.042 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-04T15:39:25.042 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-04T15:39:25.042 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:25.042 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-04T15:39:25.042 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T15:39:25.042 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-04T15:39:25.042 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-04T15:39:25.042 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-04T15:39:25.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-04T15:39:25.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-04T15:39:25.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-04T15:39:25.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:25.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-04T15:39:25.050 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-04T15:39:25.051 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-04T15:39:25.051 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-04T15:39:25.051 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-04T15:39:25.051 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-04T15:39:25.051 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-04T15:39:25.051 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-04T15:39:25.051 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-04T15:39:25.051 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:25.051 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-04T15:39:25.051 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T15:39:25.051 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-04T15:39:25.051 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-04T15:39:25.051 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:25.051 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-04T15:39:25.051 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-04T15:39:25.051 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-04T15:39:25.052 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-04T15:39:25.052 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-04T15:39:25.052 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-04T15:39:25.052 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "restful" 2024-03-04T15:39:25.052 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-03-04T15:39:25.052 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T15:39:25.052 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:25.052 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-04T15:39:25.052 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T15:39:25.052 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modified": "2024-03-04T15:39:15.692156+0000", 2024-03-04T15:39:25.052 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T15:39:25.052 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:25.052 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-04T15:39:25.052 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-03-04T15:39:25.052 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7aa28a7700 1 Processor -- start 2024-03-04T15:39:25.053 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7aa28a7700 1 -- start start 2024-03-04T15:39:25.053 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7aa28a7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a9c07b760 0x7f7a9c079bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:25.053 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7aa28a7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c07a190 con 0x7f7a9c07b760 2024-03-04T15:39:25.053 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7a9bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a9c07b760 0x7f7a9c079bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:25.053 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7a9bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a9c07b760 0x7f7a9c079bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41400/0 (socket says 172.21.15.117:41400) 2024-03-04T15:39:25.053 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7a9bfff700 1 -- 172.21.15.117:0/3543199559 learned_addr learned my addr 172.21.15.117:0/3543199559 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:25.053 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7a9bfff700 1 -- 172.21.15.117:0/3543199559 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c07a310 con 0x7f7a9c07b760 2024-03-04T15:39:25.053 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7a9bfff700 1 --2- 172.21.15.117:0/3543199559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a9c07b760 0x7f7a9c079bc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009e80 tx=0x7f7a9400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861a2d20a8ab01fe server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:25.053 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7a9affd700 1 -- 172.21.15.117:0/3543199559 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a94017070 con 0x7f7a9c07b760 2024-03-04T15:39:25.053 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7a9affd700 1 -- 172.21.15.117:0/3543199559 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f7a9400b8c0 con 0x7f7a9c07b760 2024-03-04T15:39:25.053 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7a9affd700 1 -- 172.21.15.117:0/3543199559 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a94004370 con 0x7f7a9c07b760 2024-03-04T15:39:25.053 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7aa28a7700 1 -- 172.21.15.117:0/3543199559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a9c07b760 msgr2=0x7f7a9c079bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:25.053 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7aa28a7700 1 --2- 172.21.15.117:0/3543199559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a9c07b760 0x7f7a9c079bc0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f7a94009e80 tx=0x7f7a9400b330 comp rx=0 tx=0).stop 2024-03-04T15:39:25.054 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7aa28a7700 1 -- 172.21.15.117:0/3543199559 shutdown_connections 2024-03-04T15:39:25.054 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7aa28a7700 1 --2- 172.21.15.117:0/3543199559 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a9c07b760 0x7f7a9c079bc0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:25.054 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7aa28a7700 1 -- 172.21.15.117:0/3543199559 >> 172.21.15.117:0/3543199559 conn(0x7f7a9c1011c0 msgr2=0x7f7a9c103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:25.054 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7aa28a7700 1 -- 172.21.15.117:0/3543199559 shutdown_connections 2024-03-04T15:39:25.054 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7aa28a7700 1 -- 172.21.15.117:0/3543199559 wait complete. 2024-03-04T15:39:25.054 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7aa28a7700 1 Processor -- start 2024-03-04T15:39:25.054 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7aa28a7700 1 -- start start 2024-03-04T15:39:25.054 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7aa28a7700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a9c07b760 0x7f7a9c19bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:25.054 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.244+0000 7f7aa28a7700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c10c090 con 0x7f7a9c07b760 2024-03-04T15:39:25.054 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.248+0000 7f7a9bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a9c07b760 0x7f7a9c19bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:25.054 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.248+0000 7f7a9bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a9c07b760 0x7f7a9c19bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41412/0 (socket says 172.21.15.117:41412) 2024-03-04T15:39:25.054 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.248+0000 7f7a9bfff700 1 -- 172.21.15.117:0/718659848 learned_addr learned my addr 172.21.15.117:0/718659848 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:25.054 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.248+0000 7f7a9bfff700 1 -- 172.21.15.117:0/718659848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c19c140 con 0x7f7a9c07b760 2024-03-04T15:39:25.054 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.248+0000 7f7a9bfff700 1 --2- 172.21.15.117:0/718659848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a9c07b760 0x7f7a9c19bc00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7a94004000 tx=0x7f7a94003c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:25.054 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.248+0000 7f7a997fa700 1 -- 172.21.15.117:0/718659848 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a94020020 con 0x7f7a9c07b760 2024-03-04T15:39:25.055 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.248+0000 7f7a997fa700 1 -- 172.21.15.117:0/718659848 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f7a94012890 con 0x7f7a9c07b760 2024-03-04T15:39:25.055 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.248+0000 7f7a997fa700 1 -- 172.21.15.117:0/718659848 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a94017040 con 0x7f7a9c07b760 2024-03-04T15:39:25.055 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.248+0000 7f7aa28a7700 1 -- 172.21.15.117:0/718659848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a9c19c3d0 con 0x7f7a9c07b760 2024-03-04T15:39:25.055 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.248+0000 7f7aa28a7700 1 -- 172.21.15.117:0/718659848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a9c19f0c0 con 0x7f7a9c07b760 2024-03-04T15:39:25.055 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.248+0000 7f7a997fa700 1 -- 172.21.15.117:0/718659848 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7f7a94023020 con 0x7f7a9c07b760 2024-03-04T15:39:25.055 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.248+0000 7f7a997fa700 1 -- 172.21.15.117:0/718659848 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f7a9400f050 con 0x7f7a9c07b760 2024-03-04T15:39:25.055 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.248+0000 7f7aa28a7700 1 -- 172.21.15.117:0/718659848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a9c063410 con 0x7f7a9c07b760 2024-03-04T15:39:25.055 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.252+0000 7f7a997fa700 1 -- 172.21.15.117:0/718659848 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7f7a9402a020 con 0x7f7a9c07b760 2024-03-04T15:39:25.055 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.380+0000 7f7aa28a7700 1 -- 172.21.15.117:0/718659848 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f7a9c19f4a0 con 0x7f7a9c07b760 2024-03-04T15:39:25.055 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.380+0000 7f7a997fa700 1 -- 172.21.15.117:0/718659848 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7f7a9400b860 con 0x7f7a9c07b760 2024-03-04T15:39:25.055 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.380+0000 7f7aa28a7700 1 -- 172.21.15.117:0/718659848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a9c07b760 msgr2=0x7f7a9c19bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:25.055 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.380+0000 7f7aa28a7700 1 --2- 172.21.15.117:0/718659848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a9c07b760 0x7f7a9c19bc00 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f7a94004000 tx=0x7f7a94003c50 comp rx=0 tx=0).stop 2024-03-04T15:39:25.055 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.380+0000 7f7aa28a7700 1 -- 172.21.15.117:0/718659848 shutdown_connections 2024-03-04T15:39:25.055 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.380+0000 7f7aa28a7700 1 --2- 172.21.15.117:0/718659848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a9c07b760 0x7f7a9c19bc00 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:25.056 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.380+0000 7f7aa28a7700 1 -- 172.21.15.117:0/718659848 >> 172.21.15.117:0/718659848 conn(0x7f7a9c1011c0 msgr2=0x7f7a9c101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:25.056 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.380+0000 7f7aa28a7700 1 -- 172.21.15.117:0/718659848 shutdown_connections 2024-03-04T15:39:25.056 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:24.380+0000 7f7aa28a7700 1 -- 172.21.15.117:0/718659848 wait complete. 2024-03-04T15:39:25.056 INFO:teuthology.orchestra.run.smithi117.stdout:mgr not available, waiting (1/15)... 2024-03-04T15:39:26.234 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:25 smithi117 bash[21734]: debug 2024-03-04T15:39:25.892+0000 7f1127db0200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T15:39:26.234 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:25 smithi117 bash[21734]: debug 2024-03-04T15:39:25.988+0000 7f1127db0200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T15:39:26.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:26 smithi117 bash[21734]: debug 2024-03-04T15:39:26.232+0000 7f1127db0200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T15:39:27.320 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:27 smithi117 bash[21734]: debug 2024-03-04T15:39:27.012+0000 7f1127db0200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T15:39:27.320 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:27 smithi117 bash[21734]: debug 2024-03-04T15:39:27.128+0000 7f1127db0200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T15:39:27.320 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:27 smithi117 bash[21734]: debug 2024-03-04T15:39:27.228+0000 7f1127db0200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T15:39:27.827 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:27 smithi117 bash[21734]: debug 2024-03-04T15:39:27.576+0000 7f1127db0200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T15:39:27.827 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:27 smithi117 bash[21734]: debug 2024-03-04T15:39:27.688+0000 7f1127db0200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T15:39:27.827 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:27 smithi117 bash[21734]: debug 2024-03-04T15:39:27.792+0000 7f1127db0200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T15:39:28.388 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:28 smithi117 bash[21734]: debug 2024-03-04T15:39:28.144+0000 7f1127db0200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T15:39:28.388 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:28 smithi117 bash[21480]: audit 2024-03-04T15:39:28.114382+0000 mon.a (mon.0) 8 : audit 0 from='client.? 172.21.15.117:0/2998627051' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-04T15:39:28.645 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:28 smithi117 bash[21734]: debug 2024-03-04T15:39:28.388+0000 7f1127db0200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T15:39:28.687 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-03-04T15:39:28.687 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-03-04T15:39:28.687 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsid": "e40b5504-da3c-11ee-95c6-87774f69a715", 2024-03-04T15:39:28.687 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "health": { 2024-03-04T15:39:28.687 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-04T15:39:28.687 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-04T15:39:28.687 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-04T15:39:28.687 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:28.687 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-04T15:39:28.687 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-04T15:39:28.687 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 0 2024-03-04T15:39:28.687 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-03-04T15:39:28.687 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-04T15:39:28.687 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "a" 2024-03-04T15:39:28.688 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-03-04T15:39:28.688 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-03-04T15:39:28.688 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-04T15:39:28.688 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T15:39:28.688 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-04T15:39:28.688 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-04T15:39:28.688 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:28.688 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-04T15:39:28.688 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T15:39:28.688 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-04T15:39:28.688 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-04T15:39:28.688 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-04T15:39:28.695 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-04T15:39:28.695 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-04T15:39:28.695 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-04T15:39:28.696 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:28.696 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-04T15:39:28.696 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-04T15:39:28.696 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-04T15:39:28.696 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-04T15:39:28.696 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-04T15:39:28.696 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-04T15:39:28.696 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-04T15:39:28.696 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-04T15:39:28.696 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-04T15:39:28.696 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:28.696 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-04T15:39:28.696 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T15:39:28.696 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-04T15:39:28.696 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-04T15:39:28.697 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:28.697 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-04T15:39:28.697 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-04T15:39:28.697 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-04T15:39:28.697 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-04T15:39:28.697 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-04T15:39:28.697 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-04T15:39:28.697 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "restful" 2024-03-04T15:39:28.697 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-03-04T15:39:28.697 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T15:39:28.697 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:28.697 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-04T15:39:28.697 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T15:39:28.697 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modified": "2024-03-04T15:39:15.692156+0000", 2024-03-04T15:39:28.697 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T15:39:28.697 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:28.698 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-04T15:39:28.698 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-03-04T15:39:28.698 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.968+0000 7ff95ed53700 1 Processor -- start 2024-03-04T15:39:28.698 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.968+0000 7ff95ed53700 1 -- start start 2024-03-04T15:39:28.698 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.968+0000 7ff95ed53700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff958105c60 0x7ff958106080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:28.698 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.968+0000 7ff95ed53700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff958106650 con 0x7ff958105c60 2024-03-04T15:39:28.698 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.968+0000 7ff95caef700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff958105c60 0x7ff958106080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:28.698 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.968+0000 7ff95caef700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff958105c60 0x7ff958106080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41272/0 (socket says 172.21.15.117:41272) 2024-03-04T15:39:28.698 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.968+0000 7ff95caef700 1 -- 172.21.15.117:0/1219239277 learned_addr learned my addr 172.21.15.117:0/1219239277 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:28.698 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.968+0000 7ff95caef700 1 -- 172.21.15.117:0/1219239277 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff958106e60 con 0x7ff958105c60 2024-03-04T15:39:28.698 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.968+0000 7ff95caef700 1 --2- 172.21.15.117:0/1219239277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff958105c60 0x7ff958106080 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff9500098a0 tx=0x7ff950009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=369bf3a716c39e96 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:28.698 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.968+0000 7ff9577fe700 1 -- 172.21.15.117:0/1219239277 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff950004160 con 0x7ff958105c60 2024-03-04T15:39:28.698 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.968+0000 7ff9577fe700 1 -- 172.21.15.117:0/1219239277 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7ff950004300 con 0x7ff958105c60 2024-03-04T15:39:28.698 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.968+0000 7ff9577fe700 1 -- 172.21.15.117:0/1219239277 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff950011940 con 0x7ff958105c60 2024-03-04T15:39:28.698 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff95ed53700 1 -- 172.21.15.117:0/1219239277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff958105c60 msgr2=0x7ff958106080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:28.699 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff95ed53700 1 --2- 172.21.15.117:0/1219239277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff958105c60 0x7ff958106080 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ff9500098a0 tx=0x7ff950009bb0 comp rx=0 tx=0).stop 2024-03-04T15:39:28.699 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff95ed53700 1 -- 172.21.15.117:0/1219239277 shutdown_connections 2024-03-04T15:39:28.699 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff95ed53700 1 --2- 172.21.15.117:0/1219239277 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff958105c60 0x7ff958106080 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:28.699 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff95ed53700 1 -- 172.21.15.117:0/1219239277 >> 172.21.15.117:0/1219239277 conn(0x7ff9581011c0 msgr2=0x7ff958103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:28.699 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff95ed53700 1 -- 172.21.15.117:0/1219239277 shutdown_connections 2024-03-04T15:39:28.699 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff95ed53700 1 -- 172.21.15.117:0/1219239277 wait complete. 2024-03-04T15:39:28.699 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff95ed53700 1 Processor -- start 2024-03-04T15:39:28.699 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff95ed53700 1 -- start start 2024-03-04T15:39:28.699 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff95ed53700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff958105c60 0x7ff95807b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:28.699 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff95ed53700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff958107cf0 con 0x7ff958105c60 2024-03-04T15:39:28.699 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff95caef700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff958105c60 0x7ff95807b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:28.699 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff95caef700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff958105c60 0x7ff95807b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41280/0 (socket says 172.21.15.117:41280) 2024-03-04T15:39:28.699 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff95caef700 1 -- 172.21.15.117:0/2998627051 learned_addr learned my addr 172.21.15.117:0/2998627051 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:28.699 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff95caef700 1 -- 172.21.15.117:0/2998627051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff95807b930 con 0x7ff958105c60 2024-03-04T15:39:28.699 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff95caef700 1 --2- 172.21.15.117:0/2998627051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff958105c60 0x7ff95807b3f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff950009fb0 tx=0x7ff9500119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:28.699 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff955ffb700 1 -- 172.21.15.117:0/2998627051 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff95001a430 con 0x7ff958105c60 2024-03-04T15:39:28.699 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff955ffb700 1 -- 172.21.15.117:0/2998627051 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7ff95001aa30 con 0x7ff958105c60 2024-03-04T15:39:28.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff95ed53700 1 -- 172.21.15.117:0/2998627051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff958079b20 con 0x7ff958105c60 2024-03-04T15:39:28.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff955ffb700 1 -- 172.21.15.117:0/2998627051 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff950024860 con 0x7ff958105c60 2024-03-04T15:39:28.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff95ed53700 1 -- 172.21.15.117:0/2998627051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff95807a060 con 0x7ff958105c60 2024-03-04T15:39:28.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff955ffb700 1 -- 172.21.15.117:0/2998627051 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7ff950022020 con 0x7ff958105c60 2024-03-04T15:39:28.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff955ffb700 1 -- 172.21.15.117:0/2998627051 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff950019620 con 0x7ff958105c60 2024-03-04T15:39:28.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.972+0000 7ff95ed53700 1 -- 172.21.15.117:0/2998627051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff958063410 con 0x7ff958105c60 2024-03-04T15:39:28.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:27.976+0000 7ff955ffb700 1 -- 172.21.15.117:0/2998627051 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7ff95001abe0 con 0x7ff958105c60 2024-03-04T15:39:28.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:28.112+0000 7ff95ed53700 1 -- 172.21.15.117:0/2998627051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7ff95807a5f0 con 0x7ff958105c60 2024-03-04T15:39:28.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:28.112+0000 7ff955ffb700 1 -- 172.21.15.117:0/2998627051 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7ff950032030 con 0x7ff958105c60 2024-03-04T15:39:28.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:28.112+0000 7ff95ed53700 1 -- 172.21.15.117:0/2998627051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff958105c60 msgr2=0x7ff95807b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:28.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:28.112+0000 7ff95ed53700 1 --2- 172.21.15.117:0/2998627051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff958105c60 0x7ff95807b3f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff950009fb0 tx=0x7ff9500119c0 comp rx=0 tx=0).stop 2024-03-04T15:39:28.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:28.112+0000 7ff95ed53700 1 -- 172.21.15.117:0/2998627051 shutdown_connections 2024-03-04T15:39:28.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:28.112+0000 7ff95ed53700 1 --2- 172.21.15.117:0/2998627051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff958105c60 0x7ff95807b3f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:28.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:28.112+0000 7ff95ed53700 1 -- 172.21.15.117:0/2998627051 >> 172.21.15.117:0/2998627051 conn(0x7ff9581011c0 msgr2=0x7ff958101d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:28.700 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:28.112+0000 7ff95ed53700 1 -- 172.21.15.117:0/2998627051 shutdown_connections 2024-03-04T15:39:28.701 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:28.112+0000 7ff95ed53700 1 -- 172.21.15.117:0/2998627051 wait complete. 2024-03-04T15:39:28.701 INFO:teuthology.orchestra.run.smithi117.stdout:mgr not available, waiting (2/15)... 2024-03-04T15:39:29.024 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:28 smithi117 bash[21734]: debug 2024-03-04T15:39:28.644+0000 7f1127db0200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T15:39:29.024 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:28 smithi117 bash[21734]: debug 2024-03-04T15:39:28.756+0000 7f1127db0200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T15:39:29.320 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:29 smithi117 bash[21734]: debug 2024-03-04T15:39:29.024+0000 7f1127db0200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T15:39:29.320 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:29 smithi117 bash[21734]: debug 2024-03-04T15:39:29.260+0000 7f1127db0200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T15:39:30.020 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:29 smithi117 bash[21734]: debug 2024-03-04T15:39:29.660+0000 7f1127db0200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T15:39:30.021 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:29 smithi117 bash[21734]: debug 2024-03-04T15:39:29.776+0000 7f1127db0200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T15:39:30.021 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:30 smithi117 bash[21734]: debug 2024-03-04T15:39:30.020+0000 7f1127db0200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T15:39:31.319 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:31 smithi117 bash[21734]: debug 2024-03-04T15:39:31.012+0000 7f1127db0200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T15:39:31.320 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:31 smithi117 bash[21734]: debug 2024-03-04T15:39:31.128+0000 7f1127db0200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T15:39:31.793 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:31 smithi117 bash[21734]: debug 2024-03-04T15:39:31.416+0000 7f1127db0200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T15:39:31.793 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:31 smithi117 bash[21734]: debug 2024-03-04T15:39:31.532+0000 7f1127db0200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T15:39:32.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:31 smithi117 bash[21480]: audit 2024-03-04T15:39:31.781387+0000 mon.a (mon.0) 9 : audit 0 from='client.? 172.21.15.117:0/1438867730' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-04T15:39:32.070 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:31 smithi117 bash[21734]: debug 2024-03-04T15:39:31.984+0000 7f1127db0200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T15:39:32.410 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-03-04T15:39:32.410 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-03-04T15:39:32.410 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsid": "e40b5504-da3c-11ee-95c6-87774f69a715", 2024-03-04T15:39:32.410 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "health": { 2024-03-04T15:39:32.410 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-04T15:39:32.410 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-04T15:39:32.410 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-04T15:39:32.410 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:32.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-04T15:39:32.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-04T15:39:32.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 0 2024-03-04T15:39:32.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-03-04T15:39:32.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-04T15:39:32.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "a" 2024-03-04T15:39:32.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-03-04T15:39:32.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_age": 10, 2024-03-04T15:39:32.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-04T15:39:32.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T15:39:32.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-04T15:39:32.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-04T15:39:32.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:32.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-04T15:39:32.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T15:39:32.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-04T15:39:32.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-04T15:39:32.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-04T15:39:32.419 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-04T15:39:32.419 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-04T15:39:32.419 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-04T15:39:32.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:32.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-04T15:39:32.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-04T15:39:32.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-04T15:39:32.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-04T15:39:32.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-04T15:39:32.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-04T15:39:32.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-04T15:39:32.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-04T15:39:32.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-04T15:39:32.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:32.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-04T15:39:32.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T15:39:32.420 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-04T15:39:32.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-04T15:39:32.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:32.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-04T15:39:32.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": false, 2024-03-04T15:39:32.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-04T15:39:32.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-04T15:39:32.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-04T15:39:32.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-04T15:39:32.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "restful" 2024-03-04T15:39:32.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-03-04T15:39:32.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T15:39:32.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:32.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-04T15:39:32.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T15:39:32.421 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modified": "2024-03-04T15:39:15.692156+0000", 2024-03-04T15:39:32.422 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T15:39:32.422 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:32.422 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-04T15:39:32.422 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-03-04T15:39:32.422 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.644+0000 7efe2ff91700 1 Processor -- start 2024-03-04T15:39:32.422 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.644+0000 7efe2ff91700 1 -- start start 2024-03-04T15:39:32.422 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.644+0000 7efe2ff91700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe281066a0 0x7efe28108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:32.422 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.644+0000 7efe2ff91700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe28078ea0 con 0x7efe281066a0 2024-03-04T15:39:32.422 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.644+0000 7efe2dd2d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe281066a0 0x7efe28108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:32.422 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.644+0000 7efe2dd2d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe281066a0 0x7efe28108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41286/0 (socket says 172.21.15.117:41286) 2024-03-04T15:39:32.422 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.644+0000 7efe2dd2d700 1 -- 172.21.15.117:0/3003920891 learned_addr learned my addr 172.21.15.117:0/3003920891 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:32.422 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.644+0000 7efe2dd2d700 1 -- 172.21.15.117:0/3003920891 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe28108fd0 con 0x7efe281066a0 2024-03-04T15:39:32.422 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.644+0000 7efe2dd2d700 1 --2- 172.21.15.117:0/3003920891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe281066a0 0x7efe28108a90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efe240098a0 tx=0x7efe24009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1579b320fbd80238 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:32.422 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.644+0000 7efe2cd2b700 1 -- 172.21.15.117:0/3003920891 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe24004160 con 0x7efe281066a0 2024-03-04T15:39:32.422 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.644+0000 7efe2cd2b700 1 -- 172.21.15.117:0/3003920891 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7efe24004300 con 0x7efe281066a0 2024-03-04T15:39:32.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.644+0000 7efe2cd2b700 1 -- 172.21.15.117:0/3003920891 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe24011940 con 0x7efe281066a0 2024-03-04T15:39:32.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe2ff91700 1 -- 172.21.15.117:0/3003920891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe281066a0 msgr2=0x7efe28108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:32.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe2ff91700 1 --2- 172.21.15.117:0/3003920891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe281066a0 0x7efe28108a90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efe240098a0 tx=0x7efe24009bb0 comp rx=0 tx=0).stop 2024-03-04T15:39:32.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe2ff91700 1 -- 172.21.15.117:0/3003920891 shutdown_connections 2024-03-04T15:39:32.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe2ff91700 1 --2- 172.21.15.117:0/3003920891 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe281066a0 0x7efe28108a90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:32.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe2ff91700 1 -- 172.21.15.117:0/3003920891 >> 172.21.15.117:0/3003920891 conn(0x7efe281000d0 msgr2=0x7efe28102510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:32.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe2ff91700 1 -- 172.21.15.117:0/3003920891 shutdown_connections 2024-03-04T15:39:32.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe2ff91700 1 -- 172.21.15.117:0/3003920891 wait complete. 2024-03-04T15:39:32.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe2ff91700 1 Processor -- start 2024-03-04T15:39:32.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe2ff91700 1 -- start start 2024-03-04T15:39:32.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe2ff91700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe281066a0 0x7efe2819bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:32.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe2ff91700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe28109e00 con 0x7efe281066a0 2024-03-04T15:39:32.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe2dd2d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe281066a0 0x7efe2819bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:32.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe2dd2d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe281066a0 0x7efe2819bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41294/0 (socket says 172.21.15.117:41294) 2024-03-04T15:39:32.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe2dd2d700 1 -- 172.21.15.117:0/1438867730 learned_addr learned my addr 172.21.15.117:0/1438867730 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:32.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe2dd2d700 1 -- 172.21.15.117:0/1438867730 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe2819c1a0 con 0x7efe281066a0 2024-03-04T15:39:32.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe2dd2d700 1 --2- 172.21.15.117:0/1438867730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe281066a0 0x7efe2819bc60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efe24009290 tx=0x7efe240119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:32.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe1effd700 1 -- 172.21.15.117:0/1438867730 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe24003930 con 0x7efe281066a0 2024-03-04T15:39:32.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe1effd700 1 -- 172.21.15.117:0/1438867730 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7efe24003ad0 con 0x7efe281066a0 2024-03-04T15:39:32.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe1effd700 1 -- 172.21.15.117:0/1438867730 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efe24019720 con 0x7efe281066a0 2024-03-04T15:39:32.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe2ff91700 1 -- 172.21.15.117:0/1438867730 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe2819c430 con 0x7efe281066a0 2024-03-04T15:39:32.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe2ff91700 1 -- 172.21.15.117:0/1438867730 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe2819f120 con 0x7efe281066a0 2024-03-04T15:39:32.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe1effd700 1 -- 172.21.15.117:0/1438867730 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 1) v1 ==== 807+0+0 (secure 0 0 0) 0x7efe2402d440 con 0x7efe281066a0 2024-03-04T15:39:32.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe1effd700 1 -- 172.21.15.117:0/1438867730 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7efe24024810 con 0x7efe281066a0 2024-03-04T15:39:32.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.648+0000 7efe2ff91700 1 -- 172.21.15.117:0/1438867730 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe28063410 con 0x7efe281066a0 2024-03-04T15:39:32.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.652+0000 7efe1effd700 1 -- 172.21.15.117:0/1438867730 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+73141 (secure 0 0 0) 0x7efe24043dd0 con 0x7efe281066a0 2024-03-04T15:39:32.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.776+0000 7efe2ff91700 1 -- 172.21.15.117:0/1438867730 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7efe2819f4d0 con 0x7efe281066a0 2024-03-04T15:39:32.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.780+0000 7efe1effd700 1 -- 172.21.15.117:0/1438867730 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7efe24032030 con 0x7efe281066a0 2024-03-04T15:39:32.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.780+0000 7efe2ff91700 1 -- 172.21.15.117:0/1438867730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe281066a0 msgr2=0x7efe2819bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:32.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.780+0000 7efe2ff91700 1 --2- 172.21.15.117:0/1438867730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe281066a0 0x7efe2819bc60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efe24009290 tx=0x7efe240119c0 comp rx=0 tx=0).stop 2024-03-04T15:39:32.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.780+0000 7efe2ff91700 1 -- 172.21.15.117:0/1438867730 shutdown_connections 2024-03-04T15:39:32.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.780+0000 7efe2ff91700 1 --2- 172.21.15.117:0/1438867730 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe281066a0 0x7efe2819bc60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:32.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.780+0000 7efe2ff91700 1 -- 172.21.15.117:0/1438867730 >> 172.21.15.117:0/1438867730 conn(0x7efe281000d0 msgr2=0x7efe28100bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:32.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.780+0000 7efe2ff91700 1 -- 172.21.15.117:0/1438867730 shutdown_connections 2024-03-04T15:39:32.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:31.780+0000 7efe2ff91700 1 -- 172.21.15.117:0/1438867730 wait complete. 2024-03-04T15:39:32.425 INFO:teuthology.orchestra.run.smithi117.stdout:mgr not available, waiting (3/15)... 2024-03-04T15:39:32.569 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:32 smithi117 bash[21734]: debug 2024-03-04T15:39:32.100+0000 7f1127db0200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T15:39:33.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:32 smithi117 bash[21480]: cluster 2024-03-04T15:39:32.122897+0000 mon.a (mon.0) 10 : cluster 1 Activating manager daemon y 2024-03-04T15:39:33.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:32 smithi117 bash[21480]: cluster 2024-03-04T15:39:32.126192+0000 mon.a (mon.0) 11 : cluster 0 mgrmap e2: y(active, starting, since 0.00347272s) 2024-03-04T15:39:33.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:32 smithi117 bash[21480]: audit 2024-03-04T15:39:32.128908+0000 mon.a (mon.0) 12 : audit 0 from='mgr.14100 172.21.15.117:0/1448836833' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T15:39:33.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:32 smithi117 bash[21480]: audit 2024-03-04T15:39:32.129321+0000 mon.a (mon.0) 13 : audit 0 from='mgr.14100 172.21.15.117:0/1448836833' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T15:39:33.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:32 smithi117 bash[21480]: audit 2024-03-04T15:39:32.129608+0000 mon.a (mon.0) 14 : audit 0 from='mgr.14100 172.21.15.117:0/1448836833' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T15:39:33.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:32 smithi117 bash[21480]: audit 2024-03-04T15:39:32.129923+0000 mon.a (mon.0) 15 : audit 0 from='mgr.14100 172.21.15.117:0/1448836833' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:39:33.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:32 smithi117 bash[21480]: audit 2024-03-04T15:39:32.130439+0000 mon.a (mon.0) 16 : audit 0 from='mgr.14100 172.21.15.117:0/1448836833' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-04T15:39:33.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:32 smithi117 bash[21480]: cluster 2024-03-04T15:39:32.136058+0000 mon.a (mon.0) 17 : cluster 1 Manager daemon y is now available 2024-03-04T15:39:33.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:32 smithi117 bash[21480]: audit 2024-03-04T15:39:32.147377+0000 mon.a (mon.0) 18 : audit 1 from='mgr.14100 172.21.15.117:0/1448836833' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-04T15:39:33.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:32 smithi117 bash[21480]: audit 2024-03-04T15:39:32.150139+0000 mon.a (mon.0) 19 : audit 1 from='mgr.14100 172.21.15.117:0/1448836833' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-04T15:39:33.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:32 smithi117 bash[21480]: audit 2024-03-04T15:39:32.153652+0000 mon.a (mon.0) 20 : audit 1 from='mgr.14100 172.21.15.117:0/1448836833' entity='mgr.y' 2024-03-04T15:39:33.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:32 smithi117 bash[21480]: audit 2024-03-04T15:39:32.155533+0000 mon.a (mon.0) 21 : audit 1 from='mgr.14100 172.21.15.117:0/1448836833' entity='mgr.y' 2024-03-04T15:39:33.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:32 smithi117 bash[21480]: audit 2024-03-04T15:39:32.157269+0000 mon.a (mon.0) 22 : audit 1 from='mgr.14100 172.21.15.117:0/1448836833' entity='mgr.y' 2024-03-04T15:39:34.441 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:34 smithi117 bash[21480]: cluster 2024-03-04T15:39:33.134877+0000 mon.a (mon.0) 23 : cluster 0 mgrmap e3: y(active, since 1.01215s) 2024-03-04T15:39:36.431 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-03-04T15:39:36.431 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-03-04T15:39:36.431 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsid": "e40b5504-da3c-11ee-95c6-87774f69a715", 2024-03-04T15:39:36.431 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "health": { 2024-03-04T15:39:36.431 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-03-04T15:39:36.431 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-03-04T15:39:36.431 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-03-04T15:39:36.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:36.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-03-04T15:39:36.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-03-04T15:39:36.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 0 2024-03-04T15:39:36.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-03-04T15:39:36.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-03-04T15:39:36.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "a" 2024-03-04T15:39:36.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-03-04T15:39:36.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "quorum_age": 14, 2024-03-04T15:39:36.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "monmap": { 2024-03-04T15:39:36.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T15:39:36.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-03-04T15:39:36.434 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-03-04T15:39:36.435 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:36.435 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-03-04T15:39:36.435 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T15:39:36.435 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-03-04T15:39:36.435 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-03-04T15:39:36.435 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-03-04T15:39:36.435 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-03-04T15:39:36.435 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-03-04T15:39:36.438 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-03-04T15:39:36.438 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:36.438 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-03-04T15:39:36.438 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-03-04T15:39:36.438 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-03-04T15:39:36.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-03-04T15:39:36.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-03-04T15:39:36.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-03-04T15:39:36.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-03-04T15:39:36.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-03-04T15:39:36.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-03-04T15:39:36.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:36.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-03-04T15:39:36.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T15:39:36.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-03-04T15:39:36.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-03-04T15:39:36.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:36.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-03-04T15:39:36.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-04T15:39:36.439 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-03-04T15:39:36.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modules": [ 2024-03-04T15:39:36.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "iostat", 2024-03-04T15:39:36.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "nfs", 2024-03-04T15:39:36.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "restful" 2024-03-04T15:39:36.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ], 2024-03-04T15:39:36.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T15:39:36.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:36.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-03-04T15:39:36.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-03-04T15:39:36.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "modified": "2024-03-04T15:39:15.692156+0000", 2024-03-04T15:39:36.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "services": {} 2024-03-04T15:39:36.440 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout }, 2024-03-04T15:39:36.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-03-04T15:39:36.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-03-04T15:39:36.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.332+0000 7fa57799f700 1 Processor -- start 2024-03-04T15:39:36.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.332+0000 7fa57799f700 1 -- start start 2024-03-04T15:39:36.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.332+0000 7fa57799f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa57010d850 0x7fa57010dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:36.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.332+0000 7fa57799f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57010e240 con 0x7fa57010d850 2024-03-04T15:39:36.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.332+0000 7fa57573b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa57010d850 0x7fa57010dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:36.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.332+0000 7fa57573b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa57010d850 0x7fa57010dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41366/0 (socket says 172.21.15.117:41366) 2024-03-04T15:39:36.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.332+0000 7fa57573b700 1 -- 172.21.15.117:0/399053612 learned_addr learned my addr 172.21.15.117:0/399053612 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:36.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.332+0000 7fa57573b700 1 -- 172.21.15.117:0/399053612 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa57010c550 con 0x7fa57010d850 2024-03-04T15:39:36.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.332+0000 7fa57573b700 1 --2- 172.21.15.117:0/399053612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa57010d850 0x7fa57010dc70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa5600098a0 tx=0x7fa560009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de03625ae8527386 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:36.445 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.332+0000 7fa56ffff700 1 -- 172.21.15.117:0/399053612 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa560004160 con 0x7fa57010d850 2024-03-04T15:39:36.446 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.332+0000 7fa56ffff700 1 -- 172.21.15.117:0/399053612 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fa560004300 con 0x7fa57010d850 2024-03-04T15:39:36.446 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.332+0000 7fa56ffff700 1 -- 172.21.15.117:0/399053612 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa560011940 con 0x7fa57010d850 2024-03-04T15:39:36.446 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa57799f700 1 -- 172.21.15.117:0/399053612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa57010d850 msgr2=0x7fa57010dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:36.446 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa57799f700 1 --2- 172.21.15.117:0/399053612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa57010d850 0x7fa57010dc70 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fa5600098a0 tx=0x7fa560009bb0 comp rx=0 tx=0).stop 2024-03-04T15:39:36.446 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa57799f700 1 -- 172.21.15.117:0/399053612 shutdown_connections 2024-03-04T15:39:36.446 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa57799f700 1 --2- 172.21.15.117:0/399053612 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa57010d850 0x7fa57010dc70 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:36.446 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa57799f700 1 -- 172.21.15.117:0/399053612 >> 172.21.15.117:0/399053612 conn(0x7fa57007b220 msgr2=0x7fa57007d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:36.446 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa57799f700 1 -- 172.21.15.117:0/399053612 shutdown_connections 2024-03-04T15:39:36.446 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa57799f700 1 -- 172.21.15.117:0/399053612 wait complete. 2024-03-04T15:39:36.446 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa57799f700 1 Processor -- start 2024-03-04T15:39:36.446 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa57799f700 1 -- start start 2024-03-04T15:39:36.446 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa57799f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa57010d850 0x7fa5701a0070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:36.446 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa57799f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57010d430 con 0x7fa57010d850 2024-03-04T15:39:36.446 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa57573b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa57010d850 0x7fa5701a0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:36.447 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa57573b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa57010d850 0x7fa5701a0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:41374/0 (socket says 172.21.15.117:41374) 2024-03-04T15:39:36.447 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa57573b700 1 -- 172.21.15.117:0/2770952198 learned_addr learned my addr 172.21.15.117:0/2770952198 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:36.447 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa57573b700 1 -- 172.21.15.117:0/2770952198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5701a05b0 con 0x7fa57010d850 2024-03-04T15:39:36.447 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa57573b700 1 --2- 172.21.15.117:0/2770952198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa57010d850 0x7fa5701a0070 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa560011910 tx=0x7fa5600119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:36.447 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa56e7fc700 1 -- 172.21.15.117:0/2770952198 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa560003b20 con 0x7fa57010d850 2024-03-04T15:39:36.447 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa56e7fc700 1 -- 172.21.15.117:0/2770952198 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fa56001a890 con 0x7fa57010d850 2024-03-04T15:39:36.447 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa57799f700 1 -- 172.21.15.117:0/2770952198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5701a0840 con 0x7fa57010d850 2024-03-04T15:39:36.447 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa56e7fc700 1 -- 172.21.15.117:0/2770952198 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa5600197f0 con 0x7fa57010d850 2024-03-04T15:39:36.447 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa57799f700 1 -- 172.21.15.117:0/2770952198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5701a3530 con 0x7fa57010d850 2024-03-04T15:39:36.447 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa56e7fc700 1 -- 172.21.15.117:0/2770952198 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 45916+0+0 (secure 0 0 0) 0x7fa560022020 con 0x7fa57010d850 2024-03-04T15:39:36.447 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa56e7fc700 1 --2- 172.21.15.117:0/2770952198 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7fa558038b50 0x7fa55803b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:36.447 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa57799f700 1 -- 172.21.15.117:0/2770952198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa570063410 con 0x7fa57010d850 2024-03-04T15:39:36.447 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa56e7fc700 1 -- 172.21.15.117:0/2770952198 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fa56004cc60 con 0x7fa57010d850 2024-03-04T15:39:36.447 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.336+0000 7fa574f3a700 1 --2- 172.21.15.117:0/2770952198 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7fa558038b50 0x7fa55803b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:36.447 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.340+0000 7fa574f3a700 1 --2- 172.21.15.117:0/2770952198 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7fa558038b50 0x7fa55803b010 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa56400ad30 tx=0x7fa5640093f0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:36.448 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.344+0000 7fa56e7fc700 1 -- 172.21.15.117:0/2770952198 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa56002be10 con 0x7fa57010d850 2024-03-04T15:39:36.448 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.848+0000 7fa57799f700 1 -- 172.21.15.117:0/2770952198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fa5701a3950 con 0x7fa57010d850 2024-03-04T15:39:36.448 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.848+0000 7fa56e7fc700 1 -- 172.21.15.117:0/2770952198 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1239 (secure 0 0 0) 0x7fa56001aa40 con 0x7fa57010d850 2024-03-04T15:39:36.448 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.852+0000 7fa57799f700 1 -- 172.21.15.117:0/2770952198 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7fa558038b50 msgr2=0x7fa55803b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:36.448 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.852+0000 7fa57799f700 1 --2- 172.21.15.117:0/2770952198 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7fa558038b50 0x7fa55803b010 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa56400ad30 tx=0x7fa5640093f0 comp rx=0 tx=0).stop 2024-03-04T15:39:36.449 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.852+0000 7fa57799f700 1 -- 172.21.15.117:0/2770952198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa57010d850 msgr2=0x7fa5701a0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:36.449 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.852+0000 7fa57799f700 1 --2- 172.21.15.117:0/2770952198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa57010d850 0x7fa5701a0070 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fa560011910 tx=0x7fa5600119f0 comp rx=0 tx=0).stop 2024-03-04T15:39:36.449 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.852+0000 7fa57799f700 1 -- 172.21.15.117:0/2770952198 shutdown_connections 2024-03-04T15:39:36.449 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.852+0000 7fa57799f700 1 --2- 172.21.15.117:0/2770952198 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7fa558038b50 0x7fa55803b010 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:36.449 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.852+0000 7fa57799f700 1 --2- 172.21.15.117:0/2770952198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa57010d850 0x7fa5701a0070 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:36.449 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.852+0000 7fa57799f700 1 -- 172.21.15.117:0/2770952198 >> 172.21.15.117:0/2770952198 conn(0x7fa57007b220 msgr2=0x7fa57007be00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:36.449 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.852+0000 7fa57799f700 1 -- 172.21.15.117:0/2770952198 shutdown_connections 2024-03-04T15:39:36.449 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:35.852+0000 7fa57799f700 1 -- 172.21.15.117:0/2770952198 wait complete. 2024-03-04T15:39:36.450 INFO:teuthology.orchestra.run.smithi117.stdout:mgr is available 2024-03-04T15:39:36.468 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:36 smithi117 bash[21480]: cluster 2024-03-04T15:39:35.132429+0000 mon.a (mon.0) 24 : cluster 0 mgrmap e4: y(active, since 3s) 2024-03-04T15:39:36.468 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:36 smithi117 bash[21480]: audit 2024-03-04T15:39:35.850788+0000 mon.a (mon.0) 25 : audit 0 from='client.? 172.21.15.117:0/2770952198' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-03-04T15:39:38.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:37 smithi117 bash[21480]: audit 2024-03-04T15:39:37.721894+0000 mon.a (mon.0) 26 : audit 1 from='client.? 172.21.15.117:0/3185825955' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-03-04T15:39:38.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-03-04T15:39:38.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [global] 2024-03-04T15:39:38.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout fsid = e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T15:39:38.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-03-04T15:39:38.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-03-04T15:39:38.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-03-04T15:39:38.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-03-04T15:39:38.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-03-04T15:39:38.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [mgr] 2024-03-04T15:39:38.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-03-04T15:39:38.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 2024-03-04T15:39:38.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout [osd] 2024-03-04T15:39:38.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-03-04T15:39:38.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-03-04T15:39:38.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.360+0000 7fb479be1700 1 Processor -- start 2024-03-04T15:39:38.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.360+0000 7fb479be1700 1 -- start start 2024-03-04T15:39:38.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.360+0000 7fb479be1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb47407b700 0x7fb474079b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:38.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.360+0000 7fb479be1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47407a130 con 0x7fb47407b700 2024-03-04T15:39:38.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb4737fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb47407b700 0x7fb474079b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:38.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb4737fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb47407b700 0x7fb474079b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44218/0 (socket says 172.21.15.117:44218) 2024-03-04T15:39:38.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb4737fe700 1 -- 172.21.15.117:0/1256961074 learned_addr learned my addr 172.21.15.117:0/1256961074 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:38.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb4737fe700 1 -- 172.21.15.117:0/1256961074 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47407a2b0 con 0x7fb47407b700 2024-03-04T15:39:38.319 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb4737fe700 1 --2- 172.21.15.117:0/1256961074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb47407b700 0x7fb474079b60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0098a0 tx=0x7fb45c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6bc8bb0adfe6abd server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:38.319 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb4727fc700 1 -- 172.21.15.117:0/1256961074 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb45c004160 con 0x7fb47407b700 2024-03-04T15:39:38.319 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb4727fc700 1 -- 172.21.15.117:0/1256961074 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fb45c004300 con 0x7fb47407b700 2024-03-04T15:39:38.319 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb4727fc700 1 -- 172.21.15.117:0/1256961074 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb45c011940 con 0x7fb47407b700 2024-03-04T15:39:38.319 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb479be1700 1 -- 172.21.15.117:0/1256961074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb47407b700 msgr2=0x7fb474079b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:38.319 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb479be1700 1 --2- 172.21.15.117:0/1256961074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb47407b700 0x7fb474079b60 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fb45c0098a0 tx=0x7fb45c009bb0 comp rx=0 tx=0).stop 2024-03-04T15:39:38.319 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb479be1700 1 -- 172.21.15.117:0/1256961074 shutdown_connections 2024-03-04T15:39:38.319 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb479be1700 1 --2- 172.21.15.117:0/1256961074 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb47407b700 0x7fb474079b60 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:38.319 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb479be1700 1 -- 172.21.15.117:0/1256961074 >> 172.21.15.117:0/1256961074 conn(0x7fb4741011c0 msgr2=0x7fb474103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:38.319 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb479be1700 1 -- 172.21.15.117:0/1256961074 shutdown_connections 2024-03-04T15:39:38.319 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb479be1700 1 -- 172.21.15.117:0/1256961074 wait complete. 2024-03-04T15:39:38.319 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb479be1700 1 Processor -- start 2024-03-04T15:39:38.319 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb479be1700 1 -- start start 2024-03-04T15:39:38.319 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb479be1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb47407b700 0x7fb47419bc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:38.319 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb479be1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb47410c0a0 con 0x7fb47407b700 2024-03-04T15:39:38.319 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb4737fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb47407b700 0x7fb47419bc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:38.320 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb4737fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb47407b700 0x7fb47419bc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44224/0 (socket says 172.21.15.117:44224) 2024-03-04T15:39:38.320 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb4737fe700 1 -- 172.21.15.117:0/3185825955 learned_addr learned my addr 172.21.15.117:0/3185825955 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:38.320 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb4737fe700 1 -- 172.21.15.117:0/3185825955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb47419c1b0 con 0x7fb47407b700 2024-03-04T15:39:38.320 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb4737fe700 1 --2- 172.21.15.117:0/3185825955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb47407b700 0x7fb47419bc70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009290 tx=0x7fb45c0119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:38.320 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb470ff9700 1 -- 172.21.15.117:0/3185825955 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb45c003930 con 0x7fb47407b700 2024-03-04T15:39:38.320 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb470ff9700 1 -- 172.21.15.117:0/3185825955 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fb45c003ad0 con 0x7fb47407b700 2024-03-04T15:39:38.320 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb479be1700 1 -- 172.21.15.117:0/3185825955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb47419c440 con 0x7fb47407b700 2024-03-04T15:39:38.320 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb470ff9700 1 -- 172.21.15.117:0/3185825955 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb45c019720 con 0x7fb47407b700 2024-03-04T15:39:38.320 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.364+0000 7fb479be1700 1 -- 172.21.15.117:0/3185825955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb47419f130 con 0x7fb47407b700 2024-03-04T15:39:38.320 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.368+0000 7fb470ff9700 1 -- 172.21.15.117:0/3185825955 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 45916+0+0 (secure 0 0 0) 0x7fb45c0199f0 con 0x7fb47407b700 2024-03-04T15:39:38.320 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.368+0000 7fb470ff9700 1 --2- 172.21.15.117:0/3185825955 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7fb460038b50 0x7fb46003b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:38.321 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.368+0000 7fb470ff9700 1 -- 172.21.15.117:0/3185825955 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb45c04d0b0 con 0x7fb47407b700 2024-03-04T15:39:38.321 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.368+0000 7fb479be1700 1 -- 172.21.15.117:0/3185825955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb474063410 con 0x7fb47407b700 2024-03-04T15:39:38.321 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.368+0000 7fb472ffd700 1 --2- 172.21.15.117:0/3185825955 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7fb460038b50 0x7fb46003b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:38.321 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.368+0000 7fb472ffd700 1 --2- 172.21.15.117:0/3185825955 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7fb460038b50 0x7fb46003b010 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb46400ad30 tx=0x7fb4640093f0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:38.321 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.372+0000 7fb470ff9700 1 -- 172.21.15.117:0/3185825955 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb45c04cad0 con 0x7fb47407b700 2024-03-04T15:39:38.321 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.720+0000 7fb479be1700 1 -- 172.21.15.117:0/3185825955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7fb4741a0600 con 0x7fb47407b700 2024-03-04T15:39:38.321 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.720+0000 7fb470ff9700 1 -- 172.21.15.117:0/3185825955 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+282 (secure 0 0 0) 0x7fb45c019ce0 con 0x7fb47407b700 2024-03-04T15:39:38.321 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.724+0000 7fb479be1700 1 -- 172.21.15.117:0/3185825955 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7fb460038b50 msgr2=0x7fb46003b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:38.321 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.724+0000 7fb479be1700 1 --2- 172.21.15.117:0/3185825955 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7fb460038b50 0x7fb46003b010 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fb46400ad30 tx=0x7fb4640093f0 comp rx=0 tx=0).stop 2024-03-04T15:39:38.321 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.724+0000 7fb479be1700 1 -- 172.21.15.117:0/3185825955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb47407b700 msgr2=0x7fb47419bc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:38.321 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.724+0000 7fb479be1700 1 --2- 172.21.15.117:0/3185825955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb47407b700 0x7fb47419bc70 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fb45c009290 tx=0x7fb45c0119c0 comp rx=0 tx=0).stop 2024-03-04T15:39:38.321 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.724+0000 7fb479be1700 1 -- 172.21.15.117:0/3185825955 shutdown_connections 2024-03-04T15:39:38.321 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.724+0000 7fb479be1700 1 --2- 172.21.15.117:0/3185825955 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7fb460038b50 0x7fb46003b010 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:38.321 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.724+0000 7fb479be1700 1 --2- 172.21.15.117:0/3185825955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb47407b700 0x7fb47419bc70 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:38.322 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.724+0000 7fb479be1700 1 -- 172.21.15.117:0/3185825955 >> 172.21.15.117:0/3185825955 conn(0x7fb4741011c0 msgr2=0x7fb474101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:38.322 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.724+0000 7fb479be1700 1 -- 172.21.15.117:0/3185825955 shutdown_connections 2024-03-04T15:39:38.322 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:37.724+0000 7fb479be1700 1 -- 172.21.15.117:0/3185825955 wait complete. 2024-03-04T15:39:38.322 INFO:teuthology.orchestra.run.smithi117.stdout:Enabling cephadm module... 2024-03-04T15:39:40.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:39 smithi117 bash[21480]: audit 2024-03-04T15:39:39.614316+0000 mon.a (mon.0) 27 : audit 1 from='client.? 172.21.15.117:0/288824785' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-03-04T15:39:40.070 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:39 smithi117 bash[21734]: ignoring --setuser ceph since I am not root 2024-03-04T15:39:40.071 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:39 smithi117 bash[21734]: ignoring --setgroup ceph since I am not root 2024-03-04T15:39:40.071 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:39 smithi117 bash[21734]: debug 2024-03-04T15:39:39.856+0000 7f1c6da6b200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T15:39:40.217 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.148+0000 7fd023291700 1 Processor -- start 2024-03-04T15:39:40.217 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.148+0000 7fd023291700 1 -- start start 2024-03-04T15:39:40.217 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd023291700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01c106720 0x7fd01c108b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:40.217 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd023291700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c0796f0 con 0x7fd01c106720 2024-03-04T15:39:40.217 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd02102d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01c106720 0x7fd01c108b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:40.217 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd02102d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01c106720 0x7fd01c108b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44238/0 (socket says 172.21.15.117:44238) 2024-03-04T15:39:40.217 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd02102d700 1 -- 172.21.15.117:0/3465760351 learned_addr learned my addr 172.21.15.117:0/3465760351 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:40.217 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd02102d700 1 -- 172.21.15.117:0/3465760351 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01c109050 con 0x7fd01c106720 2024-03-04T15:39:40.217 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd02102d700 1 --2- 172.21.15.117:0/3465760351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01c106720 0x7fd01c108b10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd0180098a0 tx=0x7fd018009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d8e1dd5645a98d6 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:40.217 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd013fff700 1 -- 172.21.15.117:0/3465760351 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd018004160 con 0x7fd01c106720 2024-03-04T15:39:40.217 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd013fff700 1 -- 172.21.15.117:0/3465760351 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fd018004300 con 0x7fd01c106720 2024-03-04T15:39:40.217 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd013fff700 1 -- 172.21.15.117:0/3465760351 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd018011940 con 0x7fd01c106720 2024-03-04T15:39:40.218 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd023291700 1 -- 172.21.15.117:0/3465760351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01c106720 msgr2=0x7fd01c108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:40.218 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd023291700 1 --2- 172.21.15.117:0/3465760351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01c106720 0x7fd01c108b10 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fd0180098a0 tx=0x7fd018009bb0 comp rx=0 tx=0).stop 2024-03-04T15:39:40.218 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd023291700 1 -- 172.21.15.117:0/3465760351 shutdown_connections 2024-03-04T15:39:40.218 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd023291700 1 --2- 172.21.15.117:0/3465760351 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01c106720 0x7fd01c108b10 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:40.218 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd023291700 1 -- 172.21.15.117:0/3465760351 >> 172.21.15.117:0/3465760351 conn(0x7fd01c1000f0 msgr2=0x7fd01c102550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:40.218 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd023291700 1 -- 172.21.15.117:0/3465760351 shutdown_connections 2024-03-04T15:39:40.218 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd023291700 1 -- 172.21.15.117:0/3465760351 wait complete. 2024-03-04T15:39:40.218 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd023291700 1 Processor -- start 2024-03-04T15:39:40.218 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd023291700 1 -- start start 2024-03-04T15:39:40.218 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd023291700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01c106720 0x7fd01c105fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:40.218 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd023291700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd01c109e70 con 0x7fd01c106720 2024-03-04T15:39:40.218 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd02102d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01c106720 0x7fd01c105fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:40.218 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd02102d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01c106720 0x7fd01c105fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44246/0 (socket says 172.21.15.117:44246) 2024-03-04T15:39:40.218 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd02102d700 1 -- 172.21.15.117:0/288824785 learned_addr learned my addr 172.21.15.117:0/288824785 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:40.218 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd02102d700 1 -- 172.21.15.117:0/288824785 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd01c1064e0 con 0x7fd01c106720 2024-03-04T15:39:40.219 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd02102d700 1 --2- 172.21.15.117:0/288824785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01c106720 0x7fd01c105fa0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd018011910 tx=0x7fd0180119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:40.219 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd0127fc700 1 -- 172.21.15.117:0/288824785 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd018003ac0 con 0x7fd01c106720 2024-03-04T15:39:40.219 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd0127fc700 1 -- 172.21.15.117:0/288824785 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fd01801a890 con 0x7fd01c106720 2024-03-04T15:39:40.219 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd023291700 1 -- 172.21.15.117:0/288824785 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd01c1046d0 con 0x7fd01c106720 2024-03-04T15:39:40.219 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd0127fc700 1 -- 172.21.15.117:0/288824785 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd0180197a0 con 0x7fd01c106720 2024-03-04T15:39:40.219 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.152+0000 7fd023291700 1 -- 172.21.15.117:0/288824785 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd01c104c10 con 0x7fd01c106720 2024-03-04T15:39:40.219 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.156+0000 7fd0127fc700 1 -- 172.21.15.117:0/288824785 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 4) v1 ==== 45916+0+0 (secure 0 0 0) 0x7fd018019aa0 con 0x7fd01c106720 2024-03-04T15:39:40.219 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.156+0000 7fd023291700 1 -- 172.21.15.117:0/288824785 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd01c063410 con 0x7fd01c106720 2024-03-04T15:39:40.219 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.156+0000 7fd0127fc700 1 --2- 172.21.15.117:0/288824785 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7fd008038b50 0x7fd00803b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:40.219 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.156+0000 7fd0127fc700 1 -- 172.21.15.117:0/288824785 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fd01804d370 con 0x7fd01c106720 2024-03-04T15:39:40.219 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.156+0000 7fd02082c700 1 --2- 172.21.15.117:0/288824785 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7fd008038b50 0x7fd00803b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:40.219 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.156+0000 7fd02082c700 1 --2- 172.21.15.117:0/288824785 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7fd008038b50 0x7fd00803b010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd00c006fd0 tx=0x7fd00c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:40.219 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.160+0000 7fd0127fc700 1 -- 172.21.15.117:0/288824785 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd018023df0 con 0x7fd01c106720 2024-03-04T15:39:40.219 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.612+0000 7fd023291700 1 -- 172.21.15.117:0/288824785 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fd01c105390 con 0x7fd01c106720 2024-03-04T15:39:40.219 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.664+0000 7fd0127fc700 1 -- 172.21.15.117:0/288824785 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fd01804cd90 con 0x7fd01c106720 2024-03-04T15:39:40.219 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.668+0000 7fd0127fc700 1 -- 172.21.15.117:0/288824785 <== mon.0 v2:172.21.15.117:3300/0 8 ==== mgrmap(e 5) v1 ==== 45927+0+0 (secure 0 0 0) 0x7fd01802d7c0 con 0x7fd01c106720 2024-03-04T15:39:40.220 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.668+0000 7fd023291700 1 -- 172.21.15.117:0/288824785 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7fd008038b50 msgr2=0x7fd00803b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:40.220 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.668+0000 7fd023291700 1 --2- 172.21.15.117:0/288824785 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7fd008038b50 0x7fd00803b010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd00c006fd0 tx=0x7fd00c006e40 comp rx=0 tx=0).stop 2024-03-04T15:39:40.220 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.668+0000 7fd023291700 1 -- 172.21.15.117:0/288824785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01c106720 msgr2=0x7fd01c105fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:40.220 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.668+0000 7fd023291700 1 --2- 172.21.15.117:0/288824785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01c106720 0x7fd01c105fa0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd018011910 tx=0x7fd0180119f0 comp rx=0 tx=0).stop 2024-03-04T15:39:40.220 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.668+0000 7fd023291700 1 -- 172.21.15.117:0/288824785 shutdown_connections 2024-03-04T15:39:40.220 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.668+0000 7fd023291700 1 --2- 172.21.15.117:0/288824785 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7fd008038b50 0x7fd00803b010 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:40.220 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.668+0000 7fd023291700 1 --2- 172.21.15.117:0/288824785 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd01c106720 0x7fd01c105fa0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:40.220 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.668+0000 7fd023291700 1 -- 172.21.15.117:0/288824785 >> 172.21.15.117:0/288824785 conn(0x7fd01c1000f0 msgr2=0x7fd01c100ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:40.220 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.668+0000 7fd023291700 1 -- 172.21.15.117:0/288824785 shutdown_connections 2024-03-04T15:39:40.220 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:39.668+0000 7fd023291700 1 -- 172.21.15.117:0/288824785 wait complete. 2024-03-04T15:39:41.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:40 smithi117 bash[21480]: audit 2024-03-04T15:39:39.666156+0000 mon.a (mon.0) 28 : audit 1 from='client.? 172.21.15.117:0/288824785' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-03-04T15:39:41.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:40 smithi117 bash[21480]: cluster 2024-03-04T15:39:39.670454+0000 mon.a (mon.0) 29 : cluster 0 mgrmap e5: y(active, since 7s) 2024-03-04T15:39:41.672 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:41 smithi117 bash[21734]: debug 2024-03-04T15:39:41.420+0000 7f1c6da6b200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T15:39:41.672 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:41 smithi117 bash[21734]: debug 2024-03-04T15:39:41.519+0000 7f1c6da6b200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T15:39:42.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:41 smithi117 bash[21480]: audit 2024-03-04T15:39:41.513341+0000 mon.a (mon.0) 30 : audit 0 from='client.? 172.21.15.117:0/1393162426' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-04T15:39:42.070 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:41 smithi117 bash[21734]: debug 2024-03-04T15:39:41.767+0000 7f1c6da6b200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T15:39:42.099 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-03-04T15:39:42.099 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-03-04T15:39:42.099 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-04T15:39:42.099 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-03-04T15:39:42.099 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-04T15:39:42.099 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-03-04T15:39:42.099 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f63961c4700 1 Processor -- start 2024-03-04T15:39:42.100 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f63961c4700 1 -- start start 2024-03-04T15:39:42.100 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f63961c4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6390104490 0x7f63901048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:42.100 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f63961c4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6390078ea0 con 0x7f6390104490 2024-03-04T15:39:42.100 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f638f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6390104490 0x7f63901048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:42.100 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f638f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6390104490 0x7f63901048b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44282/0 (socket says 172.21.15.117:44282) 2024-03-04T15:39:42.100 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f638f7fe700 1 -- 172.21.15.117:0/1270546448 learned_addr learned my addr 172.21.15.117:0/1270546448 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:42.100 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f638f7fe700 1 -- 172.21.15.117:0/1270546448 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6390104df0 con 0x7f6390104490 2024-03-04T15:39:42.100 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f638f7fe700 1 --2- 172.21.15.117:0/1270546448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6390104490 0x7f63901048b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f63800098a0 tx=0x7f6380009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b789d10d13d2de36 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:42.100 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f638e7fc700 1 -- 172.21.15.117:0/1270546448 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6380004160 con 0x7f6390104490 2024-03-04T15:39:42.100 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f638e7fc700 1 -- 172.21.15.117:0/1270546448 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f6380004300 con 0x7f6390104490 2024-03-04T15:39:42.100 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f638e7fc700 1 -- 172.21.15.117:0/1270546448 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6380011940 con 0x7f6390104490 2024-03-04T15:39:42.100 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f63961c4700 1 -- 172.21.15.117:0/1270546448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6390104490 msgr2=0x7f63901048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:42.100 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f63961c4700 1 --2- 172.21.15.117:0/1270546448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6390104490 0x7f63901048b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f63800098a0 tx=0x7f6380009bb0 comp rx=0 tx=0).stop 2024-03-04T15:39:42.100 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f63961c4700 1 -- 172.21.15.117:0/1270546448 shutdown_connections 2024-03-04T15:39:42.100 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f63961c4700 1 --2- 172.21.15.117:0/1270546448 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6390104490 0x7f63901048b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:42.101 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f63961c4700 1 -- 172.21.15.117:0/1270546448 >> 172.21.15.117:0/1270546448 conn(0x7f63901000d0 msgr2=0x7f6390102510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:42.101 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f63961c4700 1 -- 172.21.15.117:0/1270546448 shutdown_connections 2024-03-04T15:39:42.101 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f63961c4700 1 -- 172.21.15.117:0/1270546448 wait complete. 2024-03-04T15:39:42.101 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f63961c4700 1 Processor -- start 2024-03-04T15:39:42.101 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f63961c4700 1 -- start start 2024-03-04T15:39:42.101 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f63961c4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6390104490 0x7f639019bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:42.101 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f63961c4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6390105c80 con 0x7f6390104490 2024-03-04T15:39:42.101 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f638f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6390104490 0x7f639019bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:42.101 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f638f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6390104490 0x7f639019bc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44288/0 (socket says 172.21.15.117:44288) 2024-03-04T15:39:42.101 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f638f7fe700 1 -- 172.21.15.117:0/1393162426 learned_addr learned my addr 172.21.15.117:0/1393162426 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:42.101 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f638f7fe700 1 -- 172.21.15.117:0/1393162426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f639019c150 con 0x7f6390104490 2024-03-04T15:39:42.101 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.028+0000 7f638f7fe700 1 --2- 172.21.15.117:0/1393162426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6390104490 0x7f639019bc10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6380011910 tx=0x7f63800119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:42.101 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.032+0000 7f638cff9700 1 -- 172.21.15.117:0/1393162426 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6380003ac0 con 0x7f6390104490 2024-03-04T15:39:42.101 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.032+0000 7f638cff9700 1 -- 172.21.15.117:0/1393162426 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f638001a890 con 0x7f6390104490 2024-03-04T15:39:42.101 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.032+0000 7f638cff9700 1 -- 172.21.15.117:0/1393162426 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f63800197a0 con 0x7f6390104490 2024-03-04T15:39:42.102 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.032+0000 7f63961c4700 1 -- 172.21.15.117:0/1393162426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f639019c3e0 con 0x7f6390104490 2024-03-04T15:39:42.102 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.032+0000 7f63961c4700 1 -- 172.21.15.117:0/1393162426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f639019f0d0 con 0x7f6390104490 2024-03-04T15:39:42.102 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.032+0000 7f638cff9700 1 -- 172.21.15.117:0/1393162426 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 45927+0+0 (secure 0 0 0) 0x7f6380019a50 con 0x7f6390104490 2024-03-04T15:39:42.102 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.032+0000 7f63961c4700 1 -- 172.21.15.117:0/1393162426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6390063410 con 0x7f6390104490 2024-03-04T15:39:42.102 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.032+0000 7f638cff9700 1 --2- 172.21.15.117:0/1393162426 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f637c038bf0 0x7f637c03b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:42.102 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.032+0000 7f638cff9700 1 -- 172.21.15.117:0/1393162426 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f638004d2a0 con 0x7f6390104490 2024-03-04T15:39:42.102 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.032+0000 7f638effd700 1 -- 172.21.15.117:0/1393162426 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f637c038bf0 msgr2=0x7f637c03b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/427955642 2024-03-04T15:39:42.102 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.032+0000 7f638effd700 1 --2- 172.21.15.117:0/1393162426 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f637c038bf0 0x7f637c03b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T15:39:42.102 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.036+0000 7f638cff9700 1 -- 172.21.15.117:0/1393162426 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f638001ad90 con 0x7f6390104490 2024-03-04T15:39:42.102 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.236+0000 7f638effd700 1 -- 172.21.15.117:0/1393162426 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f637c038bf0 msgr2=0x7f637c03b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/427955642 2024-03-04T15:39:42.102 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.236+0000 7f638effd700 1 --2- 172.21.15.117:0/1393162426 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f637c038bf0 0x7f637c03b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T15:39:42.102 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.507+0000 7f63961c4700 1 -- 172.21.15.117:0/1393162426 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f639002d900 con 0x7f6390104490 2024-03-04T15:39:42.102 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.511+0000 7f638cff9700 1 -- 172.21.15.117:0/1393162426 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f638002d960 con 0x7f6390104490 2024-03-04T15:39:42.102 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.511+0000 7f63961c4700 1 -- 172.21.15.117:0/1393162426 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f637c038bf0 msgr2=0x7f637c03b0b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T15:39:42.102 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.511+0000 7f63961c4700 1 --2- 172.21.15.117:0/1393162426 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f637c038bf0 0x7f637c03b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:42.103 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.511+0000 7f63961c4700 1 -- 172.21.15.117:0/1393162426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6390104490 msgr2=0x7f639019bc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:42.103 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.511+0000 7f63961c4700 1 --2- 172.21.15.117:0/1393162426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6390104490 0x7f639019bc10 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6380011910 tx=0x7f63800119f0 comp rx=0 tx=0).stop 2024-03-04T15:39:42.103 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.511+0000 7f63961c4700 1 -- 172.21.15.117:0/1393162426 shutdown_connections 2024-03-04T15:39:42.103 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.511+0000 7f63961c4700 1 --2- 172.21.15.117:0/1393162426 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f637c038bf0 0x7f637c03b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:42.103 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.511+0000 7f63961c4700 1 --2- 172.21.15.117:0/1393162426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6390104490 0x7f639019bc10 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:42.103 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.511+0000 7f63961c4700 1 -- 172.21.15.117:0/1393162426 >> 172.21.15.117:0/1393162426 conn(0x7f63901000d0 msgr2=0x7f6390192850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:42.103 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.511+0000 7f63961c4700 1 -- 172.21.15.117:0/1393162426 shutdown_connections 2024-03-04T15:39:42.103 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:41.511+0000 7f63961c4700 1 -- 172.21.15.117:0/1393162426 wait complete. 2024-03-04T15:39:42.103 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for the mgr to restart... 2024-03-04T15:39:42.103 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mgr epoch 5... 2024-03-04T15:39:42.820 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:42 smithi117 bash[21734]: debug 2024-03-04T15:39:42.555+0000 7f1c6da6b200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T15:39:42.820 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:42 smithi117 bash[21734]: debug 2024-03-04T15:39:42.667+0000 7f1c6da6b200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T15:39:42.820 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:42 smithi117 bash[21734]: debug 2024-03-04T15:39:42.775+0000 7f1c6da6b200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T15:39:43.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:43 smithi117 bash[21734]: debug 2024-03-04T15:39:43.127+0000 7f1c6da6b200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T15:39:43.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:43 smithi117 bash[21734]: debug 2024-03-04T15:39:43.239+0000 7f1c6da6b200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T15:39:43.571 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:43 smithi117 bash[21734]: debug 2024-03-04T15:39:43.339+0000 7f1c6da6b200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T15:39:44.070 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:43 smithi117 bash[21734]: debug 2024-03-04T15:39:43.691+0000 7f1c6da6b200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T15:39:44.070 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:43 smithi117 bash[21734]: debug 2024-03-04T15:39:43.923+0000 7f1c6da6b200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T15:39:44.566 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:44 smithi117 bash[21734]: debug 2024-03-04T15:39:44.183+0000 7f1c6da6b200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T15:39:44.566 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:44 smithi117 bash[21734]: debug 2024-03-04T15:39:44.295+0000 7f1c6da6b200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T15:39:44.820 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:44 smithi117 bash[21734]: debug 2024-03-04T15:39:44.563+0000 7f1c6da6b200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T15:39:44.821 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:44 smithi117 bash[21734]: debug 2024-03-04T15:39:44.799+0000 7f1c6da6b200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T15:39:45.562 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:45 smithi117 bash[21734]: debug 2024-03-04T15:39:45.203+0000 7f1c6da6b200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T15:39:45.562 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:45 smithi117 bash[21734]: debug 2024-03-04T15:39:45.319+0000 7f1c6da6b200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T15:39:45.820 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:45 smithi117 bash[21734]: debug 2024-03-04T15:39:45.559+0000 7f1c6da6b200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T15:39:46.820 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:46 smithi117 bash[21734]: debug 2024-03-04T15:39:46.543+0000 7f1c6da6b200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T15:39:46.821 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:46 smithi117 bash[21734]: debug 2024-03-04T15:39:46.659+0000 7f1c6da6b200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T15:39:47.321 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:46 smithi117 bash[21734]: debug 2024-03-04T15:39:46.943+0000 7f1c6da6b200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T15:39:47.321 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:47 smithi117 bash[21734]: debug 2024-03-04T15:39:47.055+0000 7f1c6da6b200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T15:39:47.821 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:47 smithi117 bash[21734]: debug 2024-03-04T15:39:47.499+0000 7f1c6da6b200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T15:39:47.821 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:47 smithi117 bash[21734]: debug 2024-03-04T15:39:47.619+0000 7f1c6da6b200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T15:39:47.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:47 smithi117 bash[21480]: cluster 2024-03-04T15:39:47.643245+0000 mon.a (mon.0) 31 : cluster 1 Active manager daemon y restarted 2024-03-04T15:39:47.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:47 smithi117 bash[21480]: cluster 2024-03-04T15:39:47.643707+0000 mon.a (mon.0) 32 : cluster 1 Activating manager daemon y 2024-03-04T15:39:47.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:47 smithi117 bash[21480]: cluster 2024-03-04T15:39:47.652155+0000 mon.a (mon.0) 33 : cluster 0 osdmap e2: 0 total, 0 up, 0 in 2024-03-04T15:39:47.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:47 smithi117 bash[21480]: cluster 2024-03-04T15:39:47.652383+0000 mon.a (mon.0) 34 : cluster 0 mgrmap e6: y(active, starting, since 0.00885333s) 2024-03-04T15:39:47.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:47 smithi117 bash[21480]: audit 2024-03-04T15:39:47.654630+0000 mon.a (mon.0) 35 : audit 0 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:39:47.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:47 smithi117 bash[21480]: audit 2024-03-04T15:39:47.656420+0000 mon.a (mon.0) 36 : audit 0 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-04T15:39:47.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:47 smithi117 bash[21480]: audit 2024-03-04T15:39:47.657213+0000 mon.a (mon.0) 37 : audit 0 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T15:39:47.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:47 smithi117 bash[21480]: audit 2024-03-04T15:39:47.657538+0000 mon.a (mon.0) 38 : audit 0 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T15:39:47.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:47 smithi117 bash[21480]: audit 2024-03-04T15:39:47.657712+0000 mon.a (mon.0) 39 : audit 0 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T15:39:47.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:47 smithi117 bash[21480]: cluster 2024-03-04T15:39:47.683416+0000 mon.a (mon.0) 40 : cluster 1 Manager daemon y is now available 2024-03-04T15:39:49.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:48 smithi117 bash[21480]: cephadm 2024-03-04T15:39:47.702830+0000 mgr.y (mgr.14120) 1 : cephadm 1 Found migration_current of "None". Setting to last migration. 2024-03-04T15:39:49.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:48 smithi117 bash[21480]: audit 2024-03-04T15:39:47.704555+0000 mon.a (mon.0) 41 : audit 1 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' 2024-03-04T15:39:49.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:48 smithi117 bash[21480]: audit 2024-03-04T15:39:47.709793+0000 mon.a (mon.0) 42 : audit 1 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' 2024-03-04T15:39:49.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:48 smithi117 bash[21480]: audit 2024-03-04T15:39:47.713307+0000 mon.a (mon.0) 43 : audit 0 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:39:49.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:48 smithi117 bash[21480]: audit 2024-03-04T15:39:47.715047+0000 mon.a (mon.0) 44 : audit 0 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:39:49.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:48 smithi117 bash[21480]: audit 2024-03-04T15:39:47.719751+0000 mon.a (mon.0) 45 : audit 0 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:39:49.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:48 smithi117 bash[21480]: audit 2024-03-04T15:39:47.728828+0000 mon.a (mon.0) 46 : audit 1 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-04T15:39:49.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:48 smithi117 bash[21480]: audit 2024-03-04T15:39:47.732422+0000 mon.a (mon.0) 47 : audit 1 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-04T15:39:49.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:48 smithi117 bash[21480]: cluster 2024-03-04T15:39:48.656625+0000 mon.a (mon.0) 48 : cluster 0 mgrmap e7: y(active, since 1.01309s) 2024-03-04T15:39:49.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-03-04T15:39:49.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-03-04T15:39:49.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-04T15:39:49.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-03-04T15:39:49.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.923+0000 7f6db7929700 1 Processor -- start 2024-03-04T15:39:49.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.923+0000 7f6db7929700 1 -- start start 2024-03-04T15:39:49.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.923+0000 7f6db7929700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db007b700 0x7f6db0079b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:49.314 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.923+0000 7f6db7929700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db007a130 con 0x7f6db007b700 2024-03-04T15:39:49.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.923+0000 7f6db56c5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db007b700 0x7f6db0079b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:49.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.923+0000 7f6db56c5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db007b700 0x7f6db0079b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44304/0 (socket says 172.21.15.117:44304) 2024-03-04T15:39:49.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.923+0000 7f6db56c5700 1 -- 172.21.15.117:0/3865258449 learned_addr learned my addr 172.21.15.117:0/3865258449 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:49.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.923+0000 7f6db56c5700 1 -- 172.21.15.117:0/3865258449 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db007a2b0 con 0x7f6db007b700 2024-03-04T15:39:49.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.923+0000 7f6db56c5700 1 --2- 172.21.15.117:0/3865258449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db007b700 0x7f6db0079b60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6da40098a0 tx=0x7f6da4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d97917b9d6be9a server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:49.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.923+0000 7f6da3fff700 1 -- 172.21.15.117:0/3865258449 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6da4004160 con 0x7f6db007b700 2024-03-04T15:39:49.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.923+0000 7f6da3fff700 1 -- 172.21.15.117:0/3865258449 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f6da4004300 con 0x7f6db007b700 2024-03-04T15:39:49.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.923+0000 7f6da3fff700 1 -- 172.21.15.117:0/3865258449 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6da4011940 con 0x7f6db007b700 2024-03-04T15:39:49.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.923+0000 7f6db7929700 1 -- 172.21.15.117:0/3865258449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db007b700 msgr2=0x7f6db0079b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:49.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.923+0000 7f6db7929700 1 --2- 172.21.15.117:0/3865258449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db007b700 0x7f6db0079b60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6da40098a0 tx=0x7f6da4009bb0 comp rx=0 tx=0).stop 2024-03-04T15:39:49.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.923+0000 7f6db7929700 1 -- 172.21.15.117:0/3865258449 shutdown_connections 2024-03-04T15:39:49.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.923+0000 7f6db7929700 1 --2- 172.21.15.117:0/3865258449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db007b700 0x7f6db0079b60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:49.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.923+0000 7f6db7929700 1 -- 172.21.15.117:0/3865258449 >> 172.21.15.117:0/3865258449 conn(0x7f6db0101220 msgr2=0x7f6db0103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:49.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.923+0000 7f6db7929700 1 -- 172.21.15.117:0/3865258449 shutdown_connections 2024-03-04T15:39:49.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.923+0000 7f6db7929700 1 -- 172.21.15.117:0/3865258449 wait complete. 2024-03-04T15:39:49.315 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.927+0000 7f6db7929700 1 Processor -- start 2024-03-04T15:39:49.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.927+0000 7f6db7929700 1 -- start start 2024-03-04T15:39:49.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.927+0000 7f6db7929700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db007b700 0x7f6db019bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:49.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.927+0000 7f6db7929700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db010c0a0 con 0x7f6db007b700 2024-03-04T15:39:49.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.927+0000 7f6db56c5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db007b700 0x7f6db019bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:49.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.927+0000 7f6db56c5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db007b700 0x7f6db019bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44320/0 (socket says 172.21.15.117:44320) 2024-03-04T15:39:49.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.927+0000 7f6db56c5700 1 -- 172.21.15.117:0/3793030402 learned_addr learned my addr 172.21.15.117:0/3793030402 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:49.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.927+0000 7f6db56c5700 1 -- 172.21.15.117:0/3793030402 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db019c100 con 0x7f6db007b700 2024-03-04T15:39:49.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.927+0000 7f6db56c5700 1 --2- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db007b700 0x7f6db019bbc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6da4003e40 tx=0x7f6da40119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:49.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.927+0000 7f6da27fc700 1 -- 172.21.15.117:0/3793030402 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6da4003b20 con 0x7f6db007b700 2024-03-04T15:39:49.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.927+0000 7f6da27fc700 1 -- 172.21.15.117:0/3793030402 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f6da401a890 con 0x7f6db007b700 2024-03-04T15:39:49.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.927+0000 7f6db7929700 1 -- 172.21.15.117:0/3793030402 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db019c390 con 0x7f6db007b700 2024-03-04T15:39:49.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.927+0000 7f6da27fc700 1 -- 172.21.15.117:0/3793030402 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6da4024890 con 0x7f6db007b700 2024-03-04T15:39:49.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.927+0000 7f6db7929700 1 -- 172.21.15.117:0/3793030402 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db019f080 con 0x7f6db007b700 2024-03-04T15:39:49.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.927+0000 7f6da27fc700 1 -- 172.21.15.117:0/3793030402 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 5) v1 ==== 45927+0+0 (secure 0 0 0) 0x7f6da4022020 con 0x7f6db007b700 2024-03-04T15:39:49.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.927+0000 7f6da27fc700 1 --2- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f6d9c038bf0 0x7f6d9c03b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:49.316 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.927+0000 7f6da27fc700 1 -- 172.21.15.117:0/3793030402 --> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6d9c03b7c0 con 0x7f6d9c038bf0 2024-03-04T15:39:49.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.927+0000 7f6da27fc700 1 -- 172.21.15.117:0/3793030402 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f6da404cbb0 con 0x7f6db007b700 2024-03-04T15:39:49.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.927+0000 7f6db4ec4700 1 -- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f6d9c038bf0 msgr2=0x7f6d9c03b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/427955642 2024-03-04T15:39:49.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:42.927+0000 7f6db4ec4700 1 --2- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f6d9c038bf0 0x7f6d9c03b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T15:39:49.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:43.131+0000 7f6db4ec4700 1 -- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f6d9c038bf0 msgr2=0x7f6d9c03b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/427955642 2024-03-04T15:39:49.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:43.131+0000 7f6db4ec4700 1 --2- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f6d9c038bf0 0x7f6d9c03b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T15:39:49.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:43.535+0000 7f6db4ec4700 1 -- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f6d9c038bf0 msgr2=0x7f6d9c03b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/427955642 2024-03-04T15:39:49.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:43.535+0000 7f6db4ec4700 1 --2- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f6d9c038bf0 0x7f6d9c03b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T15:39:49.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:44.339+0000 7f6db4ec4700 1 -- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f6d9c038bf0 msgr2=0x7f6d9c03b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/427955642 2024-03-04T15:39:49.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:44.339+0000 7f6db4ec4700 1 --2- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f6d9c038bf0 0x7f6d9c03b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T15:39:49.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:45.943+0000 7f6db4ec4700 1 -- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f6d9c038bf0 msgr2=0x7f6d9c03b0b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/427955642 2024-03-04T15:39:49.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:45.943+0000 7f6db4ec4700 1 --2- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f6d9c038bf0 0x7f6d9c03b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-04T15:39:49.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:47.647+0000 7f6da27fc700 1 -- 172.21.15.117:0/3793030402 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mgrmap(e 6) v1 ==== 45694+0+0 (secure 0 0 0) 0x7f6da402d3f0 con 0x7f6db007b700 2024-03-04T15:39:49.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:47.647+0000 7f6da27fc700 1 -- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f6d9c038bf0 msgr2=0x7f6d9c03b0b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T15:39:49.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:47.647+0000 7f6da27fc700 1 --2- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f6d9c038bf0 0x7f6d9c03b0b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:49.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:48.647+0000 7f6da27fc700 1 -- 172.21.15.117:0/3793030402 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mgrmap(e 7) v1 ==== 45821+0+0 (secure 0 0 0) 0x7f6da404e270 con 0x7f6db007b700 2024-03-04T15:39:49.317 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:48.647+0000 7f6da27fc700 1 --2- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f6d9c03c520 0x7f6d9c03e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:49.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:48.647+0000 7f6da27fc700 1 -- 172.21.15.117:0/3793030402 --> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6da4001d30 con 0x7f6d9c03c520 2024-03-04T15:39:49.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:48.647+0000 7f6db4ec4700 1 --2- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f6d9c03c520 0x7f6d9c03e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:49.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:48.651+0000 7f6db4ec4700 1 --2- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f6d9c03c520 0x7f6d9c03e910 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6dac003a10 tx=0x7f6dac0092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:49.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:48.651+0000 7f6da27fc700 1 -- 172.21.15.117:0/3793030402 <== mgr.14120 v2:172.21.15.117:6800/1454790014 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f6da4001d30 con 0x7f6d9c03c520 2024-03-04T15:39:49.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:48.663+0000 7f6db7929700 1 -- 172.21.15.117:0/3793030402 --> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f6db0063410 con 0x7f6d9c03c520 2024-03-04T15:39:49.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:48.663+0000 7f6da27fc700 1 -- 172.21.15.117:0/3793030402 <== mgr.14120 v2:172.21.15.117:6800/1454790014 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f6db0063410 con 0x7f6d9c03c520 2024-03-04T15:39:49.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:48.663+0000 7f6db7929700 1 -- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f6d9c03c520 msgr2=0x7f6d9c03e910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:49.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:48.663+0000 7f6db7929700 1 --2- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f6d9c03c520 0x7f6d9c03e910 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6dac003a10 tx=0x7f6dac0092b0 comp rx=0 tx=0).stop 2024-03-04T15:39:49.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:48.663+0000 7f6db7929700 1 -- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db007b700 msgr2=0x7f6db019bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:49.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:48.663+0000 7f6db7929700 1 --2- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db007b700 0x7f6db019bbc0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6da4003e40 tx=0x7f6da40119c0 comp rx=0 tx=0).stop 2024-03-04T15:39:49.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:48.663+0000 7f6db7929700 1 -- 172.21.15.117:0/3793030402 shutdown_connections 2024-03-04T15:39:49.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:48.663+0000 7f6db7929700 1 --2- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f6d9c03c520 0x7f6d9c03e910 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:49.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:48.663+0000 7f6db7929700 1 --2- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6db007b700 0x7f6db019bbc0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:49.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:48.663+0000 7f6db7929700 1 --2- 172.21.15.117:0/3793030402 >> [v2:172.21.15.117:6800/427955642,v1:172.21.15.117:6801/427955642] conn(0x7f6d9c038bf0 0x7f6d9c03b0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:49.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:48.663+0000 7f6db7929700 1 -- 172.21.15.117:0/3793030402 >> 172.21.15.117:0/3793030402 conn(0x7f6db0101220 msgr2=0x7f6db0101d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:49.318 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:48.663+0000 7f6db7929700 1 -- 172.21.15.117:0/3793030402 shutdown_connections 2024-03-04T15:39:49.319 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:48.663+0000 7f6db7929700 1 -- 172.21.15.117:0/3793030402 wait complete. 2024-03-04T15:39:49.319 INFO:teuthology.orchestra.run.smithi117.stdout:mgr epoch 5 is available 2024-03-04T15:39:49.319 INFO:teuthology.orchestra.run.smithi117.stdout:Setting orchestrator backend to cephadm... 2024-03-04T15:39:49.982 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:49 smithi117 bash[21480]: audit 2024-03-04T15:39:48.654647+0000 mgr.y (mgr.14120) 2 : audit 0 from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-04T15:39:49.982 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:49 smithi117 bash[21480]: audit 2024-03-04T15:39:48.667967+0000 mgr.y (mgr.14120) 3 : audit 0 from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-04T15:39:49.982 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:49 smithi117 bash[21480]: audit 2024-03-04T15:39:48.911502+0000 mon.a (mon.0) 49 : audit 1 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' 2024-03-04T15:39:49.982 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:49 smithi117 bash[21480]: audit 2024-03-04T15:39:48.917294+0000 mon.a (mon.0) 50 : audit 1 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' 2024-03-04T15:39:51.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:50 smithi117 bash[21480]: cephadm 2024-03-04T15:39:49.582062+0000 mgr.y (mgr.14120) 4 : cephadm 1 [04/Mar/2024:15:39:49] ENGINE Bus STARTING 2024-03-04T15:39:51.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:50 smithi117 bash[21480]: cephadm 2024-03-04T15:39:49.684230+0000 mgr.y (mgr.14120) 5 : cephadm 1 [04/Mar/2024:15:39:49] ENGINE Serving on http://172.21.15.117:8765 2024-03-04T15:39:51.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:50 smithi117 bash[21480]: audit 2024-03-04T15:39:49.842716+0000 mon.a (mon.0) 51 : audit 0 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:39:51.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:50 smithi117 bash[21480]: cluster 2024-03-04T15:39:49.922200+0000 mon.a (mon.0) 52 : cluster 0 mgrmap e8: y(active, since 2s) 2024-03-04T15:39:51.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:50 smithi117 bash[21480]: audit 2024-03-04T15:39:50.561882+0000 mon.a (mon.0) 53 : audit 1 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' 2024-03-04T15:39:51.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:50 smithi117 bash[21480]: audit 2024-03-04T15:39:50.611224+0000 mon.a (mon.0) 54 : audit 0 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:39:51.185 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e358c700 1 Processor -- start 2024-03-04T15:39:51.185 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e358c700 1 -- start start 2024-03-04T15:39:51.185 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e358c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77dc107e70 0x7f77dc108290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:51.185 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e358c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc1087d0 con 0x7f77dc107e70 2024-03-04T15:39:51.185 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e1328700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77dc107e70 0x7f77dc108290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:51.185 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e1328700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77dc107e70 0x7f77dc108290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38398/0 (socket says 172.21.15.117:38398) 2024-03-04T15:39:51.185 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e1328700 1 -- 172.21.15.117:0/913354656 learned_addr learned my addr 172.21.15.117:0/913354656 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:51.185 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e1328700 1 -- 172.21.15.117:0/913354656 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc108fd0 con 0x7f77dc107e70 2024-03-04T15:39:51.185 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e1328700 1 --2- 172.21.15.117:0/913354656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77dc107e70 0x7f77dc108290 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f77d80098a0 tx=0x7f77d8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4800eee34ab22c8 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:51.185 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77cffff700 1 -- 172.21.15.117:0/913354656 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77d8004160 con 0x7f77dc107e70 2024-03-04T15:39:51.185 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77cffff700 1 -- 172.21.15.117:0/913354656 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f77d8004300 con 0x7f77dc107e70 2024-03-04T15:39:51.185 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77cffff700 1 -- 172.21.15.117:0/913354656 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77d8011940 con 0x7f77dc107e70 2024-03-04T15:39:51.185 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e358c700 1 -- 172.21.15.117:0/913354656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77dc107e70 msgr2=0x7f77dc108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:51.185 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e358c700 1 --2- 172.21.15.117:0/913354656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77dc107e70 0x7f77dc108290 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f77d80098a0 tx=0x7f77d8009bb0 comp rx=0 tx=0).stop 2024-03-04T15:39:51.186 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e358c700 1 -- 172.21.15.117:0/913354656 shutdown_connections 2024-03-04T15:39:51.186 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e358c700 1 --2- 172.21.15.117:0/913354656 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77dc107e70 0x7f77dc108290 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:51.186 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e358c700 1 -- 172.21.15.117:0/913354656 >> 172.21.15.117:0/913354656 conn(0x7f77dc07b220 msgr2=0x7f77dc105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:51.186 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e358c700 1 -- 172.21.15.117:0/913354656 shutdown_connections 2024-03-04T15:39:51.186 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e358c700 1 -- 172.21.15.117:0/913354656 wait complete. 2024-03-04T15:39:51.186 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e358c700 1 Processor -- start 2024-03-04T15:39:51.186 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e358c700 1 -- start start 2024-03-04T15:39:51.186 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e358c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77dc107e70 0x7f77dc19bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:51.186 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e358c700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77dc109cd0 con 0x7f77dc107e70 2024-03-04T15:39:51.186 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e1328700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77dc107e70 0x7f77dc19bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:51.186 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e1328700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77dc107e70 0x7f77dc19bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38402/0 (socket says 172.21.15.117:38402) 2024-03-04T15:39:51.186 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e1328700 1 -- 172.21.15.117:0/3321597203 learned_addr learned my addr 172.21.15.117:0/3321597203 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:51.186 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.159+0000 7f77e1328700 1 -- 172.21.15.117:0/3321597203 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77dc19c100 con 0x7f77dc107e70 2024-03-04T15:39:51.186 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.163+0000 7f77e1328700 1 --2- 172.21.15.117:0/3321597203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77dc107e70 0x7f77dc19bbc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009870 tx=0x7f77d80119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:51.187 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.163+0000 7f77ce7fc700 1 -- 172.21.15.117:0/3321597203 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77d801a430 con 0x7f77dc107e70 2024-03-04T15:39:51.187 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.163+0000 7f77ce7fc700 1 -- 172.21.15.117:0/3321597203 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f77d801aa30 con 0x7f77dc107e70 2024-03-04T15:39:51.187 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.163+0000 7f77ce7fc700 1 -- 172.21.15.117:0/3321597203 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f77d80248d0 con 0x7f77dc107e70 2024-03-04T15:39:51.187 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.163+0000 7f77e358c700 1 -- 172.21.15.117:0/3321597203 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77dc19c390 con 0x7f77dc107e70 2024-03-04T15:39:51.187 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.163+0000 7f77e358c700 1 -- 172.21.15.117:0/3321597203 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77dc19f080 con 0x7f77dc107e70 2024-03-04T15:39:51.187 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.163+0000 7f77ce7fc700 1 -- 172.21.15.117:0/3321597203 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45927+0+0 (secure 0 0 0) 0x7f77d8022020 con 0x7f77dc107e70 2024-03-04T15:39:51.187 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.163+0000 7f77ce7fc700 1 --2- 172.21.15.117:0/3321597203 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f77c8038ba0 0x7f77c803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:51.187 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.163+0000 7f77ce7fc700 1 -- 172.21.15.117:0/3321597203 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f77d804cf10 con 0x7f77dc107e70 2024-03-04T15:39:51.187 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.163+0000 7f77e358c700 1 -- 172.21.15.117:0/3321597203 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f77dc10c290 con 0x7f77dc107e70 2024-03-04T15:39:51.187 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.163+0000 7f77e0b27700 1 --2- 172.21.15.117:0/3321597203 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f77c8038ba0 0x7f77c803b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:51.187 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.163+0000 7f77e0b27700 1 --2- 172.21.15.117:0/3321597203 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f77c8038ba0 0x7f77c803b060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77d0006fd0 tx=0x7f77d0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:51.187 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.167+0000 7f77ce7fc700 1 -- 172.21.15.117:0/3321597203 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f77d8019c70 con 0x7f77dc107e70 2024-03-04T15:39:51.187 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.551+0000 7f77e358c700 1 -- 172.21.15.117:0/3321597203 --> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f77dc105b70 con 0x7f77c8038ba0 2024-03-04T15:39:51.187 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.559+0000 7f77ce7fc700 1 -- 172.21.15.117:0/3321597203 <== mgr.14120 v2:172.21.15.117:6800/1454790014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f77dc105b70 con 0x7f77c8038ba0 2024-03-04T15:39:51.187 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.563+0000 7f77e358c700 1 -- 172.21.15.117:0/3321597203 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f77c8038ba0 msgr2=0x7f77c803b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:51.187 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.563+0000 7f77e358c700 1 --2- 172.21.15.117:0/3321597203 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f77c8038ba0 0x7f77c803b060 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f77d0006fd0 tx=0x7f77d0006e40 comp rx=0 tx=0).stop 2024-03-04T15:39:51.188 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.563+0000 7f77e358c700 1 -- 172.21.15.117:0/3321597203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77dc107e70 msgr2=0x7f77dc19bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:51.188 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.563+0000 7f77e358c700 1 --2- 172.21.15.117:0/3321597203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77dc107e70 0x7f77dc19bbc0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f77d8009870 tx=0x7f77d80119c0 comp rx=0 tx=0).stop 2024-03-04T15:39:51.188 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.563+0000 7f77e358c700 1 -- 172.21.15.117:0/3321597203 shutdown_connections 2024-03-04T15:39:51.188 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.563+0000 7f77e358c700 1 --2- 172.21.15.117:0/3321597203 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f77c8038ba0 0x7f77c803b060 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:51.188 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.563+0000 7f77e358c700 1 --2- 172.21.15.117:0/3321597203 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f77dc107e70 0x7f77dc19bbc0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:51.188 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.563+0000 7f77e358c700 1 -- 172.21.15.117:0/3321597203 >> 172.21.15.117:0/3321597203 conn(0x7f77dc07b220 msgr2=0x7f77dc105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:51.188 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.563+0000 7f77e358c700 1 -- 172.21.15.117:0/3321597203 shutdown_connections 2024-03-04T15:39:51.188 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:50.563+0000 7f77e358c700 1 -- 172.21.15.117:0/3321597203 wait complete. 2024-03-04T15:39:52.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:51 smithi117 bash[21480]: cephadm 2024-03-04T15:39:49.802356+0000 mgr.y (mgr.14120) 6 : cephadm 1 [04/Mar/2024:15:39:49] ENGINE Serving on https://172.21.15.117:7150 2024-03-04T15:39:52.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:51 smithi117 bash[21480]: cephadm 2024-03-04T15:39:49.802560+0000 mgr.y (mgr.14120) 7 : cephadm 1 [04/Mar/2024:15:39:49] ENGINE Bus STARTED 2024-03-04T15:39:52.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:51 smithi117 bash[21480]: audit 2024-03-04T15:39:50.555338+0000 mgr.y (mgr.14120) 8 : audit 0 from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:39:52.941 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout value unchanged 2024-03-04T15:39:52.941 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.023+0000 7f267ebb4700 1 Processor -- start 2024-03-04T15:39:52.941 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.023+0000 7f267ebb4700 1 -- start start 2024-03-04T15:39:52.941 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.023+0000 7f267ebb4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2678107e70 0x7f2678108290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:52.941 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.023+0000 7f267ebb4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26781087d0 con 0x7f2678107e70 2024-03-04T15:39:52.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.023+0000 7f267c950700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2678107e70 0x7f2678108290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:52.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.023+0000 7f267c950700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2678107e70 0x7f2678108290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38404/0 (socket says 172.21.15.117:38404) 2024-03-04T15:39:52.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.023+0000 7f267c950700 1 -- 172.21.15.117:0/2773897382 learned_addr learned my addr 172.21.15.117:0/2773897382 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:52.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.023+0000 7f267c950700 1 -- 172.21.15.117:0/2773897382 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2678108fd0 con 0x7f2678107e70 2024-03-04T15:39:52.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.023+0000 7f267c950700 1 --2- 172.21.15.117:0/2773897382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2678107e70 0x7f2678108290 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f26680098a0 tx=0x7f2668009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf91d0741dd502ef server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:52.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.023+0000 7f26777fe700 1 -- 172.21.15.117:0/2773897382 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2668004160 con 0x7f2678107e70 2024-03-04T15:39:52.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.023+0000 7f26777fe700 1 -- 172.21.15.117:0/2773897382 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f2668004300 con 0x7f2678107e70 2024-03-04T15:39:52.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.023+0000 7f26777fe700 1 -- 172.21.15.117:0/2773897382 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2668011940 con 0x7f2678107e70 2024-03-04T15:39:52.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.023+0000 7f267ebb4700 1 -- 172.21.15.117:0/2773897382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2678107e70 msgr2=0x7f2678108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:52.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.023+0000 7f267ebb4700 1 --2- 172.21.15.117:0/2773897382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2678107e70 0x7f2678108290 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f26680098a0 tx=0x7f2668009bb0 comp rx=0 tx=0).stop 2024-03-04T15:39:52.942 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.023+0000 7f267ebb4700 1 -- 172.21.15.117:0/2773897382 shutdown_connections 2024-03-04T15:39:52.943 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.023+0000 7f267ebb4700 1 --2- 172.21.15.117:0/2773897382 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2678107e70 0x7f2678108290 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:52.943 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.023+0000 7f267ebb4700 1 -- 172.21.15.117:0/2773897382 >> 172.21.15.117:0/2773897382 conn(0x7f26781034b0 msgr2=0x7f26781058a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:52.943 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.023+0000 7f267ebb4700 1 -- 172.21.15.117:0/2773897382 shutdown_connections 2024-03-04T15:39:52.943 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.023+0000 7f267ebb4700 1 -- 172.21.15.117:0/2773897382 wait complete. 2024-03-04T15:39:52.943 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.027+0000 7f267ebb4700 1 Processor -- start 2024-03-04T15:39:52.943 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.027+0000 7f267ebb4700 1 -- start start 2024-03-04T15:39:52.943 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.027+0000 7f267ebb4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2678107e70 0x7f267819bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:52.943 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.027+0000 7f267ebb4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2678109cd0 con 0x7f2678107e70 2024-03-04T15:39:52.943 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.027+0000 7f267c950700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2678107e70 0x7f267819bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:52.943 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.027+0000 7f267c950700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2678107e70 0x7f267819bc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38408/0 (socket says 172.21.15.117:38408) 2024-03-04T15:39:52.944 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.027+0000 7f267c950700 1 -- 172.21.15.117:0/3607303965 learned_addr learned my addr 172.21.15.117:0/3607303965 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:52.944 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.027+0000 7f267c950700 1 -- 172.21.15.117:0/3607303965 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f267819c160 con 0x7f2678107e70 2024-03-04T15:39:52.944 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.027+0000 7f267c950700 1 --2- 172.21.15.117:0/3607303965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2678107e70 0x7f267819bc20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2668009290 tx=0x7f26680119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:52.944 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.027+0000 7f2675ffb700 1 -- 172.21.15.117:0/3607303965 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2668003930 con 0x7f2678107e70 2024-03-04T15:39:52.944 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.027+0000 7f2675ffb700 1 -- 172.21.15.117:0/3607303965 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f2668003ad0 con 0x7f2678107e70 2024-03-04T15:39:52.944 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.027+0000 7f267ebb4700 1 -- 172.21.15.117:0/3607303965 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f267819c3f0 con 0x7f2678107e70 2024-03-04T15:39:52.944 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.027+0000 7f2675ffb700 1 -- 172.21.15.117:0/3607303965 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2668019720 con 0x7f2678107e70 2024-03-04T15:39:52.944 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.027+0000 7f267ebb4700 1 -- 172.21.15.117:0/3607303965 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f267819f0e0 con 0x7f2678107e70 2024-03-04T15:39:52.944 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.027+0000 7f2675ffb700 1 -- 172.21.15.117:0/3607303965 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45927+0+0 (secure 0 0 0) 0x7f2668019a20 con 0x7f2678107e70 2024-03-04T15:39:52.944 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.027+0000 7f267ebb4700 1 -- 172.21.15.117:0/3607303965 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f267810c290 con 0x7f2678107e70 2024-03-04T15:39:52.944 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.027+0000 7f2675ffb700 1 --2- 172.21.15.117:0/3607303965 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f2664038b50 0x7f266403b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:52.944 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.027+0000 7f2675ffb700 1 -- 172.21.15.117:0/3607303965 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f266804d490 con 0x7f2678107e70 2024-03-04T15:39:52.944 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.027+0000 7f2677fff700 1 --2- 172.21.15.117:0/3607303965 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f2664038b50 0x7f266403b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:52.945 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.027+0000 7f2677fff700 1 --2- 172.21.15.117:0/3607303965 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f2664038b50 0x7f266403b010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2660006fd0 tx=0x7f2660006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:52.945 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.035+0000 7f2675ffb700 1 -- 172.21.15.117:0/3607303965 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f266804cde0 con 0x7f2678107e70 2024-03-04T15:39:52.945 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.387+0000 7f267ebb4700 1 -- 172.21.15.117:0/3607303965 --> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f26781046d0 con 0x7f2664038b50 2024-03-04T15:39:52.945 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.387+0000 7f2675ffb700 1 -- 172.21.15.117:0/3607303965 <== mgr.14120 v2:172.21.15.117:6800/1454790014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f26781046d0 con 0x7f2664038b50 2024-03-04T15:39:52.945 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.391+0000 7f267ebb4700 1 -- 172.21.15.117:0/3607303965 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f2664038b50 msgr2=0x7f266403b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:52.945 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.391+0000 7f267ebb4700 1 --2- 172.21.15.117:0/3607303965 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f2664038b50 0x7f266403b010 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f2660006fd0 tx=0x7f2660006e40 comp rx=0 tx=0).stop 2024-03-04T15:39:52.945 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.391+0000 7f267ebb4700 1 -- 172.21.15.117:0/3607303965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2678107e70 msgr2=0x7f267819bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:52.945 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.391+0000 7f267ebb4700 1 --2- 172.21.15.117:0/3607303965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2678107e70 0x7f267819bc20 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2668009290 tx=0x7f26680119c0 comp rx=0 tx=0).stop 2024-03-04T15:39:52.945 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.391+0000 7f267ebb4700 1 -- 172.21.15.117:0/3607303965 shutdown_connections 2024-03-04T15:39:52.945 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.391+0000 7f267ebb4700 1 --2- 172.21.15.117:0/3607303965 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f2664038b50 0x7f266403b010 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:52.945 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.391+0000 7f267ebb4700 1 --2- 172.21.15.117:0/3607303965 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2678107e70 0x7f267819bc20 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:52.945 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.391+0000 7f267ebb4700 1 -- 172.21.15.117:0/3607303965 >> 172.21.15.117:0/3607303965 conn(0x7f26781034b0 msgr2=0x7f2678103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:52.945 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.391+0000 7f267ebb4700 1 -- 172.21.15.117:0/3607303965 shutdown_connections 2024-03-04T15:39:52.945 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:52.391+0000 7f267ebb4700 1 -- 172.21.15.117:0/3607303965 wait complete. 2024-03-04T15:39:52.946 INFO:teuthology.orchestra.run.smithi117.stdout:Generating ssh key... 2024-03-04T15:39:53.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:52 smithi117 bash[21480]: audit 2024-03-04T15:39:52.391546+0000 mgr.y (mgr.14120) 9 : audit 0 from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:39:55.039 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:54 smithi117 bash[21480]: audit 2024-03-04T15:39:54.225747+0000 mgr.y (mgr.14120) 10 : audit 0 from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:39:55.039 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:54 smithi117 bash[21480]: cephadm 2024-03-04T15:39:54.225961+0000 mgr.y (mgr.14120) 11 : cephadm 1 Generating ssh key... 2024-03-04T15:39:55.039 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:55 smithi117 bash[21734]: Generating public/private rsa key pair. 2024-03-04T15:39:55.039 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:55 smithi117 bash[21734]: Your identification has been saved in /tmp/tmpgbsqdhxp/key. 2024-03-04T15:39:55.039 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:55 smithi117 bash[21734]: Your public key has been saved in /tmp/tmpgbsqdhxp/key.pub. 2024-03-04T15:39:55.039 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:55 smithi117 bash[21734]: The key fingerprint is: 2024-03-04T15:39:55.039 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:55 smithi117 bash[21734]: SHA256:PthZJ/g6wiWLcAthSCLc6olsRqelZWC4SDO+j926/50 ceph-e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T15:39:55.039 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:55 smithi117 bash[21734]: The key's randomart image is: 2024-03-04T15:39:55.040 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:55 smithi117 bash[21734]: +---[RSA 3072]----+ 2024-03-04T15:39:55.040 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:55 smithi117 bash[21734]: |o . | 2024-03-04T15:39:55.040 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:55 smithi117 bash[21734]: |+O . | 2024-03-04T15:39:55.040 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:55 smithi117 bash[21734]: |Oo= | 2024-03-04T15:39:55.040 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:55 smithi117 bash[21734]: |++o= . | 2024-03-04T15:39:55.040 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:55 smithi117 bash[21734]: |=.X. S o . | 2024-03-04T15:39:55.040 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:55 smithi117 bash[21734]: |.Xo . .+.+ o | 2024-03-04T15:39:55.040 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:55 smithi117 bash[21734]: |o ++.+.+= . | 2024-03-04T15:39:55.040 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:55 smithi117 bash[21734]: | . oo.+ oo. | 2024-03-04T15:39:55.040 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:55 smithi117 bash[21734]: | o+..o.E | 2024-03-04T15:39:55.040 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:39:55 smithi117 bash[21734]: +----[SHA256]-----+ 2024-03-04T15:39:55.653 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc77dfe4700 1 Processor -- start 2024-03-04T15:39:55.653 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc77dfe4700 1 -- start start 2024-03-04T15:39:55.653 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc77dfe4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc778105c60 0x7fc778106080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:55.653 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc77dfe4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc778106650 con 0x7fc778105c60 2024-03-04T15:39:55.653 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc7777fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc778105c60 0x7fc778106080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:55.653 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc7777fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc778105c60 0x7fc778106080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38420/0 (socket says 172.21.15.117:38420) 2024-03-04T15:39:55.654 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc7777fe700 1 -- 172.21.15.117:0/2714576619 learned_addr learned my addr 172.21.15.117:0/2714576619 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:55.654 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc7777fe700 1 -- 172.21.15.117:0/2714576619 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc778106e60 con 0x7fc778105c60 2024-03-04T15:39:55.654 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc7777fe700 1 --2- 172.21.15.117:0/2714576619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc778105c60 0x7fc778106080 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc7700098a0 tx=0x7fc770009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1556ca3f94c167f server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:55.654 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc7767fc700 1 -- 172.21.15.117:0/2714576619 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc770004160 con 0x7fc778105c60 2024-03-04T15:39:55.654 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc7767fc700 1 -- 172.21.15.117:0/2714576619 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fc770004300 con 0x7fc778105c60 2024-03-04T15:39:55.654 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc7767fc700 1 -- 172.21.15.117:0/2714576619 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc770011940 con 0x7fc778105c60 2024-03-04T15:39:55.654 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc77dfe4700 1 -- 172.21.15.117:0/2714576619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc778105c60 msgr2=0x7fc778106080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:55.654 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc77dfe4700 1 --2- 172.21.15.117:0/2714576619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc778105c60 0x7fc778106080 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fc7700098a0 tx=0x7fc770009bb0 comp rx=0 tx=0).stop 2024-03-04T15:39:55.654 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc77dfe4700 1 -- 172.21.15.117:0/2714576619 shutdown_connections 2024-03-04T15:39:55.654 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc77dfe4700 1 --2- 172.21.15.117:0/2714576619 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc778105c60 0x7fc778106080 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:55.654 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc77dfe4700 1 -- 172.21.15.117:0/2714576619 >> 172.21.15.117:0/2714576619 conn(0x7fc778101220 msgr2=0x7fc778103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:55.654 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc77dfe4700 1 -- 172.21.15.117:0/2714576619 shutdown_connections 2024-03-04T15:39:55.654 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc77dfe4700 1 -- 172.21.15.117:0/2714576619 wait complete. 2024-03-04T15:39:55.654 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc77dfe4700 1 Processor -- start 2024-03-04T15:39:55.654 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc77dfe4700 1 -- start start 2024-03-04T15:39:55.654 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc77dfe4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc778105c60 0x7fc77807b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:55.655 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc77dfe4700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc778107cf0 con 0x7fc778105c60 2024-03-04T15:39:55.655 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc7777fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc778105c60 0x7fc77807b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:55.655 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc7777fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc778105c60 0x7fc77807b3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38434/0 (socket says 172.21.15.117:38434) 2024-03-04T15:39:55.655 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc7777fe700 1 -- 172.21.15.117:0/2816184962 learned_addr learned my addr 172.21.15.117:0/2816184962 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:55.655 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc7777fe700 1 -- 172.21.15.117:0/2816184962 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc77807b930 con 0x7fc778105c60 2024-03-04T15:39:55.655 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc7777fe700 1 --2- 172.21.15.117:0/2816184962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc778105c60 0x7fc77807b3f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc770011910 tx=0x7fc7700119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:55.655 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc774ff9700 1 -- 172.21.15.117:0/2816184962 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc770003b20 con 0x7fc778105c60 2024-03-04T15:39:55.655 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc774ff9700 1 -- 172.21.15.117:0/2816184962 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7fc77001a890 con 0x7fc778105c60 2024-03-04T15:39:55.655 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc77dfe4700 1 -- 172.21.15.117:0/2816184962 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc778079b20 con 0x7fc778105c60 2024-03-04T15:39:55.655 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc774ff9700 1 -- 172.21.15.117:0/2816184962 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc7700197f0 con 0x7fc778105c60 2024-03-04T15:39:55.655 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.863+0000 7fc77dfe4700 1 -- 172.21.15.117:0/2816184962 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc77807a060 con 0x7fc778105c60 2024-03-04T15:39:55.655 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.867+0000 7fc774ff9700 1 -- 172.21.15.117:0/2816184962 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45927+0+0 (secure 0 0 0) 0x7fc770022020 con 0x7fc778105c60 2024-03-04T15:39:55.655 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.867+0000 7fc774ff9700 1 --2- 172.21.15.117:0/2816184962 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7fc764038ba0 0x7fc76403b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:55.655 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.867+0000 7fc774ff9700 1 -- 172.21.15.117:0/2816184962 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fc77004ce80 con 0x7fc778105c60 2024-03-04T15:39:55.655 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.867+0000 7fc776ffd700 1 --2- 172.21.15.117:0/2816184962 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7fc764038ba0 0x7fc76403b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:55.655 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.867+0000 7fc77dfe4700 1 -- 172.21.15.117:0/2816184962 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc778063410 con 0x7fc778105c60 2024-03-04T15:39:55.655 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.867+0000 7fc776ffd700 1 --2- 172.21.15.117:0/2816184962 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7fc764038ba0 0x7fc76403b060 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc76000ad30 tx=0x7fc7600093f0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:55.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:53.871+0000 7fc774ff9700 1 -- 172.21.15.117:0/2816184962 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc77004cbe0 con 0x7fc778105c60 2024-03-04T15:39:55.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:54.223+0000 7fc77dfe4700 1 -- 172.21.15.117:0/2816184962 --> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7fc77807a530 con 0x7fc764038ba0 2024-03-04T15:39:55.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:55.019+0000 7fc774ff9700 1 -- 172.21.15.117:0/2816184962 <== mgr.14120 v2:172.21.15.117:6800/1454790014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fc77807a530 con 0x7fc764038ba0 2024-03-04T15:39:55.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:55.023+0000 7fc77dfe4700 1 -- 172.21.15.117:0/2816184962 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7fc764038ba0 msgr2=0x7fc76403b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:55.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:55.023+0000 7fc77dfe4700 1 --2- 172.21.15.117:0/2816184962 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7fc764038ba0 0x7fc76403b060 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc76000ad30 tx=0x7fc7600093f0 comp rx=0 tx=0).stop 2024-03-04T15:39:55.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:55.023+0000 7fc77dfe4700 1 -- 172.21.15.117:0/2816184962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc778105c60 msgr2=0x7fc77807b3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:55.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:55.023+0000 7fc77dfe4700 1 --2- 172.21.15.117:0/2816184962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc778105c60 0x7fc77807b3f0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fc770011910 tx=0x7fc7700119f0 comp rx=0 tx=0).stop 2024-03-04T15:39:55.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:55.023+0000 7fc77dfe4700 1 -- 172.21.15.117:0/2816184962 shutdown_connections 2024-03-04T15:39:55.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:55.023+0000 7fc77dfe4700 1 --2- 172.21.15.117:0/2816184962 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7fc764038ba0 0x7fc76403b060 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:55.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:55.023+0000 7fc77dfe4700 1 --2- 172.21.15.117:0/2816184962 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc778105c60 0x7fc77807b3f0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:55.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:55.023+0000 7fc77dfe4700 1 -- 172.21.15.117:0/2816184962 >> 172.21.15.117:0/2816184962 conn(0x7fc778101220 msgr2=0x7fc778101e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:55.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:55.023+0000 7fc77dfe4700 1 -- 172.21.15.117:0/2816184962 shutdown_connections 2024-03-04T15:39:55.656 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:55.023+0000 7fc77dfe4700 1 -- 172.21.15.117:0/2816184962 wait complete. 2024-03-04T15:39:56.300 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:56 smithi117 bash[21480]: audit 2024-03-04T15:39:55.017264+0000 mon.a (mon.0) 55 : audit 1 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' 2024-03-04T15:39:56.300 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:56 smithi117 bash[21480]: audit 2024-03-04T15:39:55.022511+0000 mon.a (mon.0) 56 : audit 1 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' 2024-03-04T15:39:57.492 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCopvF0TLtYEXloWdTRgEm4DhVivvyJ9bLfq/J1+mogDpYEXcGWJFOzuzwukvJbgN+cZpGs7o/ysA1AEPO6OmWRM91bwy/q7IMtr4TmhNJiYA+ZqVzg6krnmjWS7lwNsDwpdAcW2GRI+pHPBzTuiLk9x/X7bkRcZnkP6xzJr4g1zqDcgscwUPezy6g1q+AZPVOiiz4I7nl/y3X+btops+NpyukVl4UgwcLU4oWHc0zTyKQ30QPwubyEc6tfYSEQkwn1oTQAhgnZSd8yqKhcekyEtI+hJYjFlyuGQEfh5+wiZTINPQFJMSvbquADnt3NVgrJft7vDMcLrNcBSkhjVwso4ArO+lPqlO+TIfJLFLzRM6GU9pdVXj1YQ4SsGUCbJ0XMSNGklD7oIK9CCYCmQ4RVlX2ZJ2Zqwij1ufoaXSh29UcN42dJeYoJhFrGO/nDw5Ha/xLpxBkfaYkHTZCsiX8zm492nG4JUZniKr/93aLz1FY5TqYDugl0l8j2mVBmjF8= ceph-e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T15:39:57.492 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.499+0000 7f8a5509b700 1 Processor -- start 2024-03-04T15:39:57.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a5509b700 1 -- start start 2024-03-04T15:39:57.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a5509b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a50105c70 0x7f8a50106090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:57.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a5509b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a50106660 con 0x7f8a50105c70 2024-03-04T15:39:57.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a4ed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a50105c70 0x7f8a50106090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:57.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a4ed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a50105c70 0x7f8a50106090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38436/0 (socket says 172.21.15.117:38436) 2024-03-04T15:39:57.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a4ed9d700 1 -- 172.21.15.117:0/3060875920 learned_addr learned my addr 172.21.15.117:0/3060875920 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:57.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a4ed9d700 1 -- 172.21.15.117:0/3060875920 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a50106e70 con 0x7f8a50105c70 2024-03-04T15:39:57.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a4ed9d700 1 --2- 172.21.15.117:0/3060875920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a50105c70 0x7f8a50106090 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8a440098a0 tx=0x7f8a44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1cca4f8fdbe944 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:57.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a4dd9b700 1 -- 172.21.15.117:0/3060875920 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a44004160 con 0x7f8a50105c70 2024-03-04T15:39:57.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a4dd9b700 1 -- 172.21.15.117:0/3060875920 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f8a44004300 con 0x7f8a50105c70 2024-03-04T15:39:57.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a4dd9b700 1 -- 172.21.15.117:0/3060875920 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a44011940 con 0x7f8a50105c70 2024-03-04T15:39:57.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a5509b700 1 -- 172.21.15.117:0/3060875920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a50105c70 msgr2=0x7f8a50106090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:57.493 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a5509b700 1 --2- 172.21.15.117:0/3060875920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a50105c70 0x7f8a50106090 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f8a440098a0 tx=0x7f8a44009bb0 comp rx=0 tx=0).stop 2024-03-04T15:39:57.494 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a5509b700 1 -- 172.21.15.117:0/3060875920 shutdown_connections 2024-03-04T15:39:57.494 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a5509b700 1 --2- 172.21.15.117:0/3060875920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a50105c70 0x7f8a50106090 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:57.494 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a5509b700 1 -- 172.21.15.117:0/3060875920 >> 172.21.15.117:0/3060875920 conn(0x7f8a501011f0 msgr2=0x7f8a50103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:57.494 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a5509b700 1 -- 172.21.15.117:0/3060875920 shutdown_connections 2024-03-04T15:39:57.494 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a5509b700 1 -- 172.21.15.117:0/3060875920 wait complete. 2024-03-04T15:39:57.494 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a5509b700 1 Processor -- start 2024-03-04T15:39:57.494 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a5509b700 1 -- start start 2024-03-04T15:39:57.494 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a5509b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a50105c70 0x7f8a50199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:57.494 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a5509b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a50107d00 con 0x7f8a50105c70 2024-03-04T15:39:57.494 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a4ed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a50105c70 0x7f8a50199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:57.494 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a4ed9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a50105c70 0x7f8a50199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38452/0 (socket says 172.21.15.117:38452) 2024-03-04T15:39:57.494 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a4ed9d700 1 -- 172.21.15.117:0/851560847 learned_addr learned my addr 172.21.15.117:0/851560847 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:39:57.494 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a4ed9d700 1 -- 172.21.15.117:0/851560847 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a50199f60 con 0x7f8a50105c70 2024-03-04T15:39:57.494 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.503+0000 7f8a4ed9d700 1 --2- 172.21.15.117:0/851560847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a50105c70 0x7f8a50199a20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8a44003e40 tx=0x7f8a440119c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:57.495 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.507+0000 7f8a37fff700 1 -- 172.21.15.117:0/851560847 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a4401a430 con 0x7f8a50105c70 2024-03-04T15:39:57.495 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.507+0000 7f8a37fff700 1 -- 172.21.15.117:0/851560847 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f8a4401aa30 con 0x7f8a50105c70 2024-03-04T15:39:57.495 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.507+0000 7f8a37fff700 1 -- 172.21.15.117:0/851560847 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8a44024960 con 0x7f8a50105c70 2024-03-04T15:39:57.495 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.507+0000 7f8a5509b700 1 -- 172.21.15.117:0/851560847 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a5019a1f0 con 0x7f8a50105c70 2024-03-04T15:39:57.495 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.507+0000 7f8a5509b700 1 -- 172.21.15.117:0/851560847 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a5019cee0 con 0x7f8a50105c70 2024-03-04T15:39:57.495 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.507+0000 7f8a37fff700 1 -- 172.21.15.117:0/851560847 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45927+0+0 (secure 0 0 0) 0x7f8a44022020 con 0x7f8a50105c70 2024-03-04T15:39:57.495 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.507+0000 7f8a5509b700 1 -- 172.21.15.117:0/851560847 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a50063410 con 0x7f8a50105c70 2024-03-04T15:39:57.495 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.507+0000 7f8a37fff700 1 --2- 172.21.15.117:0/851560847 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f8a38038ba0 0x7f8a3803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:39:57.495 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.507+0000 7f8a37fff700 1 -- 172.21.15.117:0/851560847 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f8a4404ce90 con 0x7f8a50105c70 2024-03-04T15:39:57.495 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.507+0000 7f8a4e59c700 1 --2- 172.21.15.117:0/851560847 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f8a38038ba0 0x7f8a3803b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:39:57.495 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.507+0000 7f8a4e59c700 1 --2- 172.21.15.117:0/851560847 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f8a38038ba0 0x7f8a3803b060 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8a40006fd0 tx=0x7f8a40006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:39:57.495 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.511+0000 7f8a37fff700 1 -- 172.21.15.117:0/851560847 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f8a44050030 con 0x7f8a50105c70 2024-03-04T15:39:57.495 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.863+0000 7f8a5509b700 1 -- 172.21.15.117:0/851560847 --> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f8a50102460 con 0x7f8a38038ba0 2024-03-04T15:39:57.495 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.863+0000 7f8a37fff700 1 -- 172.21.15.117:0/851560847 <== mgr.14120 v2:172.21.15.117:6800/1454790014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f8a50102460 con 0x7f8a38038ba0 2024-03-04T15:39:57.495 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.867+0000 7f8a5509b700 1 -- 172.21.15.117:0/851560847 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f8a38038ba0 msgr2=0x7f8a3803b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:57.496 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.867+0000 7f8a5509b700 1 --2- 172.21.15.117:0/851560847 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f8a38038ba0 0x7f8a3803b060 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f8a40006fd0 tx=0x7f8a40006e40 comp rx=0 tx=0).stop 2024-03-04T15:39:57.496 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.867+0000 7f8a5509b700 1 -- 172.21.15.117:0/851560847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a50105c70 msgr2=0x7f8a50199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:39:57.496 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.867+0000 7f8a5509b700 1 --2- 172.21.15.117:0/851560847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a50105c70 0x7f8a50199a20 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8a44003e40 tx=0x7f8a440119c0 comp rx=0 tx=0).stop 2024-03-04T15:39:57.496 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.867+0000 7f8a5509b700 1 -- 172.21.15.117:0/851560847 shutdown_connections 2024-03-04T15:39:57.496 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.867+0000 7f8a5509b700 1 --2- 172.21.15.117:0/851560847 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f8a38038ba0 0x7f8a3803b060 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:57.496 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.867+0000 7f8a5509b700 1 --2- 172.21.15.117:0/851560847 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8a50105c70 0x7f8a50199a20 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:39:57.496 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.867+0000 7f8a5509b700 1 -- 172.21.15.117:0/851560847 >> 172.21.15.117:0/851560847 conn(0x7f8a501011f0 msgr2=0x7f8a50101d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:39:57.496 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.867+0000 7f8a5509b700 1 -- 172.21.15.117:0/851560847 shutdown_connections 2024-03-04T15:39:57.496 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:56.867+0000 7f8a5509b700 1 -- 172.21.15.117:0/851560847 wait complete. 2024-03-04T15:39:57.496 INFO:teuthology.orchestra.run.smithi117.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-03-04T15:39:57.496 INFO:teuthology.orchestra.run.smithi117.stdout:Adding key to root@localhost authorized_keys... 2024-03-04T15:39:57.497 INFO:teuthology.orchestra.run.smithi117.stdout:Adding host smithi117... 2024-03-04T15:39:58.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:57 smithi117 bash[21480]: audit 2024-03-04T15:39:56.867205+0000 mgr.y (mgr.14120) 12 : audit 0 from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:40:00.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:59 smithi117 bash[21480]: audit 2024-03-04T15:39:58.748912+0000 mgr.y (mgr.14120) 13 : audit 0 from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi117", "addr": "172.21.15.117", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:40:00.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:39:59 smithi117 bash[21480]: cephadm 2024-03-04T15:39:59.371345+0000 mgr.y (mgr.14120) 14 : cephadm 1 Deploying cephadm binary to smithi117 2024-03-04T15:40:05.095 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Added host 'smithi117' with addr '172.21.15.117' 2024-03-04T15:40:05.095 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.355+0000 7f4c969aa700 1 Processor -- start 2024-03-04T15:40:05.095 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.355+0000 7f4c969aa700 1 -- start start 2024-03-04T15:40:05.095 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.355+0000 7f4c969aa700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c90104b40 0x7f4c90104f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:05.095 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.355+0000 7f4c969aa700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c90105530 con 0x7f4c90104b40 2024-03-04T15:40:05.095 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.355+0000 7f4c8ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c90104b40 0x7f4c90104f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:05.095 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.355+0000 7f4c8ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c90104b40 0x7f4c90104f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56682/0 (socket says 172.21.15.117:56682) 2024-03-04T15:40:05.095 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.355+0000 7f4c8ffff700 1 -- 172.21.15.117:0/2644981098 learned_addr learned my addr 172.21.15.117:0/2644981098 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:05.095 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.355+0000 7f4c8ffff700 1 -- 172.21.15.117:0/2644981098 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c90105d40 con 0x7f4c90104b40 2024-03-04T15:40:05.095 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.355+0000 7f4c8ffff700 1 --2- 172.21.15.117:0/2644981098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c90104b40 0x7f4c90104f60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000bb40 tx=0x7f4c8000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53cd06d1a3c2899d server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:05.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.355+0000 7f4c8effd700 1 -- 172.21.15.117:0/2644981098 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c8000dbe0 con 0x7f4c90104b40 2024-03-04T15:40:05.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.355+0000 7f4c8effd700 1 -- 172.21.15.117:0/2644981098 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f4c8000dd80 con 0x7f4c90104b40 2024-03-04T15:40:05.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.355+0000 7f4c8effd700 1 -- 172.21.15.117:0/2644981098 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c80003e00 con 0x7f4c90104b40 2024-03-04T15:40:05.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.355+0000 7f4c969aa700 1 -- 172.21.15.117:0/2644981098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c90104b40 msgr2=0x7f4c90104f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:05.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.355+0000 7f4c969aa700 1 --2- 172.21.15.117:0/2644981098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c90104b40 0x7f4c90104f60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000bb40 tx=0x7f4c8000be50 comp rx=0 tx=0).stop 2024-03-04T15:40:05.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.355+0000 7f4c969aa700 1 -- 172.21.15.117:0/2644981098 shutdown_connections 2024-03-04T15:40:05.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.355+0000 7f4c969aa700 1 --2- 172.21.15.117:0/2644981098 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c90104b40 0x7f4c90104f60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:05.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.355+0000 7f4c969aa700 1 -- 172.21.15.117:0/2644981098 >> 172.21.15.117:0/2644981098 conn(0x7f4c901000a0 msgr2=0x7f4c90102520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:05.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.355+0000 7f4c969aa700 1 -- 172.21.15.117:0/2644981098 shutdown_connections 2024-03-04T15:40:05.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.355+0000 7f4c969aa700 1 -- 172.21.15.117:0/2644981098 wait complete. 2024-03-04T15:40:05.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.359+0000 7f4c969aa700 1 Processor -- start 2024-03-04T15:40:05.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.359+0000 7f4c969aa700 1 -- start start 2024-03-04T15:40:05.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.359+0000 7f4c969aa700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c90104b40 0x7f4c9019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:05.096 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.359+0000 7f4c969aa700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c90106b10 con 0x7f4c90104b40 2024-03-04T15:40:05.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.359+0000 7f4c8ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c90104b40 0x7f4c9019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:05.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.359+0000 7f4c8ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c90104b40 0x7f4c9019bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56694/0 (socket says 172.21.15.117:56694) 2024-03-04T15:40:05.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.359+0000 7f4c8ffff700 1 -- 172.21.15.117:0/1046354517 learned_addr learned my addr 172.21.15.117:0/1046354517 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:05.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.359+0000 7f4c8ffff700 1 -- 172.21.15.117:0/1046354517 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c9019c1d0 con 0x7f4c90104b40 2024-03-04T15:40:05.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.359+0000 7f4c8ffff700 1 --2- 172.21.15.117:0/1046354517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c90104b40 0x7f4c9019bc90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4c80003b20 tx=0x7f4c800038d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:05.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.359+0000 7f4c8d7fa700 1 -- 172.21.15.117:0/1046354517 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c80010630 con 0x7f4c90104b40 2024-03-04T15:40:05.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.359+0000 7f4c8d7fa700 1 -- 172.21.15.117:0/1046354517 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f4c8001d070 con 0x7f4c90104b40 2024-03-04T15:40:05.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.359+0000 7f4c8d7fa700 1 -- 172.21.15.117:0/1046354517 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4c800219f0 con 0x7f4c90104b40 2024-03-04T15:40:05.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.359+0000 7f4c969aa700 1 -- 172.21.15.117:0/1046354517 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c9019c460 con 0x7f4c90104b40 2024-03-04T15:40:05.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.359+0000 7f4c969aa700 1 -- 172.21.15.117:0/1046354517 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c9019f150 con 0x7f4c90104b40 2024-03-04T15:40:05.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.359+0000 7f4c8d7fa700 1 -- 172.21.15.117:0/1046354517 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45927+0+0 (secure 0 0 0) 0x7f4c8001f030 con 0x7f4c90104b40 2024-03-04T15:40:05.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.359+0000 7f4c8d7fa700 1 --2- 172.21.15.117:0/1046354517 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f4c78038bf0 0x7f4c7803b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:05.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.359+0000 7f4c969aa700 1 -- 172.21.15.117:0/1046354517 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c90063410 con 0x7f4c90104b40 2024-03-04T15:40:05.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.359+0000 7f4c8d7fa700 1 -- 172.21.15.117:0/1046354517 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4c8004cf40 con 0x7f4c90104b40 2024-03-04T15:40:05.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.359+0000 7f4c8f7fe700 1 --2- 172.21.15.117:0/1046354517 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f4c78038bf0 0x7f4c7803b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:05.097 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.359+0000 7f4c8f7fe700 1 --2- 172.21.15.117:0/1046354517 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f4c78038bf0 0x7f4c7803b0b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4c84006fd0 tx=0x7f4c84006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:05.098 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.367+0000 7f4c8d7fa700 1 -- 172.21.15.117:0/1046354517 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4c80015070 con 0x7f4c90104b40 2024-03-04T15:40:05.098 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:39:58.743+0000 7f4c969aa700 1 -- 172.21.15.117:0/1046354517 --> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi117", "addr": "172.21.15.117", "target": ["mon-mgr", ""]}) v1 -- 0x7f4c90062220 con 0x7f4c78038bf0 2024-03-04T15:40:05.098 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:04.266+0000 7f4c8d7fa700 1 -- 172.21.15.117:0/1046354517 <== mgr.14120 v2:172.21.15.117:6800/1454790014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f4c90062220 con 0x7f4c78038bf0 2024-03-04T15:40:05.098 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:04.270+0000 7f4c969aa700 1 -- 172.21.15.117:0/1046354517 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f4c78038bf0 msgr2=0x7f4c7803b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:05.098 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:04.270+0000 7f4c969aa700 1 --2- 172.21.15.117:0/1046354517 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f4c78038bf0 0x7f4c7803b0b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4c84006fd0 tx=0x7f4c84006e40 comp rx=0 tx=0).stop 2024-03-04T15:40:05.098 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:04.270+0000 7f4c969aa700 1 -- 172.21.15.117:0/1046354517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c90104b40 msgr2=0x7f4c9019bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:05.098 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:04.270+0000 7f4c969aa700 1 --2- 172.21.15.117:0/1046354517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c90104b40 0x7f4c9019bc90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f4c80003b20 tx=0x7f4c800038d0 comp rx=0 tx=0).stop 2024-03-04T15:40:05.098 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:04.270+0000 7f4c969aa700 1 -- 172.21.15.117:0/1046354517 shutdown_connections 2024-03-04T15:40:05.098 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:04.270+0000 7f4c969aa700 1 --2- 172.21.15.117:0/1046354517 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f4c78038bf0 0x7f4c7803b0b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:05.098 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:04.270+0000 7f4c969aa700 1 --2- 172.21.15.117:0/1046354517 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4c90104b40 0x7f4c9019bc90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:05.098 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:04.270+0000 7f4c969aa700 1 -- 172.21.15.117:0/1046354517 >> 172.21.15.117:0/1046354517 conn(0x7f4c901000a0 msgr2=0x7f4c901922d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:05.098 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:04.270+0000 7f4c969aa700 1 -- 172.21.15.117:0/1046354517 shutdown_connections 2024-03-04T15:40:05.098 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:04.270+0000 7f4c969aa700 1 -- 172.21.15.117:0/1046354517 wait complete. 2024-03-04T15:40:05.098 INFO:teuthology.orchestra.run.smithi117.stdout:Deploying unmanaged mon service... 2024-03-04T15:40:05.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:05 smithi117 bash[21480]: audit 2024-03-04T15:40:04.271035+0000 mon.a (mon.0) 57 : audit 1 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' 2024-03-04T15:40:05.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:05 smithi117 bash[21480]: cephadm 2024-03-04T15:40:04.271604+0000 mgr.y (mgr.14120) 15 : cephadm 1 Added host smithi117 2024-03-04T15:40:05.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:05 smithi117 bash[21480]: audit 2024-03-04T15:40:04.319009+0000 mon.a (mon.0) 58 : audit 0 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:40:07.624 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-03-04T15:40:07.624 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.410+0000 7f5f0abe0700 1 Processor -- start 2024-03-04T15:40:07.624 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.410+0000 7f5f0abe0700 1 -- start start 2024-03-04T15:40:07.624 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.410+0000 7f5f0abe0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f041066a0 0x7f5f04108a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:07.624 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.410+0000 7f5f0abe0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f04078ea0 con 0x7f5f041066a0 2024-03-04T15:40:07.624 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.410+0000 7f5f0897c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f041066a0 0x7f5f04108a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:07.624 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0897c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f041066a0 0x7f5f04108a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56710/0 (socket says 172.21.15.117:56710) 2024-03-04T15:40:07.624 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0897c700 1 -- 172.21.15.117:0/3556338374 learned_addr learned my addr 172.21.15.117:0/3556338374 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:07.624 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0897c700 1 -- 172.21.15.117:0/3556338374 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f04108fd0 con 0x7f5f041066a0 2024-03-04T15:40:07.624 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0897c700 1 --2- 172.21.15.117:0/3556338374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f041066a0 0x7f5f04108a90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80098a0 tx=0x7f5ef8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5ede4a8d2ca6240 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:07.625 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f037fe700 1 -- 172.21.15.117:0/3556338374 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ef8004160 con 0x7f5f041066a0 2024-03-04T15:40:07.625 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f037fe700 1 -- 172.21.15.117:0/3556338374 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f5ef8004300 con 0x7f5f041066a0 2024-03-04T15:40:07.625 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f037fe700 1 -- 172.21.15.117:0/3556338374 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ef8011940 con 0x7f5f041066a0 2024-03-04T15:40:07.625 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0abe0700 1 -- 172.21.15.117:0/3556338374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f041066a0 msgr2=0x7f5f04108a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:07.625 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0abe0700 1 --2- 172.21.15.117:0/3556338374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f041066a0 0x7f5f04108a90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5ef80098a0 tx=0x7f5ef8009bb0 comp rx=0 tx=0).stop 2024-03-04T15:40:07.625 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0abe0700 1 -- 172.21.15.117:0/3556338374 shutdown_connections 2024-03-04T15:40:07.625 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0abe0700 1 --2- 172.21.15.117:0/3556338374 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f041066a0 0x7f5f04108a90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:07.625 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0abe0700 1 -- 172.21.15.117:0/3556338374 >> 172.21.15.117:0/3556338374 conn(0x7f5f041000d0 msgr2=0x7f5f04102510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:07.625 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0abe0700 1 -- 172.21.15.117:0/3556338374 shutdown_connections 2024-03-04T15:40:07.625 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0abe0700 1 -- 172.21.15.117:0/3556338374 wait complete. 2024-03-04T15:40:07.625 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0abe0700 1 Processor -- start 2024-03-04T15:40:07.625 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0abe0700 1 -- start start 2024-03-04T15:40:07.625 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0abe0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f041066a0 0x7f5f04199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:07.625 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0abe0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f04109e00 con 0x7f5f041066a0 2024-03-04T15:40:07.625 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0897c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f041066a0 0x7f5f04199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:07.625 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0897c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f041066a0 0x7f5f04199a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56726/0 (socket says 172.21.15.117:56726) 2024-03-04T15:40:07.626 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0897c700 1 -- 172.21.15.117:0/3455670638 learned_addr learned my addr 172.21.15.117:0/3455670638 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:07.626 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0897c700 1 -- 172.21.15.117:0/3455670638 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f04199fd0 con 0x7f5f041066a0 2024-03-04T15:40:07.626 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0897c700 1 --2- 172.21.15.117:0/3455670638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f041066a0 0x7f5f04199a90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8011910 tx=0x7f5ef80119f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:07.626 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f01ffb700 1 -- 172.21.15.117:0/3455670638 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ef8003ac0 con 0x7f5f041066a0 2024-03-04T15:40:07.626 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f01ffb700 1 -- 172.21.15.117:0/3455670638 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1011+0+0 (secure 0 0 0) 0x7f5ef801a890 con 0x7f5f041066a0 2024-03-04T15:40:07.626 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f01ffb700 1 -- 172.21.15.117:0/3455670638 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5ef80197a0 con 0x7f5f041066a0 2024-03-04T15:40:07.626 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0abe0700 1 -- 172.21.15.117:0/3455670638 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f0419a260 con 0x7f5f041066a0 2024-03-04T15:40:07.626 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.414+0000 7f5f0abe0700 1 -- 172.21.15.117:0/3455670638 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f0419cf00 con 0x7f5f041066a0 2024-03-04T15:40:07.626 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.418+0000 7f5f01ffb700 1 -- 172.21.15.117:0/3455670638 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45927+0+0 (secure 0 0 0) 0x7f5ef801a430 con 0x7f5f041066a0 2024-03-04T15:40:07.626 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.418+0000 7f5f0abe0700 1 -- 172.21.15.117:0/3455670638 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f04063410 con 0x7f5f041066a0 2024-03-04T15:40:07.626 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.418+0000 7f5f01ffb700 1 --2- 172.21.15.117:0/3455670638 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f5ef4038bf0 0x7f5ef403b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:07.626 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.418+0000 7f5f01ffb700 1 -- 172.21.15.117:0/3455670638 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f5ef804cf00 con 0x7f5f041066a0 2024-03-04T15:40:07.626 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.418+0000 7f5f03fff700 1 --2- 172.21.15.117:0/3455670638 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f5ef4038bf0 0x7f5ef403b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:07.626 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.418+0000 7f5f03fff700 1 --2- 172.21.15.117:0/3455670638 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f5ef4038bf0 0x7f5ef403b0b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0006fd0 tx=0x7f5ef0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:07.626 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.422+0000 7f5f01ffb700 1 -- 172.21.15.117:0/3455670638 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5ef801aa40 con 0x7f5f041066a0 2024-03-04T15:40:07.626 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.818+0000 7f5f0abe0700 1 -- 172.21.15.117:0/3455670638 --> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f5f04101340 con 0x7f5ef4038bf0 2024-03-04T15:40:07.627 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.826+0000 7f5f01ffb700 1 -- 172.21.15.117:0/3455670638 <== mgr.14120 v2:172.21.15.117:6800/1454790014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f5f04101340 con 0x7f5ef4038bf0 2024-03-04T15:40:07.627 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.830+0000 7f5f0abe0700 1 -- 172.21.15.117:0/3455670638 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f5ef4038bf0 msgr2=0x7f5ef403b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:07.627 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.830+0000 7f5f0abe0700 1 --2- 172.21.15.117:0/3455670638 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f5ef4038bf0 0x7f5ef403b0b0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f5ef0006fd0 tx=0x7f5ef0006e40 comp rx=0 tx=0).stop 2024-03-04T15:40:07.627 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.830+0000 7f5f0abe0700 1 -- 172.21.15.117:0/3455670638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f041066a0 msgr2=0x7f5f04199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:07.627 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.830+0000 7f5f0abe0700 1 --2- 172.21.15.117:0/3455670638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f041066a0 0x7f5f04199a90 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5ef8011910 tx=0x7f5ef80119f0 comp rx=0 tx=0).stop 2024-03-04T15:40:07.627 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.830+0000 7f5f0abe0700 1 -- 172.21.15.117:0/3455670638 shutdown_connections 2024-03-04T15:40:07.627 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.830+0000 7f5f0abe0700 1 --2- 172.21.15.117:0/3455670638 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f5ef4038bf0 0x7f5ef403b0b0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:07.627 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.830+0000 7f5f0abe0700 1 --2- 172.21.15.117:0/3455670638 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5f041066a0 0x7f5f04199a90 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:07.627 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.830+0000 7f5f0abe0700 1 -- 172.21.15.117:0/3455670638 >> 172.21.15.117:0/3455670638 conn(0x7f5f041000d0 msgr2=0x7f5f04100c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:07.627 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.830+0000 7f5f0abe0700 1 -- 172.21.15.117:0/3455670638 shutdown_connections 2024-03-04T15:40:07.627 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:06.830+0000 7f5f0abe0700 1 -- 172.21.15.117:0/3455670638 wait complete. 2024-03-04T15:40:07.627 INFO:teuthology.orchestra.run.smithi117.stdout:Deploying unmanaged mgr service... 2024-03-04T15:40:08.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:07 smithi117 bash[21480]: audit 2024-03-04T15:40:06.824008+0000 mgr.y (mgr.14120) 16 : audit 0 from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:40:08.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:07 smithi117 bash[21480]: cephadm 2024-03-04T15:40:06.827254+0000 mgr.y (mgr.14120) 17 : cephadm 1 Saving service mon spec with placement count:5 2024-03-04T15:40:08.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:07 smithi117 bash[21480]: audit 2024-03-04T15:40:06.830475+0000 mon.a (mon.0) 59 : audit 1 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' 2024-03-04T15:40:08.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:07 smithi117 bash[21480]: audit 2024-03-04T15:40:07.297635+0000 mon.a (mon.0) 60 : audit 1 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' 2024-03-04T15:40:08.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:07 smithi117 bash[21480]: cluster 2024-03-04T15:40:07.658599+0000 mgr.y (mgr.14120) 18 : cluster 0 pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:40:09.156 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:08 smithi117 bash[21480]: audit 2024-03-04T15:40:07.838834+0000 mon.a (mon.0) 61 : audit 1 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' 2024-03-04T15:40:09.584 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-03-04T15:40:09.584 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.550+0000 7f36d9f8f700 1 Processor -- start 2024-03-04T15:40:09.584 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.550+0000 7f36d9f8f700 1 -- start start 2024-03-04T15:40:09.584 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.550+0000 7f36d9f8f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36d4105c60 0x7f36d4106080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:09.584 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.550+0000 7f36d9f8f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d4106650 con 0x7f36d4105c60 2024-03-04T15:40:09.584 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.550+0000 7f36d37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36d4105c60 0x7f36d4106080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:09.584 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.550+0000 7f36d37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36d4105c60 0x7f36d4106080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58788/0 (socket says 172.21.15.117:58788) 2024-03-04T15:40:09.585 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.550+0000 7f36d37fe700 1 -- 172.21.15.117:0/3469499218 learned_addr learned my addr 172.21.15.117:0/3469499218 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:09.585 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.550+0000 7f36d37fe700 1 -- 172.21.15.117:0/3469499218 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d4106e60 con 0x7f36d4105c60 2024-03-04T15:40:09.585 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d37fe700 1 --2- 172.21.15.117:0/3469499218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36d4105c60 0x7f36d4106080 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f36c40098a0 tx=0x7f36c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4bb31066834f3e5 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:09.585 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d27fc700 1 -- 172.21.15.117:0/3469499218 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36c4004160 con 0x7f36d4105c60 2024-03-04T15:40:09.585 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d27fc700 1 -- 172.21.15.117:0/3469499218 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f36c4004300 con 0x7f36d4105c60 2024-03-04T15:40:09.585 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d27fc700 1 -- 172.21.15.117:0/3469499218 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36c40118f0 con 0x7f36d4105c60 2024-03-04T15:40:09.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d9f8f700 1 -- 172.21.15.117:0/3469499218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36d4105c60 msgr2=0x7f36d4106080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:09.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d9f8f700 1 --2- 172.21.15.117:0/3469499218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36d4105c60 0x7f36d4106080 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f36c40098a0 tx=0x7f36c4009bb0 comp rx=0 tx=0).stop 2024-03-04T15:40:09.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d9f8f700 1 -- 172.21.15.117:0/3469499218 shutdown_connections 2024-03-04T15:40:09.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d9f8f700 1 --2- 172.21.15.117:0/3469499218 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36d4105c60 0x7f36d4106080 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:09.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d9f8f700 1 -- 172.21.15.117:0/3469499218 >> 172.21.15.117:0/3469499218 conn(0x7f36d41011c0 msgr2=0x7f36d4103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:09.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d9f8f700 1 -- 172.21.15.117:0/3469499218 shutdown_connections 2024-03-04T15:40:09.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d9f8f700 1 -- 172.21.15.117:0/3469499218 wait complete. 2024-03-04T15:40:09.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d9f8f700 1 Processor -- start 2024-03-04T15:40:09.589 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d9f8f700 1 -- start start 2024-03-04T15:40:09.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d9f8f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36d4105c60 0x7f36d4199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:09.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d9f8f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d410a090 con 0x7f36d4105c60 2024-03-04T15:40:09.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36d4105c60 0x7f36d4199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:09.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36d4105c60 0x7f36d4199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58804/0 (socket says 172.21.15.117:58804) 2024-03-04T15:40:09.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d37fe700 1 -- 172.21.15.117:0/4211419834 learned_addr learned my addr 172.21.15.117:0/4211419834 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:09.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d37fe700 1 -- 172.21.15.117:0/4211419834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d4199fb0 con 0x7f36d4105c60 2024-03-04T15:40:09.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d37fe700 1 --2- 172.21.15.117:0/4211419834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36d4105c60 0x7f36d4199a70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009870 tx=0x7f36c4011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:09.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d0ff9700 1 -- 172.21.15.117:0/4211419834 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36c40038b0 con 0x7f36d4105c60 2024-03-04T15:40:09.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d0ff9700 1 -- 172.21.15.117:0/4211419834 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f36c4003a50 con 0x7f36d4105c60 2024-03-04T15:40:09.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d9f8f700 1 -- 172.21.15.117:0/4211419834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d419a240 con 0x7f36d4105c60 2024-03-04T15:40:09.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d9f8f700 1 -- 172.21.15.117:0/4211419834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d419cf30 con 0x7f36d4105c60 2024-03-04T15:40:09.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d0ff9700 1 -- 172.21.15.117:0/4211419834 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f36c4024410 con 0x7f36d4105c60 2024-03-04T15:40:09.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.554+0000 7f36d0ff9700 1 -- 172.21.15.117:0/4211419834 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45927+0+0 (secure 0 0 0) 0x7f36c4022020 con 0x7f36d4105c60 2024-03-04T15:40:09.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.558+0000 7f36d0ff9700 1 --2- 172.21.15.117:0/4211419834 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f36bc038bf0 0x7f36bc03b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:09.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.558+0000 7f36d0ff9700 1 -- 172.21.15.117:0/4211419834 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f36c404ccf0 con 0x7f36d4105c60 2024-03-04T15:40:09.590 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.558+0000 7f36d2ffd700 1 --2- 172.21.15.117:0/4211419834 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f36bc038bf0 0x7f36bc03b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:09.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.558+0000 7f36d9f8f700 1 -- 172.21.15.117:0/4211419834 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d4063410 con 0x7f36d4105c60 2024-03-04T15:40:09.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.558+0000 7f36d2ffd700 1 --2- 172.21.15.117:0/4211419834 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f36bc038bf0 0x7f36bc03b0b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f36c8006fd0 tx=0x7f36c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:09.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.562+0000 7f36d0ff9700 1 -- 172.21.15.117:0/4211419834 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f36c400b3e0 con 0x7f36d4105c60 2024-03-04T15:40:09.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.938+0000 7f36d9f8f700 1 -- 172.21.15.117:0/4211419834 --> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f36d4102450 con 0x7f36bc038bf0 2024-03-04T15:40:09.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.946+0000 7f36d0ff9700 1 -- 172.21.15.117:0/4211419834 <== mgr.14120 v2:172.21.15.117:6800/1454790014 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f36d4102450 con 0x7f36bc038bf0 2024-03-04T15:40:09.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.946+0000 7f36d9f8f700 1 -- 172.21.15.117:0/4211419834 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f36bc038bf0 msgr2=0x7f36bc03b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:09.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.946+0000 7f36d9f8f700 1 --2- 172.21.15.117:0/4211419834 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f36bc038bf0 0x7f36bc03b0b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f36c8006fd0 tx=0x7f36c8006e40 comp rx=0 tx=0).stop 2024-03-04T15:40:09.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.946+0000 7f36d9f8f700 1 -- 172.21.15.117:0/4211419834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36d4105c60 msgr2=0x7f36d4199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:09.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.946+0000 7f36d9f8f700 1 --2- 172.21.15.117:0/4211419834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36d4105c60 0x7f36d4199a70 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f36c4009870 tx=0x7f36c4011ef0 comp rx=0 tx=0).stop 2024-03-04T15:40:09.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.946+0000 7f36d9f8f700 1 -- 172.21.15.117:0/4211419834 shutdown_connections 2024-03-04T15:40:09.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.946+0000 7f36d9f8f700 1 --2- 172.21.15.117:0/4211419834 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f36bc038bf0 0x7f36bc03b0b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:09.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.946+0000 7f36d9f8f700 1 --2- 172.21.15.117:0/4211419834 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f36d4105c60 0x7f36d4199a70 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:09.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.946+0000 7f36d9f8f700 1 -- 172.21.15.117:0/4211419834 >> 172.21.15.117:0/4211419834 conn(0x7f36d41011c0 msgr2=0x7f36d4101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:09.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.946+0000 7f36d9f8f700 1 -- 172.21.15.117:0/4211419834 shutdown_connections 2024-03-04T15:40:09.591 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:08.946+0000 7f36d9f8f700 1 -- 172.21.15.117:0/4211419834 wait complete. 2024-03-04T15:40:10.222 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:09 smithi117 bash[21480]: audit 2024-03-04T15:40:08.943872+0000 mgr.y (mgr.14120) 19 : audit 0 from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:40:10.222 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:09 smithi117 bash[21480]: cephadm 2024-03-04T15:40:08.945441+0000 mgr.y (mgr.14120) 20 : cephadm 1 Saving service mgr spec with placement count:2 2024-03-04T15:40:10.222 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:09 smithi117 bash[21480]: audit 2024-03-04T15:40:08.948607+0000 mon.a (mon.0) 62 : audit 1 from='mgr.14120 172.21.15.117:0/196675734' entity='mgr.y' 2024-03-04T15:40:10.222 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:09 smithi117 bash[21480]: cluster 2024-03-04T15:40:09.658924+0000 mgr.y (mgr.14120) 21 : cluster 0 pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:40:11.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f8740096700 1 Processor -- start 2024-03-04T15:40:11.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f8740096700 1 -- start start 2024-03-04T15:40:11.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f8740096700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f873807b760 0x7f8738079bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:11.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f8740096700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873807a190 con 0x7f873807b760 2024-03-04T15:40:11.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f873de32700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f873807b760 0x7f8738079bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:11.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f873de32700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f873807b760 0x7f8738079bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58812/0 (socket says 172.21.15.117:58812) 2024-03-04T15:40:11.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f873de32700 1 -- 172.21.15.117:0/310852572 learned_addr learned my addr 172.21.15.117:0/310852572 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:11.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f873de32700 1 -- 172.21.15.117:0/310852572 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873807a310 con 0x7f873807b760 2024-03-04T15:40:11.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f873de32700 1 --2- 172.21.15.117:0/310852572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f873807b760 0x7f8738079bc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8730009e80 tx=0x7f873000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=888c1516a74af39c server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:11.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f873ce30700 1 -- 172.21.15.117:0/310852572 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8730017070 con 0x7f873807b760 2024-03-04T15:40:11.423 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f873ce30700 1 -- 172.21.15.117:0/310852572 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f873000b8c0 con 0x7f873807b760 2024-03-04T15:40:11.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f873ce30700 1 -- 172.21.15.117:0/310852572 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8730004370 con 0x7f873807b760 2024-03-04T15:40:11.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f8740096700 1 -- 172.21.15.117:0/310852572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f873807b760 msgr2=0x7f8738079bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:11.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f8740096700 1 --2- 172.21.15.117:0/310852572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f873807b760 0x7f8738079bc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8730009e80 tx=0x7f873000b330 comp rx=0 tx=0).stop 2024-03-04T15:40:11.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f8740096700 1 -- 172.21.15.117:0/310852572 shutdown_connections 2024-03-04T15:40:11.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f8740096700 1 --2- 172.21.15.117:0/310852572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f873807b760 0x7f8738079bc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:11.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f8740096700 1 -- 172.21.15.117:0/310852572 >> 172.21.15.117:0/310852572 conn(0x7f87381011f0 msgr2=0x7f8738103630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:11.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f8740096700 1 -- 172.21.15.117:0/310852572 shutdown_connections 2024-03-04T15:40:11.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f8740096700 1 -- 172.21.15.117:0/310852572 wait complete. 2024-03-04T15:40:11.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f8740096700 1 Processor -- start 2024-03-04T15:40:11.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f8740096700 1 -- start start 2024-03-04T15:40:11.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f8740096700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f873807b760 0x7f87381a0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:11.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f8740096700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8738107d90 con 0x7f873807b760 2024-03-04T15:40:11.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f873de32700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f873807b760 0x7f87381a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:11.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f873de32700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f873807b760 0x7f87381a0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58816/0 (socket says 172.21.15.117:58816) 2024-03-04T15:40:11.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f873de32700 1 -- 172.21.15.117:0/4212016343 learned_addr learned my addr 172.21.15.117:0/4212016343 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:11.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.430+0000 7f873de32700 1 -- 172.21.15.117:0/4212016343 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87381a0580 con 0x7f873807b760 2024-03-04T15:40:11.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.434+0000 7f873de32700 1 --2- 172.21.15.117:0/4212016343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f873807b760 0x7f87381a0040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8730009e50 tx=0x7f87300044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:11.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.434+0000 7f872affd700 1 -- 172.21.15.117:0/4212016343 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8730021020 con 0x7f873807b760 2024-03-04T15:40:11.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.434+0000 7f872affd700 1 -- 172.21.15.117:0/4212016343 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f8730003b50 con 0x7f873807b760 2024-03-04T15:40:11.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.434+0000 7f8740096700 1 -- 172.21.15.117:0/4212016343 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87381a0810 con 0x7f873807b760 2024-03-04T15:40:11.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.434+0000 7f8740096700 1 -- 172.21.15.117:0/4212016343 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87381a3500 con 0x7f873807b760 2024-03-04T15:40:11.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.434+0000 7f872affd700 1 -- 172.21.15.117:0/4212016343 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8730017040 con 0x7f873807b760 2024-03-04T15:40:11.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.434+0000 7f872affd700 1 -- 172.21.15.117:0/4212016343 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45927+0+0 (secure 0 0 0) 0x7f8730024020 con 0x7f873807b760 2024-03-04T15:40:11.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.434+0000 7f8740096700 1 -- 172.21.15.117:0/4212016343 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f871c005150 con 0x7f873807b760 2024-03-04T15:40:11.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.434+0000 7f872affd700 1 --2- 172.21.15.117:0/4212016343 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f8724038b50 0x7f872403b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:11.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.434+0000 7f872affd700 1 -- 172.21.15.117:0/4212016343 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f873000f050 con 0x7f873807b760 2024-03-04T15:40:11.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.434+0000 7f873d631700 1 --2- 172.21.15.117:0/4212016343 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f8724038b50 0x7f872403b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:11.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.434+0000 7f873d631700 1 --2- 172.21.15.117:0/4212016343 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f8724038b50 0x7f872403b010 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f872c006fd0 tx=0x7f872c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:11.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.438+0000 7f872affd700 1 -- 172.21.15.117:0/4212016343 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f873002b020 con 0x7f873807b760 2024-03-04T15:40:11.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.790+0000 7f8740096700 1 -- 172.21.15.117:0/4212016343 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f871c005440 con 0x7f873807b760 2024-03-04T15:40:11.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.790+0000 7f872affd700 1 -- 172.21.15.117:0/4212016343 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f8730012da0 con 0x7f873807b760 2024-03-04T15:40:11.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.794+0000 7f8740096700 1 -- 172.21.15.117:0/4212016343 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f8724038b50 msgr2=0x7f872403b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:11.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.794+0000 7f8740096700 1 --2- 172.21.15.117:0/4212016343 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f8724038b50 0x7f872403b010 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f872c006fd0 tx=0x7f872c006e40 comp rx=0 tx=0).stop 2024-03-04T15:40:11.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.794+0000 7f8740096700 1 -- 172.21.15.117:0/4212016343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f873807b760 msgr2=0x7f87381a0040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:11.430 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.794+0000 7f8740096700 1 --2- 172.21.15.117:0/4212016343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f873807b760 0x7f87381a0040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8730009e50 tx=0x7f87300044c0 comp rx=0 tx=0).stop 2024-03-04T15:40:11.430 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.794+0000 7f8740096700 1 -- 172.21.15.117:0/4212016343 shutdown_connections 2024-03-04T15:40:11.430 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.794+0000 7f8740096700 1 --2- 172.21.15.117:0/4212016343 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f8724038b50 0x7f872403b010 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:11.430 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.794+0000 7f8740096700 1 --2- 172.21.15.117:0/4212016343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f873807b760 0x7f87381a0040 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:11.430 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.794+0000 7f8740096700 1 -- 172.21.15.117:0/4212016343 >> 172.21.15.117:0/4212016343 conn(0x7f87381011f0 msgr2=0x7f8738196f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:11.430 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.794+0000 7f8740096700 1 -- 172.21.15.117:0/4212016343 shutdown_connections 2024-03-04T15:40:11.430 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:10.794+0000 7f8740096700 1 -- 172.21.15.117:0/4212016343 wait complete. 2024-03-04T15:40:12.053 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:11 smithi117 bash[21480]: audit 2024-03-04T15:40:10.796263+0000 mon.a (mon.0) 63 : audit 1 from='client.? 172.21.15.117:0/4212016343' entity='client.admin' 2024-03-04T15:40:12.053 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:11 smithi117 bash[21480]: cluster 2024-03-04T15:40:11.659349+0000 mgr.y (mgr.14120) 22 : cluster 0 pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:40:13.195 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.234+0000 7f25658ba700 1 Processor -- start 2024-03-04T15:40:13.195 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.234+0000 7f25658ba700 1 -- start start 2024-03-04T15:40:13.195 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.234+0000 7f25658ba700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560105c70 0x7f2560106090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:13.195 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.234+0000 7f25658ba700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2560106660 con 0x7f2560105c70 2024-03-04T15:40:13.195 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.234+0000 7f255effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560105c70 0x7f2560106090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:13.195 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.234+0000 7f255effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560105c70 0x7f2560106090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58828/0 (socket says 172.21.15.117:58828) 2024-03-04T15:40:13.195 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.234+0000 7f255effd700 1 -- 172.21.15.117:0/104253706 learned_addr learned my addr 172.21.15.117:0/104253706 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:13.195 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.234+0000 7f255effd700 1 -- 172.21.15.117:0/104253706 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2560106e70 con 0x7f2560105c70 2024-03-04T15:40:13.195 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.234+0000 7f255effd700 1 --2- 172.21.15.117:0/104253706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560105c70 0x7f2560106090 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f25500098a0 tx=0x7f2550009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9207493070915ada server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:13.196 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.234+0000 7f255dffb700 1 -- 172.21.15.117:0/104253706 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2550004160 con 0x7f2560105c70 2024-03-04T15:40:13.196 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.234+0000 7f255dffb700 1 -- 172.21.15.117:0/104253706 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f2550004300 con 0x7f2560105c70 2024-03-04T15:40:13.200 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.234+0000 7f255dffb700 1 -- 172.21.15.117:0/104253706 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25500118f0 con 0x7f2560105c70 2024-03-04T15:40:13.200 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f25658ba700 1 -- 172.21.15.117:0/104253706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560105c70 msgr2=0x7f2560106090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:13.200 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f25658ba700 1 --2- 172.21.15.117:0/104253706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560105c70 0x7f2560106090 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f25500098a0 tx=0x7f2550009bb0 comp rx=0 tx=0).stop 2024-03-04T15:40:13.200 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f25658ba700 1 -- 172.21.15.117:0/104253706 shutdown_connections 2024-03-04T15:40:13.200 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f25658ba700 1 --2- 172.21.15.117:0/104253706 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560105c70 0x7f2560106090 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:13.200 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f25658ba700 1 -- 172.21.15.117:0/104253706 >> 172.21.15.117:0/104253706 conn(0x7f25601011f0 msgr2=0x7f2560103650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:13.200 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f25658ba700 1 -- 172.21.15.117:0/104253706 shutdown_connections 2024-03-04T15:40:13.200 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f25658ba700 1 -- 172.21.15.117:0/104253706 wait complete. 2024-03-04T15:40:13.200 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f25658ba700 1 Processor -- start 2024-03-04T15:40:13.200 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f25658ba700 1 -- start start 2024-03-04T15:40:13.201 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f25658ba700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560105c70 0x7f25601999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:13.201 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f25658ba700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f256010a0a0 con 0x7f2560105c70 2024-03-04T15:40:13.201 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f255effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560105c70 0x7f25601999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:13.201 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f255effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560105c70 0x7f25601999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58834/0 (socket says 172.21.15.117:58834) 2024-03-04T15:40:13.201 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f255effd700 1 -- 172.21.15.117:0/1290520914 learned_addr learned my addr 172.21.15.117:0/1290520914 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:13.201 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f255effd700 1 -- 172.21.15.117:0/1290520914 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2560199f20 con 0x7f2560105c70 2024-03-04T15:40:13.201 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f255effd700 1 --2- 172.21.15.117:0/1290520914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560105c70 0x7f25601999e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2550003f60 tx=0x7f25500119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:13.201 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f25648b8700 1 -- 172.21.15.117:0/1290520914 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2550003a40 con 0x7f2560105c70 2024-03-04T15:40:13.201 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f25648b8700 1 -- 172.21.15.117:0/1290520914 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f25500248b0 con 0x7f2560105c70 2024-03-04T15:40:13.201 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f25648b8700 1 -- 172.21.15.117:0/1290520914 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f255001a670 con 0x7f2560105c70 2024-03-04T15:40:13.201 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f25658ba700 1 -- 172.21.15.117:0/1290520914 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f256019a1b0 con 0x7f2560105c70 2024-03-04T15:40:13.201 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f25658ba700 1 -- 172.21.15.117:0/1290520914 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f256019cea0 con 0x7f2560105c70 2024-03-04T15:40:13.201 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f25648b8700 1 -- 172.21.15.117:0/1290520914 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45927+0+0 (secure 0 0 0) 0x7f255001a920 con 0x7f2560105c70 2024-03-04T15:40:13.201 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f25648b8700 1 --2- 172.21.15.117:0/1290520914 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f2548038ba0 0x7f254803b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:13.201 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f25648b8700 1 -- 172.21.15.117:0/1290520914 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f255004cd20 con 0x7f2560105c70 2024-03-04T15:40:13.202 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f25658ba700 1 -- 172.21.15.117:0/1290520914 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2560063410 con 0x7f2560105c70 2024-03-04T15:40:13.202 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.238+0000 7f255e7fc700 1 --2- 172.21.15.117:0/1290520914 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f2548038ba0 0x7f254803b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:13.202 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.242+0000 7f255e7fc700 1 --2- 172.21.15.117:0/1290520914 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f2548038ba0 0x7f254803b060 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2554006fd0 tx=0x7f2554006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:13.202 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.246+0000 7f25648b8700 1 -- 172.21.15.117:0/1290520914 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2550019c70 con 0x7f2560105c70 2024-03-04T15:40:13.202 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.594+0000 7f25658ba700 1 -- 172.21.15.117:0/1290520914 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f256019d2b0 con 0x7f2560105c70 2024-03-04T15:40:13.202 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.598+0000 7f25648b8700 1 -- 172.21.15.117:0/1290520914 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f255004ca80 con 0x7f2560105c70 2024-03-04T15:40:13.202 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.602+0000 7f25658ba700 1 -- 172.21.15.117:0/1290520914 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f2548038ba0 msgr2=0x7f254803b060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:13.202 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.602+0000 7f25658ba700 1 --2- 172.21.15.117:0/1290520914 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f2548038ba0 0x7f254803b060 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f2554006fd0 tx=0x7f2554006e40 comp rx=0 tx=0).stop 2024-03-04T15:40:13.202 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.602+0000 7f25658ba700 1 -- 172.21.15.117:0/1290520914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560105c70 msgr2=0x7f25601999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:13.202 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.602+0000 7f25658ba700 1 --2- 172.21.15.117:0/1290520914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560105c70 0x7f25601999e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2550003f60 tx=0x7f25500119b0 comp rx=0 tx=0).stop 2024-03-04T15:40:13.202 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.602+0000 7f25658ba700 1 -- 172.21.15.117:0/1290520914 shutdown_connections 2024-03-04T15:40:13.202 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.602+0000 7f25658ba700 1 --2- 172.21.15.117:0/1290520914 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f2548038ba0 0x7f254803b060 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:13.202 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.602+0000 7f25658ba700 1 --2- 172.21.15.117:0/1290520914 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2560105c70 0x7f25601999e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:13.202 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.602+0000 7f25658ba700 1 -- 172.21.15.117:0/1290520914 >> 172.21.15.117:0/1290520914 conn(0x7f25601011f0 msgr2=0x7f2560101cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:13.202 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.602+0000 7f25658ba700 1 -- 172.21.15.117:0/1290520914 shutdown_connections 2024-03-04T15:40:13.203 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:12.602+0000 7f25658ba700 1 -- 172.21.15.117:0/1290520914 wait complete. 2024-03-04T15:40:13.203 INFO:teuthology.orchestra.run.smithi117.stdout:Enabling the dashboard module... 2024-03-04T15:40:13.858 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:13 smithi117 bash[21480]: audit 2024-03-04T15:40:12.604019+0000 mon.a (mon.0) 64 : audit 1 from='client.? 172.21.15.117:0/1290520914' entity='client.admin' 2024-03-04T15:40:15.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:14 smithi117 bash[21480]: cluster 2024-03-04T15:40:13.659957+0000 mgr.y (mgr.14120) 23 : cluster 0 pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:40:15.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:14 smithi117 bash[21480]: audit 2024-03-04T15:40:14.520366+0000 mon.a (mon.0) 65 : audit 1 from='client.? 172.21.15.117:0/4069975253' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-03-04T15:40:15.072 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:14 smithi117 bash[21734]: ignoring --setuser ceph since I am not root 2024-03-04T15:40:15.072 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:14 smithi117 bash[21734]: ignoring --setgroup ceph since I am not root 2024-03-04T15:40:15.072 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:14 smithi117 bash[21734]: debug 2024-03-04T15:40:14.798+0000 7f5841d0f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T15:40:15.340 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.030+0000 7f3ad6b33700 1 Processor -- start 2024-03-04T15:40:15.340 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.030+0000 7f3ad6b33700 1 -- start start 2024-03-04T15:40:15.340 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.030+0000 7f3ad6b33700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad007b720 0x7f3ad0079b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:15.341 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.030+0000 7f3ad6b33700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad007a150 con 0x7f3ad007b720 2024-03-04T15:40:15.341 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.030+0000 7f3ad48cf700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad007b720 0x7f3ad0079b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:15.341 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.030+0000 7f3ad48cf700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad007b720 0x7f3ad0079b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58840/0 (socket says 172.21.15.117:58840) 2024-03-04T15:40:15.341 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.030+0000 7f3ad48cf700 1 -- 172.21.15.117:0/3150286291 learned_addr learned my addr 172.21.15.117:0/3150286291 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:15.341 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3ad48cf700 1 -- 172.21.15.117:0/3150286291 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad007a2d0 con 0x7f3ad007b720 2024-03-04T15:40:15.341 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3ad48cf700 1 --2- 172.21.15.117:0/3150286291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad007b720 0x7f3ad0079b80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009e80 tx=0x7f3ac000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec0b50aa8590550b server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:15.341 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3acf7fe700 1 -- 172.21.15.117:0/3150286291 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ac0017070 con 0x7f3ad007b720 2024-03-04T15:40:15.345 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3acf7fe700 1 -- 172.21.15.117:0/3150286291 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f3ac000b8c0 con 0x7f3ad007b720 2024-03-04T15:40:15.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3acf7fe700 1 -- 172.21.15.117:0/3150286291 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ac0004370 con 0x7f3ad007b720 2024-03-04T15:40:15.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3ad6b33700 1 -- 172.21.15.117:0/3150286291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad007b720 msgr2=0x7f3ad0079b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:15.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3ad6b33700 1 --2- 172.21.15.117:0/3150286291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad007b720 0x7f3ad0079b80 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009e80 tx=0x7f3ac000b330 comp rx=0 tx=0).stop 2024-03-04T15:40:15.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3ad6b33700 1 -- 172.21.15.117:0/3150286291 shutdown_connections 2024-03-04T15:40:15.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3ad6b33700 1 --2- 172.21.15.117:0/3150286291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad007b720 0x7f3ad0079b80 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:15.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3ad6b33700 1 -- 172.21.15.117:0/3150286291 >> 172.21.15.117:0/3150286291 conn(0x7f3ad01011c0 msgr2=0x7f3ad0103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:15.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3ad6b33700 1 -- 172.21.15.117:0/3150286291 shutdown_connections 2024-03-04T15:40:15.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3ad6b33700 1 -- 172.21.15.117:0/3150286291 wait complete. 2024-03-04T15:40:15.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3ad6b33700 1 Processor -- start 2024-03-04T15:40:15.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3ad6b33700 1 -- start start 2024-03-04T15:40:15.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3ad6b33700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad007b720 0x7f3ad019bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:15.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3ad6b33700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ad010c1a0 con 0x7f3ad007b720 2024-03-04T15:40:15.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3ad48cf700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad007b720 0x7f3ad019bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:15.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3ad48cf700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad007b720 0x7f3ad019bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58846/0 (socket says 172.21.15.117:58846) 2024-03-04T15:40:15.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3ad48cf700 1 -- 172.21.15.117:0/4069975253 learned_addr learned my addr 172.21.15.117:0/4069975253 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:15.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3ad48cf700 1 -- 172.21.15.117:0/4069975253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ad019c190 con 0x7f3ad007b720 2024-03-04T15:40:15.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3ad48cf700 1 --2- 172.21.15.117:0/4069975253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad007b720 0x7f3ad019bc50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009e50 tx=0x7f3ac0003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:15.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3acdffb700 1 -- 172.21.15.117:0/4069975253 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ac0021020 con 0x7f3ad007b720 2024-03-04T15:40:15.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3ad6b33700 1 -- 172.21.15.117:0/4069975253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ad019c420 con 0x7f3ad007b720 2024-03-04T15:40:15.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3acdffb700 1 -- 172.21.15.117:0/4069975253 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f3ac00094d0 con 0x7f3ad007b720 2024-03-04T15:40:15.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3acdffb700 1 -- 172.21.15.117:0/4069975253 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ac0017040 con 0x7f3ad007b720 2024-03-04T15:40:15.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.034+0000 7f3ad6b33700 1 -- 172.21.15.117:0/4069975253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ad019f110 con 0x7f3ad007b720 2024-03-04T15:40:15.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.038+0000 7f3acdffb700 1 -- 172.21.15.117:0/4069975253 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 8) v1 ==== 45927+0+0 (secure 0 0 0) 0x7f3ac0024020 con 0x7f3ad007b720 2024-03-04T15:40:15.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.038+0000 7f3acdffb700 1 --2- 172.21.15.117:0/4069975253 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f3abc038b50 0x7f3abc03b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:15.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.038+0000 7f3ad6b33700 1 -- 172.21.15.117:0/4069975253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ad0063410 con 0x7f3ad007b720 2024-03-04T15:40:15.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.038+0000 7f3acdffb700 1 -- 172.21.15.117:0/4069975253 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3ac000f050 con 0x7f3ad007b720 2024-03-04T15:40:15.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.038+0000 7f3acffff700 1 --2- 172.21.15.117:0/4069975253 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f3abc038b50 0x7f3abc03b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:15.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.038+0000 7f3acffff700 1 --2- 172.21.15.117:0/4069975253 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f3abc038b50 0x7f3abc03b010 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8006fd0 tx=0x7f3ab8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:15.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.042+0000 7f3acdffb700 1 -- 172.21.15.117:0/4069975253 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3ac002b020 con 0x7f3ad007b720 2024-03-04T15:40:15.348 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.514+0000 7f3ad6b33700 1 -- 172.21.15.117:0/4069975253 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f3ad019f470 con 0x7f3ad007b720 2024-03-04T15:40:15.348 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.606+0000 7f3acdffb700 1 -- 172.21.15.117:0/4069975253 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f3ac004cb20 con 0x7f3ad007b720 2024-03-04T15:40:15.348 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.610+0000 7f3acdffb700 1 -- 172.21.15.117:0/4069975253 <== mon.0 v2:172.21.15.117:3300/0 8 ==== mgrmap(e 9) v1 ==== 45940+0+0 (secure 0 0 0) 0x7f3ac001bc90 con 0x7f3ad007b720 2024-03-04T15:40:15.348 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.610+0000 7f3ad6b33700 1 -- 172.21.15.117:0/4069975253 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f3abc038b50 msgr2=0x7f3abc03b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:15.348 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.610+0000 7f3ad6b33700 1 --2- 172.21.15.117:0/4069975253 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f3abc038b50 0x7f3abc03b010 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f3ab8006fd0 tx=0x7f3ab8006e40 comp rx=0 tx=0).stop 2024-03-04T15:40:15.348 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.610+0000 7f3ad6b33700 1 -- 172.21.15.117:0/4069975253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad007b720 msgr2=0x7f3ad019bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:15.348 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.610+0000 7f3ad6b33700 1 --2- 172.21.15.117:0/4069975253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad007b720 0x7f3ad019bc50 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3ac0009e50 tx=0x7f3ac0003d00 comp rx=0 tx=0).stop 2024-03-04T15:40:15.348 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.610+0000 7f3ad6b33700 1 -- 172.21.15.117:0/4069975253 shutdown_connections 2024-03-04T15:40:15.348 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.610+0000 7f3ad6b33700 1 --2- 172.21.15.117:0/4069975253 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f3abc038b50 0x7f3abc03b010 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:15.348 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.610+0000 7f3ad6b33700 1 --2- 172.21.15.117:0/4069975253 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3ad007b720 0x7f3ad019bc50 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:15.348 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.610+0000 7f3ad6b33700 1 -- 172.21.15.117:0/4069975253 >> 172.21.15.117:0/4069975253 conn(0x7f3ad01011c0 msgr2=0x7f3ad0101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:15.348 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.610+0000 7f3ad6b33700 1 -- 172.21.15.117:0/4069975253 shutdown_connections 2024-03-04T15:40:15.348 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:14.610+0000 7f3ad6b33700 1 -- 172.21.15.117:0/4069975253 wait complete. 2024-03-04T15:40:15.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:15 smithi117 bash[21480]: audit 2024-03-04T15:40:14.609698+0000 mon.a (mon.0) 66 : audit 1 from='client.? 172.21.15.117:0/4069975253' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-03-04T15:40:15.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:15 smithi117 bash[21480]: cluster 2024-03-04T15:40:14.615480+0000 mon.a (mon.0) 67 : cluster 0 mgrmap e9: y(active, since 26s) 2024-03-04T15:40:16.663 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:16 smithi117 bash[21734]: debug 2024-03-04T15:40:16.374+0000 7f5841d0f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T15:40:16.663 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:16 smithi117 bash[21734]: debug 2024-03-04T15:40:16.470+0000 7f5841d0f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T15:40:17.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:16 smithi117 bash[21480]: audit 2024-03-04T15:40:16.648310+0000 mon.a (mon.0) 68 : audit 0 from='client.? 172.21.15.117:0/3431845729' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-03-04T15:40:17.072 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:16 smithi117 bash[21734]: debug 2024-03-04T15:40:16.726+0000 7f5841d0f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T15:40:17.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-03-04T15:40:17.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-03-04T15:40:17.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "available": true, 2024-03-04T15:40:17.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-03-04T15:40:17.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-03-04T15:40:17.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-03-04T15:40:17.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec4d58a700 1 Processor -- start 2024-03-04T15:40:17.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec4d58a700 1 -- start start 2024-03-04T15:40:17.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec4d58a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec48106720 0x7fec48108b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:17.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec4d58a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec480796b0 con 0x7fec48106720 2024-03-04T15:40:17.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec46ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec48106720 0x7fec48108b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:17.334 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec46ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec48106720 0x7fec48108b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58878/0 (socket says 172.21.15.117:58878) 2024-03-04T15:40:17.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec46ffd700 1 -- 172.21.15.117:0/526529394 learned_addr learned my addr 172.21.15.117:0/526529394 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:17.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec46ffd700 1 -- 172.21.15.117:0/526529394 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec48109050 con 0x7fec48106720 2024-03-04T15:40:17.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec46ffd700 1 --2- 172.21.15.117:0/526529394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec48106720 0x7fec48108b10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fec380098a0 tx=0x7fec38009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ec900a721a6954 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:17.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec45ffb700 1 -- 172.21.15.117:0/526529394 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec38004160 con 0x7fec48106720 2024-03-04T15:40:17.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec45ffb700 1 -- 172.21.15.117:0/526529394 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fec38004300 con 0x7fec48106720 2024-03-04T15:40:17.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec45ffb700 1 -- 172.21.15.117:0/526529394 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec380118f0 con 0x7fec48106720 2024-03-04T15:40:17.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec4d58a700 1 -- 172.21.15.117:0/526529394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec48106720 msgr2=0x7fec48108b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:17.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec4d58a700 1 --2- 172.21.15.117:0/526529394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec48106720 0x7fec48108b10 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fec380098a0 tx=0x7fec38009bb0 comp rx=0 tx=0).stop 2024-03-04T15:40:17.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec4d58a700 1 -- 172.21.15.117:0/526529394 shutdown_connections 2024-03-04T15:40:17.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec4d58a700 1 --2- 172.21.15.117:0/526529394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec48106720 0x7fec48108b10 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:17.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec4d58a700 1 -- 172.21.15.117:0/526529394 >> 172.21.15.117:0/526529394 conn(0x7fec481000b0 msgr2=0x7fec48102510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:17.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec4d58a700 1 -- 172.21.15.117:0/526529394 shutdown_connections 2024-03-04T15:40:17.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec4d58a700 1 -- 172.21.15.117:0/526529394 wait complete. 2024-03-04T15:40:17.335 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec4d58a700 1 Processor -- start 2024-03-04T15:40:17.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec4d58a700 1 -- start start 2024-03-04T15:40:17.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec4d58a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec48106720 0x7fec48105fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:17.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec4d58a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec48109f30 con 0x7fec48106720 2024-03-04T15:40:17.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec46ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec48106720 0x7fec48105fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:17.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec46ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec48106720 0x7fec48105fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58892/0 (socket says 172.21.15.117:58892) 2024-03-04T15:40:17.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.174+0000 7fec46ffd700 1 -- 172.21.15.117:0/3431845729 learned_addr learned my addr 172.21.15.117:0/3431845729 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:17.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.178+0000 7fec46ffd700 1 -- 172.21.15.117:0/3431845729 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec481064e0 con 0x7fec48106720 2024-03-04T15:40:17.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.178+0000 7fec46ffd700 1 --2- 172.21.15.117:0/3431845729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec48106720 0x7fec48105fa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fec38003f60 tx=0x7fec38011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:17.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.178+0000 7fec2ffff700 1 -- 172.21.15.117:0/3431845729 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec380038b0 con 0x7fec48106720 2024-03-04T15:40:17.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.178+0000 7fec2ffff700 1 -- 172.21.15.117:0/3431845729 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fec38003a50 con 0x7fec48106720 2024-03-04T15:40:17.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.178+0000 7fec4d58a700 1 -- 172.21.15.117:0/3431845729 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec481046d0 con 0x7fec48106720 2024-03-04T15:40:17.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.178+0000 7fec2ffff700 1 -- 172.21.15.117:0/3431845729 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec38024410 con 0x7fec48106720 2024-03-04T15:40:17.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.178+0000 7fec4d58a700 1 -- 172.21.15.117:0/3431845729 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec48104c10 con 0x7fec48106720 2024-03-04T15:40:17.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.178+0000 7fec2ffff700 1 -- 172.21.15.117:0/3431845729 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 9) v1 ==== 45940+0+0 (secure 0 0 0) 0x7fec38022020 con 0x7fec48106720 2024-03-04T15:40:17.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.178+0000 7fec4d58a700 1 -- 172.21.15.117:0/3431845729 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec48063410 con 0x7fec48106720 2024-03-04T15:40:17.336 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.178+0000 7fec2ffff700 1 --2- 172.21.15.117:0/3431845729 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7fec30038c50 0x7fec3003b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:17.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.178+0000 7fec2ffff700 1 -- 172.21.15.117:0/3431845729 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fec3804caa0 con 0x7fec48106720 2024-03-04T15:40:17.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.178+0000 7fec467fc700 1 -- 172.21.15.117:0/3431845729 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7fec30038c50 msgr2=0x7fec3003b110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/1454790014 2024-03-04T15:40:17.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.178+0000 7fec467fc700 1 --2- 172.21.15.117:0/3431845729 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7fec30038c50 0x7fec3003b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T15:40:17.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.182+0000 7fec2ffff700 1 -- 172.21.15.117:0/3431845729 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fec3800b3e0 con 0x7fec48106720 2024-03-04T15:40:17.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.382+0000 7fec467fc700 1 -- 172.21.15.117:0/3431845729 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7fec30038c50 msgr2=0x7fec3003b110 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/1454790014 2024-03-04T15:40:17.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.382+0000 7fec467fc700 1 --2- 172.21.15.117:0/3431845729 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7fec30038c50 0x7fec3003b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T15:40:17.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.642+0000 7fec4d58a700 1 -- 172.21.15.117:0/3431845729 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7fec48105160 con 0x7fec48106720 2024-03-04T15:40:17.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.642+0000 7fec2ffff700 1 -- 172.21.15.117:0/3431845729 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7fec380172b0 con 0x7fec48106720 2024-03-04T15:40:17.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.646+0000 7fec4d58a700 1 -- 172.21.15.117:0/3431845729 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7fec30038c50 msgr2=0x7fec3003b110 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T15:40:17.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.646+0000 7fec4d58a700 1 --2- 172.21.15.117:0/3431845729 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7fec30038c50 0x7fec3003b110 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:17.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.646+0000 7fec4d58a700 1 -- 172.21.15.117:0/3431845729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec48106720 msgr2=0x7fec48105fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:17.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.646+0000 7fec4d58a700 1 --2- 172.21.15.117:0/3431845729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec48106720 0x7fec48105fa0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fec38003f60 tx=0x7fec38011ef0 comp rx=0 tx=0).stop 2024-03-04T15:40:17.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.646+0000 7fec4d58a700 1 -- 172.21.15.117:0/3431845729 shutdown_connections 2024-03-04T15:40:17.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.646+0000 7fec4d58a700 1 --2- 172.21.15.117:0/3431845729 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7fec30038c50 0x7fec3003b110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:17.337 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.646+0000 7fec4d58a700 1 --2- 172.21.15.117:0/3431845729 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fec48106720 0x7fec48105fa0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:17.338 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.646+0000 7fec4d58a700 1 -- 172.21.15.117:0/3431845729 >> 172.21.15.117:0/3431845729 conn(0x7fec481000b0 msgr2=0x7fec48100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:17.338 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.646+0000 7fec4d58a700 1 -- 172.21.15.117:0/3431845729 shutdown_connections 2024-03-04T15:40:17.338 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:16.646+0000 7fec4d58a700 1 -- 172.21.15.117:0/3431845729 wait complete. 2024-03-04T15:40:17.338 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for the mgr to restart... 2024-03-04T15:40:17.338 INFO:teuthology.orchestra.run.smithi117.stdout:Waiting for mgr epoch 9... 2024-03-04T15:40:17.634 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:17 smithi117 bash[21734]: debug 2024-03-04T15:40:17.518+0000 7f5841d0f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T15:40:17.910 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:17 smithi117 bash[21734]: debug 2024-03-04T15:40:17.630+0000 7f5841d0f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T15:40:17.911 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:17 smithi117 bash[21734]: debug 2024-03-04T15:40:17.734+0000 7f5841d0f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T15:40:18.208 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:18 smithi117 bash[21734]: debug 2024-03-04T15:40:18.090+0000 7f5841d0f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T15:40:18.208 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:18 smithi117 bash[21734]: debug 2024-03-04T15:40:18.202+0000 7f5841d0f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T15:40:18.571 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:18 smithi117 bash[21734]: debug 2024-03-04T15:40:18.306+0000 7f5841d0f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T15:40:19.071 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:18 smithi117 bash[21734]: debug 2024-03-04T15:40:18.658+0000 7f5841d0f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T15:40:19.071 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:18 smithi117 bash[21734]: debug 2024-03-04T15:40:18.898+0000 7f5841d0f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T15:40:19.545 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:19 smithi117 bash[21734]: debug 2024-03-04T15:40:19.158+0000 7f5841d0f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T15:40:19.546 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:19 smithi117 bash[21734]: debug 2024-03-04T15:40:19.270+0000 7f5841d0f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T15:40:19.821 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:19 smithi117 bash[21734]: debug 2024-03-04T15:40:19.542+0000 7f5841d0f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T15:40:19.821 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:19 smithi117 bash[21734]: debug 2024-03-04T15:40:19.782+0000 7f5841d0f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T15:40:20.555 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:20 smithi117 bash[21734]: debug 2024-03-04T15:40:20.190+0000 7f5841d0f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T15:40:20.555 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:20 smithi117 bash[21734]: debug 2024-03-04T15:40:20.306+0000 7f5841d0f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T15:40:20.555 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:20 smithi117 bash[21734]: debug 2024-03-04T15:40:20.550+0000 7f5841d0f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T15:40:21.993 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:21 smithi117 bash[21734]: debug 2024-03-04T15:40:21.582+0000 7f5841d0f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T15:40:21.993 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:21 smithi117 bash[21734]: debug 2024-03-04T15:40:21.694+0000 7f5841d0f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T15:40:21.993 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:21 smithi117 bash[21734]: debug 2024-03-04T15:40:21.986+0000 7f5841d0f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T15:40:22.321 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:22 smithi117 bash[21734]: debug 2024-03-04T15:40:22.102+0000 7f5841d0f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T15:40:22.821 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:22 smithi117 bash[21734]: debug 2024-03-04T15:40:22.546+0000 7f5841d0f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T15:40:22.821 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:40:22 smithi117 bash[21734]: debug 2024-03-04T15:40:22.666+0000 7f5841d0f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T15:40:22.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:22 smithi117 bash[21480]: cluster 2024-03-04T15:40:22.691889+0000 mon.a (mon.0) 69 : cluster 1 Active manager daemon y restarted 2024-03-04T15:40:22.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:22 smithi117 bash[21480]: cluster 2024-03-04T15:40:22.692350+0000 mon.a (mon.0) 70 : cluster 1 Activating manager daemon y 2024-03-04T15:40:22.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:22 smithi117 bash[21480]: cluster 2024-03-04T15:40:22.695822+0000 mon.a (mon.0) 71 : cluster 0 osdmap e3: 0 total, 0 up, 0 in 2024-03-04T15:40:22.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:22 smithi117 bash[21480]: cluster 2024-03-04T15:40:22.695944+0000 mon.a (mon.0) 72 : cluster 0 mgrmap e10: y(active, starting, since 0.00380241s) 2024-03-04T15:40:22.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:22 smithi117 bash[21480]: audit 2024-03-04T15:40:22.700046+0000 mon.a (mon.0) 73 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:40:22.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:22 smithi117 bash[21480]: audit 2024-03-04T15:40:22.702472+0000 mon.a (mon.0) 74 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-04T15:40:22.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:22 smithi117 bash[21480]: audit 2024-03-04T15:40:22.703593+0000 mon.a (mon.0) 75 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T15:40:22.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:22 smithi117 bash[21480]: audit 2024-03-04T15:40:22.703747+0000 mon.a (mon.0) 76 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T15:40:22.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:22 smithi117 bash[21480]: audit 2024-03-04T15:40:22.703905+0000 mon.a (mon.0) 77 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T15:40:22.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:22 smithi117 bash[21480]: cluster 2024-03-04T15:40:22.727046+0000 mon.a (mon.0) 78 : cluster 1 Manager daemon y is now available 2024-03-04T15:40:24.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:23 smithi117 bash[21480]: audit 2024-03-04T15:40:22.755388+0000 mon.a (mon.0) 79 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:40:24.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:23 smithi117 bash[21480]: audit 2024-03-04T15:40:22.759209+0000 mon.a (mon.0) 80 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:40:24.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:23 smithi117 bash[21480]: audit 2024-03-04T15:40:22.771163+0000 mon.a (mon.0) 81 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-04T15:40:24.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:23 smithi117 bash[21480]: audit 2024-03-04T15:40:22.774607+0000 mon.a (mon.0) 82 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-04T15:40:24.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:23 smithi117 bash[21480]: cephadm 2024-03-04T15:40:23.121776+0000 mgr.y (mgr.14152) 1 : cephadm 1 [04/Mar/2024:15:40:23] ENGINE Bus STARTING 2024-03-04T15:40:24.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:23 smithi117 bash[21480]: cephadm 2024-03-04T15:40:23.223746+0000 mgr.y (mgr.14152) 2 : cephadm 1 [04/Mar/2024:15:40:23] ENGINE Serving on http://172.21.15.117:8765 2024-03-04T15:40:24.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:23 smithi117 bash[21480]: cephadm 2024-03-04T15:40:23.340945+0000 mgr.y (mgr.14152) 3 : cephadm 1 [04/Mar/2024:15:40:23] ENGINE Serving on https://172.21.15.117:7150 2024-03-04T15:40:24.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:23 smithi117 bash[21480]: cephadm 2024-03-04T15:40:23.341296+0000 mgr.y (mgr.14152) 4 : cephadm 1 [04/Mar/2024:15:40:23] ENGINE Bus STARTED 2024-03-04T15:40:24.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:23 smithi117 bash[21480]: cluster 2024-03-04T15:40:23.705956+0000 mon.a (mon.0) 83 : cluster 0 mgrmap e11: y(active, since 1.01379s) 2024-03-04T15:40:24.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout { 2024-03-04T15:40:24.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-03-04T15:40:24.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout "initialized": true 2024-03-04T15:40:24.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout } 2024-03-04T15:40:24.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.094+0000 7f67d9c2b700 1 Processor -- start 2024-03-04T15:40:24.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.094+0000 7f67d9c2b700 1 -- start start 2024-03-04T15:40:24.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.094+0000 7f67d9c2b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67d41066e0 0x7f67d4108ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:24.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.094+0000 7f67d9c2b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d40796f0 con 0x7f67d41066e0 2024-03-04T15:40:24.424 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.094+0000 7f67d37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67d41066e0 0x7f67d4108ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:24.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.094+0000 7f67d37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67d41066e0 0x7f67d4108ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37236/0 (socket says 172.21.15.117:37236) 2024-03-04T15:40:24.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.094+0000 7f67d37fe700 1 -- 172.21.15.117:0/3102556733 learned_addr learned my addr 172.21.15.117:0/3102556733 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:24.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.094+0000 7f67d37fe700 1 -- 172.21.15.117:0/3102556733 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d4109010 con 0x7f67d41066e0 2024-03-04T15:40:24.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.094+0000 7f67d37fe700 1 --2- 172.21.15.117:0/3102556733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67d41066e0 0x7f67d4108ad0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f67c80098a0 tx=0x7f67c8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d224ef62a5300204 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:24.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.094+0000 7f67d27fc700 1 -- 172.21.15.117:0/3102556733 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67c8004160 con 0x7f67d41066e0 2024-03-04T15:40:24.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.094+0000 7f67d27fc700 1 -- 172.21.15.117:0/3102556733 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f67c8004300 con 0x7f67d41066e0 2024-03-04T15:40:24.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.094+0000 7f67d27fc700 1 -- 172.21.15.117:0/3102556733 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67c80118f0 con 0x7f67d41066e0 2024-03-04T15:40:24.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.094+0000 7f67d9c2b700 1 -- 172.21.15.117:0/3102556733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67d41066e0 msgr2=0x7f67d4108ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:24.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.094+0000 7f67d9c2b700 1 --2- 172.21.15.117:0/3102556733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67d41066e0 0x7f67d4108ad0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f67c80098a0 tx=0x7f67c8009bb0 comp rx=0 tx=0).stop 2024-03-04T15:40:24.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.094+0000 7f67d9c2b700 1 -- 172.21.15.117:0/3102556733 shutdown_connections 2024-03-04T15:40:24.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.094+0000 7f67d9c2b700 1 --2- 172.21.15.117:0/3102556733 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67d41066e0 0x7f67d4108ad0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:24.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.094+0000 7f67d9c2b700 1 -- 172.21.15.117:0/3102556733 >> 172.21.15.117:0/3102556733 conn(0x7f67d41000b0 msgr2=0x7f67d4102510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:24.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.094+0000 7f67d9c2b700 1 -- 172.21.15.117:0/3102556733 shutdown_connections 2024-03-04T15:40:24.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.094+0000 7f67d9c2b700 1 -- 172.21.15.117:0/3102556733 wait complete. 2024-03-04T15:40:24.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.098+0000 7f67d9c2b700 1 Processor -- start 2024-03-04T15:40:24.425 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.098+0000 7f67d9c2b700 1 -- start start 2024-03-04T15:40:24.426 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.098+0000 7f67d9c2b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67d41066e0 0x7f67d4105f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:24.426 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.098+0000 7f67d9c2b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d4109f50 con 0x7f67d41066e0 2024-03-04T15:40:24.426 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.098+0000 7f67d37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67d41066e0 0x7f67d4105f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:24.426 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.098+0000 7f67d37fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67d41066e0 0x7f67d4105f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37246/0 (socket says 172.21.15.117:37246) 2024-03-04T15:40:24.426 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.098+0000 7f67d37fe700 1 -- 172.21.15.117:0/3920369818 learned_addr learned my addr 172.21.15.117:0/3920369818 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:24.426 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.098+0000 7f67d37fe700 1 -- 172.21.15.117:0/3920369818 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d41064a0 con 0x7f67d41066e0 2024-03-04T15:40:24.426 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.098+0000 7f67d37fe700 1 --2- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67d41066e0 0x7f67d4105f60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f67c8003f60 tx=0x7f67c80119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:24.426 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.098+0000 7f67d0ff9700 1 -- 172.21.15.117:0/3920369818 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67c8003a40 con 0x7f67d41066e0 2024-03-04T15:40:24.426 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.098+0000 7f67d0ff9700 1 -- 172.21.15.117:0/3920369818 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f67c80248b0 con 0x7f67d41066e0 2024-03-04T15:40:24.426 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.098+0000 7f67d0ff9700 1 -- 172.21.15.117:0/3920369818 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f67c801a750 con 0x7f67d41066e0 2024-03-04T15:40:24.426 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.098+0000 7f67d9c2b700 1 -- 172.21.15.117:0/3920369818 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d4104690 con 0x7f67d41066e0 2024-03-04T15:40:24.426 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.098+0000 7f67d9c2b700 1 -- 172.21.15.117:0/3920369818 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d4104bd0 con 0x7f67d41066e0 2024-03-04T15:40:24.426 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.098+0000 7f67d0ff9700 1 -- 172.21.15.117:0/3920369818 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 9) v1 ==== 45940+0+0 (secure 0 0 0) 0x7f67c8024430 con 0x7f67d41066e0 2024-03-04T15:40:24.426 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.098+0000 7f67d0ff9700 1 --2- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f67bc038c00 0x7f67bc03b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:24.426 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.098+0000 7f67ba7fc700 1 -- 172.21.15.117:0/3920369818 --> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f67b4000d10 con 0x7f67bc038c00 2024-03-04T15:40:24.426 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.098+0000 7f67d0ff9700 1 -- 172.21.15.117:0/3920369818 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f67c804d290 con 0x7f67d41066e0 2024-03-04T15:40:24.427 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.098+0000 7f67d2ffd700 1 -- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f67bc038c00 msgr2=0x7f67bc03b0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/1454790014 2024-03-04T15:40:24.427 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.098+0000 7f67d2ffd700 1 --2- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f67bc038c00 0x7f67bc03b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T15:40:24.427 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.302+0000 7f67d2ffd700 1 -- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f67bc038c00 msgr2=0x7f67bc03b0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/1454790014 2024-03-04T15:40:24.427 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.302+0000 7f67d2ffd700 1 --2- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f67bc038c00 0x7f67bc03b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T15:40:24.427 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.706+0000 7f67d2ffd700 1 -- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f67bc038c00 msgr2=0x7f67bc03b0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/1454790014 2024-03-04T15:40:24.427 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:18.706+0000 7f67d2ffd700 1 --2- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f67bc038c00 0x7f67bc03b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T15:40:24.427 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:19.510+0000 7f67d2ffd700 1 -- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f67bc038c00 msgr2=0x7f67bc03b0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/1454790014 2024-03-04T15:40:24.427 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:19.510+0000 7f67d2ffd700 1 --2- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f67bc038c00 0x7f67bc03b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T15:40:24.427 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:21.114+0000 7f67d2ffd700 1 -- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f67bc038c00 msgr2=0x7f67bc03b0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/1454790014 2024-03-04T15:40:24.427 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:21.114+0000 7f67d2ffd700 1 --2- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f67bc038c00 0x7f67bc03b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-04T15:40:24.427 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:22.690+0000 7f67d0ff9700 1 -- 172.21.15.117:0/3920369818 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mgrmap(e 10) v1 ==== 45707+0+0 (secure 0 0 0) 0x7f67c802d910 con 0x7f67d41066e0 2024-03-04T15:40:24.427 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:22.690+0000 7f67d0ff9700 1 -- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f67bc038c00 msgr2=0x7f67bc03b0c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T15:40:24.427 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:22.690+0000 7f67d0ff9700 1 --2- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f67bc038c00 0x7f67bc03b0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:24.427 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:23.694+0000 7f67d0ff9700 1 -- 172.21.15.117:0/3920369818 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mgrmap(e 11) v1 ==== 45834+0+0 (secure 0 0 0) 0x7f67c804e540 con 0x7f67d41066e0 2024-03-04T15:40:24.427 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:23.694+0000 7f67d0ff9700 1 --2- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f67bc03c500 0x7f67bc03e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:24.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:23.694+0000 7f67d0ff9700 1 -- 172.21.15.117:0/3920369818 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f67c8001d30 con 0x7f67bc03c500 2024-03-04T15:40:24.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:23.698+0000 7f67d2ffd700 1 --2- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f67bc03c500 0x7f67bc03e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:24.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:23.698+0000 7f67d2ffd700 1 --2- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f67bc03c500 0x7f67bc03e8f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f67c4003a10 tx=0x7f67c4009270 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:24.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:23.698+0000 7f67d0ff9700 1 -- 172.21.15.117:0/3920369818 <== mgr.14152 v2:172.21.15.117:6800/2865911351 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+7774 (secure 0 0 0) 0x7f67c8001d30 con 0x7f67bc03c500 2024-03-04T15:40:24.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:23.710+0000 7f67ba7fc700 1 -- 172.21.15.117:0/3920369818 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f67b4002810 con 0x7f67bc03c500 2024-03-04T15:40:24.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:23.714+0000 7f67d0ff9700 1 -- 172.21.15.117:0/3920369818 <== mgr.14152 v2:172.21.15.117:6800/2865911351 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f67b4002810 con 0x7f67bc03c500 2024-03-04T15:40:24.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:23.714+0000 7f67ba7fc700 1 -- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f67bc03c500 msgr2=0x7f67bc03e8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:24.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:23.714+0000 7f67ba7fc700 1 --2- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f67bc03c500 0x7f67bc03e8f0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f67c4003a10 tx=0x7f67c4009270 comp rx=0 tx=0).stop 2024-03-04T15:40:24.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:23.714+0000 7f67ba7fc700 1 -- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67d41066e0 msgr2=0x7f67d4105f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:24.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:23.714+0000 7f67ba7fc700 1 --2- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67d41066e0 0x7f67d4105f60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f67c8003f60 tx=0x7f67c80119b0 comp rx=0 tx=0).stop 2024-03-04T15:40:24.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:23.714+0000 7f67ba7fc700 1 -- 172.21.15.117:0/3920369818 shutdown_connections 2024-03-04T15:40:24.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:23.714+0000 7f67ba7fc700 1 --2- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f67bc03c500 0x7f67bc03e8f0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:24.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:23.714+0000 7f67ba7fc700 1 --2- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f67d41066e0 0x7f67d4105f60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:24.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:23.714+0000 7f67ba7fc700 1 --2- 172.21.15.117:0/3920369818 >> [v2:172.21.15.117:6800/1454790014,v1:172.21.15.117:6801/1454790014] conn(0x7f67bc038c00 0x7f67bc03b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:24.428 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:23.714+0000 7f67ba7fc700 1 -- 172.21.15.117:0/3920369818 >> 172.21.15.117:0/3920369818 conn(0x7f67d41000b0 msgr2=0x7f67d4100b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:24.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:23.714+0000 7f67ba7fc700 1 -- 172.21.15.117:0/3920369818 shutdown_connections 2024-03-04T15:40:24.429 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:23.714+0000 7f67ba7fc700 1 -- 172.21.15.117:0/3920369818 wait complete. 2024-03-04T15:40:24.429 INFO:teuthology.orchestra.run.smithi117.stdout:mgr epoch 9 is available 2024-03-04T15:40:24.429 INFO:teuthology.orchestra.run.smithi117.stdout:Generating a dashboard self-signed certificate... 2024-03-04T15:40:25.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:24 smithi117 bash[21480]: audit 2024-03-04T15:40:23.703676+0000 mgr.y (mgr.14152) 5 : audit 0 from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-03-04T15:40:25.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:24 smithi117 bash[21480]: audit 2024-03-04T15:40:23.716894+0000 mgr.y (mgr.14152) 6 : audit 0 from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-03-04T15:40:26.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:25 smithi117 bash[21480]: cluster 2024-03-04T15:40:24.752473+0000 mon.a (mon.0) 84 : cluster 0 mgrmap e12: y(active, since 2s) 2024-03-04T15:40:26.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:25 smithi117 bash[21480]: audit 2024-03-04T15:40:25.637865+0000 mgr.y (mgr.14152) 7 : audit 0 from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:40:26.344 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-03-04T15:40:26.344 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.254+0000 7fc3ba4a6700 1 Processor -- start 2024-03-04T15:40:26.344 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.254+0000 7fc3ba4a6700 1 -- start start 2024-03-04T15:40:26.344 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3ba4a6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3b410d850 0x7fc3b410dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:26.344 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3ba4a6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b410e240 con 0x7fc3b410d850 2024-03-04T15:40:26.344 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3b3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3b410d850 0x7fc3b410dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:26.344 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3b3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3b410d850 0x7fc3b410dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37334/0 (socket says 172.21.15.117:37334) 2024-03-04T15:40:26.344 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3b3fff700 1 -- 172.21.15.117:0/1979651974 learned_addr learned my addr 172.21.15.117:0/1979651974 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:26.344 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3b3fff700 1 -- 172.21.15.117:0/1979651974 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b410c550 con 0x7fc3b410d850 2024-03-04T15:40:26.344 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3b3fff700 1 --2- 172.21.15.117:0/1979651974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3b410d850 0x7fc3b410dc70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80098a0 tx=0x7fc3a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2ff1b6dc8c9fc98 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:26.345 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3b2ffd700 1 -- 172.21.15.117:0/1979651974 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3a8004160 con 0x7fc3b410d850 2024-03-04T15:40:26.345 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3b2ffd700 1 -- 172.21.15.117:0/1979651974 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fc3a8004300 con 0x7fc3b410d850 2024-03-04T15:40:26.345 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3b2ffd700 1 -- 172.21.15.117:0/1979651974 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3a80118f0 con 0x7fc3b410d850 2024-03-04T15:40:26.345 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3ba4a6700 1 -- 172.21.15.117:0/1979651974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3b410d850 msgr2=0x7fc3b410dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:26.345 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3ba4a6700 1 --2- 172.21.15.117:0/1979651974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3b410d850 0x7fc3b410dc70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc3a80098a0 tx=0x7fc3a8009bb0 comp rx=0 tx=0).stop 2024-03-04T15:40:26.345 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3ba4a6700 1 -- 172.21.15.117:0/1979651974 shutdown_connections 2024-03-04T15:40:26.345 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3ba4a6700 1 --2- 172.21.15.117:0/1979651974 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3b410d850 0x7fc3b410dc70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:26.345 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3ba4a6700 1 -- 172.21.15.117:0/1979651974 >> 172.21.15.117:0/1979651974 conn(0x7fc3b407b220 msgr2=0x7fc3b407d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:26.345 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3ba4a6700 1 -- 172.21.15.117:0/1979651974 shutdown_connections 2024-03-04T15:40:26.345 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3ba4a6700 1 -- 172.21.15.117:0/1979651974 wait complete. 2024-03-04T15:40:26.345 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3ba4a6700 1 Processor -- start 2024-03-04T15:40:26.345 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3ba4a6700 1 -- start start 2024-03-04T15:40:26.345 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3ba4a6700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3b410d850 0x7fc3b419fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:26.345 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3ba4a6700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3b410e3c0 con 0x7fc3b410d850 2024-03-04T15:40:26.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3b3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3b410d850 0x7fc3b419fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:26.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3b3fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3b410d850 0x7fc3b419fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37348/0 (socket says 172.21.15.117:37348) 2024-03-04T15:40:26.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3b3fff700 1 -- 172.21.15.117:0/1917054838 learned_addr learned my addr 172.21.15.117:0/1917054838 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:26.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3b3fff700 1 -- 172.21.15.117:0/1917054838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b41a0530 con 0x7fc3b410d850 2024-03-04T15:40:26.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.258+0000 7fc3b3fff700 1 --2- 172.21.15.117:0/1917054838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3b410d850 0x7fc3b419fff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009870 tx=0x7fc3a8011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:26.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.262+0000 7fc3b17fa700 1 -- 172.21.15.117:0/1917054838 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3a80038b0 con 0x7fc3b410d850 2024-03-04T15:40:26.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.262+0000 7fc3b17fa700 1 -- 172.21.15.117:0/1917054838 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fc3a8003a50 con 0x7fc3b410d850 2024-03-04T15:40:26.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.262+0000 7fc3ba4a6700 1 -- 172.21.15.117:0/1917054838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3b41a07c0 con 0x7fc3b410d850 2024-03-04T15:40:26.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.262+0000 7fc3ba4a6700 1 -- 172.21.15.117:0/1917054838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3b41a34b0 con 0x7fc3b410d850 2024-03-04T15:40:26.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.262+0000 7fc3b17fa700 1 -- 172.21.15.117:0/1917054838 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3a80245a0 con 0x7fc3b410d850 2024-03-04T15:40:26.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.262+0000 7fc3b17fa700 1 -- 172.21.15.117:0/1917054838 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 12) v1 ==== 45940+0+0 (secure 0 0 0) 0x7fc3a8022020 con 0x7fc3b410d850 2024-03-04T15:40:26.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.262+0000 7fc3b17fa700 1 --2- 172.21.15.117:0/1917054838 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fc39c038bf0 0x7fc39c03b0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:26.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.262+0000 7fc3b17fa700 1 -- 172.21.15.117:0/1917054838 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fc3a804cd40 con 0x7fc3b410d850 2024-03-04T15:40:26.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.262+0000 7fc3ba4a6700 1 -- 172.21.15.117:0/1917054838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3a0005150 con 0x7fc3b410d850 2024-03-04T15:40:26.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.262+0000 7fc3b37fe700 1 --2- 172.21.15.117:0/1917054838 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fc39c038bf0 0x7fc39c03b0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:26.346 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.262+0000 7fc3b37fe700 1 --2- 172.21.15.117:0/1917054838 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fc39c038bf0 0x7fc39c03b0b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4006fd0 tx=0x7fc3a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:26.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.266+0000 7fc3b17fa700 1 -- 172.21.15.117:0/1917054838 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc3a8021d60 con 0x7fc3b410d850 2024-03-04T15:40:26.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.634+0000 7fc3ba4a6700 1 -- 172.21.15.117:0/1917054838 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7fc3a0000bf0 con 0x7fc39c038bf0 2024-03-04T15:40:26.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.766+0000 7fc3b17fa700 1 -- 172.21.15.117:0/1917054838 <== mgr.14152 v2:172.21.15.117:6800/2865911351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7fc3a0000bf0 con 0x7fc39c038bf0 2024-03-04T15:40:26.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.770+0000 7fc3ba4a6700 1 -- 172.21.15.117:0/1917054838 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fc39c038bf0 msgr2=0x7fc39c03b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:26.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.770+0000 7fc3ba4a6700 1 --2- 172.21.15.117:0/1917054838 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fc39c038bf0 0x7fc39c03b0b0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fc3a4006fd0 tx=0x7fc3a4006e40 comp rx=0 tx=0).stop 2024-03-04T15:40:26.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.770+0000 7fc3ba4a6700 1 -- 172.21.15.117:0/1917054838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3b410d850 msgr2=0x7fc3b419fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:26.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.770+0000 7fc3ba4a6700 1 --2- 172.21.15.117:0/1917054838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3b410d850 0x7fc3b419fff0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc3a8009870 tx=0x7fc3a8011ef0 comp rx=0 tx=0).stop 2024-03-04T15:40:26.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.770+0000 7fc3ba4a6700 1 -- 172.21.15.117:0/1917054838 shutdown_connections 2024-03-04T15:40:26.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.770+0000 7fc3ba4a6700 1 --2- 172.21.15.117:0/1917054838 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fc39c038bf0 0x7fc39c03b0b0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:26.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.770+0000 7fc3ba4a6700 1 --2- 172.21.15.117:0/1917054838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc3b410d850 0x7fc3b419fff0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:26.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.770+0000 7fc3ba4a6700 1 -- 172.21.15.117:0/1917054838 >> 172.21.15.117:0/1917054838 conn(0x7fc3b407b220 msgr2=0x7fc3b407bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:26.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.770+0000 7fc3ba4a6700 1 -- 172.21.15.117:0/1917054838 shutdown_connections 2024-03-04T15:40:26.347 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:25.770+0000 7fc3ba4a6700 1 -- 172.21.15.117:0/1917054838 wait complete. 2024-03-04T15:40:26.347 INFO:teuthology.orchestra.run.smithi117.stdout:Creating initial admin user... 2024-03-04T15:40:27.038 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:26 smithi117 bash[21480]: audit 2024-03-04T15:40:25.766640+0000 mon.a (mon.0) 85 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:40:27.038 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:26 smithi117 bash[21480]: audit 2024-03-04T15:40:25.771941+0000 mon.a (mon.0) 86 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:40:28.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:27 smithi117 bash[21480]: audit 2024-03-04T15:40:27.614509+0000 mgr.y (mgr.14152) 8 : audit 0 from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:40:28.499 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$tKnPIsZeRArPK0VAdZ.FBuCzbHcur4HEYC7mWTLbS3sFDd94/rdmy", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1709566827, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-03-04T15:40:28.499 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.214+0000 7f7a6cc1a700 1 Processor -- start 2024-03-04T15:40:28.499 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.214+0000 7f7a6cc1a700 1 -- start start 2024-03-04T15:40:28.499 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.214+0000 7f7a6cc1a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68107e70 0x7f7a68108290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:28.499 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.214+0000 7f7a6cc1a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a681087d0 con 0x7f7a68107e70 2024-03-04T15:40:28.499 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.214+0000 7f7a6659c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68107e70 0x7f7a68108290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:28.499 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.214+0000 7f7a6659c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68107e70 0x7f7a68108290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51710/0 (socket says 172.21.15.117:51710) 2024-03-04T15:40:28.499 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.214+0000 7f7a6659c700 1 -- 172.21.15.117:0/2796227463 learned_addr learned my addr 172.21.15.117:0/2796227463 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:28.499 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.214+0000 7f7a6659c700 1 -- 172.21.15.117:0/2796227463 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a68108fd0 con 0x7f7a68107e70 2024-03-04T15:40:28.499 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.214+0000 7f7a6659c700 1 --2- 172.21.15.117:0/2796227463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68107e70 0x7f7a68108290 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7a580098a0 tx=0x7f7a58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d4607412bf8423 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:28.500 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.214+0000 7f7a6559a700 1 -- 172.21.15.117:0/2796227463 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a58004160 con 0x7f7a68107e70 2024-03-04T15:40:28.500 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6559a700 1 -- 172.21.15.117:0/2796227463 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f7a58004300 con 0x7f7a68107e70 2024-03-04T15:40:28.500 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6559a700 1 -- 172.21.15.117:0/2796227463 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a580118f0 con 0x7f7a68107e70 2024-03-04T15:40:28.500 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6cc1a700 1 -- 172.21.15.117:0/2796227463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68107e70 msgr2=0x7f7a68108290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:28.500 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6cc1a700 1 --2- 172.21.15.117:0/2796227463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68107e70 0x7f7a68108290 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7a580098a0 tx=0x7f7a58009bb0 comp rx=0 tx=0).stop 2024-03-04T15:40:28.500 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6cc1a700 1 -- 172.21.15.117:0/2796227463 shutdown_connections 2024-03-04T15:40:28.500 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6cc1a700 1 --2- 172.21.15.117:0/2796227463 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68107e70 0x7f7a68108290 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:28.500 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6cc1a700 1 -- 172.21.15.117:0/2796227463 >> 172.21.15.117:0/2796227463 conn(0x7f7a6807b240 msgr2=0x7f7a68105d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:28.500 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6cc1a700 1 -- 172.21.15.117:0/2796227463 shutdown_connections 2024-03-04T15:40:28.500 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6cc1a700 1 -- 172.21.15.117:0/2796227463 wait complete. 2024-03-04T15:40:28.500 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6cc1a700 1 Processor -- start 2024-03-04T15:40:28.500 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6cc1a700 1 -- start start 2024-03-04T15:40:28.500 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6cc1a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68107e70 0x7f7a6819bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:28.500 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6cc1a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6810c290 con 0x7f7a68107e70 2024-03-04T15:40:28.500 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6659c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68107e70 0x7f7a6819bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:28.501 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6659c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68107e70 0x7f7a6819bbd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51712/0 (socket says 172.21.15.117:51712) 2024-03-04T15:40:28.501 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6659c700 1 -- 172.21.15.117:0/3959743780 learned_addr learned my addr 172.21.15.117:0/3959743780 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:28.501 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6659c700 1 -- 172.21.15.117:0/3959743780 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a6819c110 con 0x7f7a68107e70 2024-03-04T15:40:28.501 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6659c700 1 --2- 172.21.15.117:0/3959743780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68107e70 0x7f7a6819bbd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7a58011970 tx=0x7f7a58011a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:28.501 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a5f7fe700 1 -- 172.21.15.117:0/3959743780 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a5801a430 con 0x7f7a68107e70 2024-03-04T15:40:28.501 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6cc1a700 1 -- 172.21.15.117:0/3959743780 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a6819c3a0 con 0x7f7a68107e70 2024-03-04T15:40:28.501 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a5f7fe700 1 -- 172.21.15.117:0/3959743780 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f7a5801aa50 con 0x7f7a68107e70 2024-03-04T15:40:28.501 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a5f7fe700 1 -- 172.21.15.117:0/3959743780 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a58019910 con 0x7f7a68107e70 2024-03-04T15:40:28.501 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6cc1a700 1 -- 172.21.15.117:0/3959743780 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a6819f090 con 0x7f7a68107e70 2024-03-04T15:40:28.501 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a5f7fe700 1 -- 172.21.15.117:0/3959743780 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 12) v1 ==== 45940+0+0 (secure 0 0 0) 0x7f7a58022020 con 0x7f7a68107e70 2024-03-04T15:40:28.501 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a5f7fe700 1 --2- 172.21.15.117:0/3959743780 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f7a54038c00 0x7f7a5403b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:28.501 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a5f7fe700 1 -- 172.21.15.117:0/3959743780 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7a5804d3e0 con 0x7f7a68107e70 2024-03-04T15:40:28.501 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a6cc1a700 1 -- 172.21.15.117:0/3959743780 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6810c410 con 0x7f7a68107e70 2024-03-04T15:40:28.501 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.218+0000 7f7a65d9b700 1 --2- 172.21.15.117:0/3959743780 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f7a54038c00 0x7f7a5403b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:28.501 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.222+0000 7f7a65d9b700 1 --2- 172.21.15.117:0/3959743780 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f7a54038c00 0x7f7a5403b0c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7a50006fd0 tx=0x7f7a50006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:28.502 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.226+0000 7f7a5f7fe700 1 -- 172.21.15.117:0/3959743780 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7a5802c8c0 con 0x7f7a68107e70 2024-03-04T15:40:28.502 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.610+0000 7f7a6cc1a700 1 -- 172.21.15.117:0/3959743780 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7a68105a80 con 0x7f7a54038c00 2024-03-04T15:40:28.502 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.850+0000 7f7a5f7fe700 1 -- 172.21.15.117:0/3959743780 <== mgr.14152 v2:172.21.15.117:6800/2865911351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f7a68105a80 con 0x7f7a54038c00 2024-03-04T15:40:28.502 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.854+0000 7f7a6cc1a700 1 -- 172.21.15.117:0/3959743780 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f7a54038c00 msgr2=0x7f7a5403b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:28.502 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.854+0000 7f7a6cc1a700 1 --2- 172.21.15.117:0/3959743780 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f7a54038c00 0x7f7a5403b0c0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7a50006fd0 tx=0x7f7a50006e40 comp rx=0 tx=0).stop 2024-03-04T15:40:28.502 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.854+0000 7f7a6cc1a700 1 -- 172.21.15.117:0/3959743780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68107e70 msgr2=0x7f7a6819bbd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:28.502 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.854+0000 7f7a6cc1a700 1 --2- 172.21.15.117:0/3959743780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68107e70 0x7f7a6819bbd0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7a58011970 tx=0x7f7a58011a50 comp rx=0 tx=0).stop 2024-03-04T15:40:28.502 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.854+0000 7f7a6cc1a700 1 -- 172.21.15.117:0/3959743780 shutdown_connections 2024-03-04T15:40:28.502 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.854+0000 7f7a6cc1a700 1 --2- 172.21.15.117:0/3959743780 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f7a54038c00 0x7f7a5403b0c0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:28.502 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.854+0000 7f7a6cc1a700 1 --2- 172.21.15.117:0/3959743780 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7a68107e70 0x7f7a6819bbd0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:28.502 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.854+0000 7f7a6cc1a700 1 -- 172.21.15.117:0/3959743780 >> 172.21.15.117:0/3959743780 conn(0x7f7a6807b240 msgr2=0x7f7a68105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:28.502 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.854+0000 7f7a6cc1a700 1 -- 172.21.15.117:0/3959743780 shutdown_connections 2024-03-04T15:40:28.502 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:27.854+0000 7f7a6cc1a700 1 -- 172.21.15.117:0/3959743780 wait complete. 2024-03-04T15:40:28.502 INFO:teuthology.orchestra.run.smithi117.stdout:Fetching dashboard port number... 2024-03-04T15:40:29.109 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:28 smithi117 bash[21480]: audit 2024-03-04T15:40:27.855380+0000 mon.a (mon.0) 87 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:40:30.002 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:29 smithi117 bash[21480]: cluster 2024-03-04T15:40:28.865429+0000 mon.a (mon.0) 88 : cluster 0 mgrmap e13: y(active, since 6s) 2024-03-04T15:40:30.002 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:29 smithi117 bash[21480]: audit 2024-03-04T15:40:29.616335+0000 mon.a (mon.0) 89 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:40:30.002 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:29 smithi117 bash[21480]: audit 2024-03-04T15:40:29.717492+0000 mon.a (mon.0) 90 : audit 0 from='client.? 172.21.15.117:0/811056911' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-03-04T15:40:30.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stdout 8443 2024-03-04T15:40:30.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.314+0000 7fa9e6ce8700 1 Processor -- start 2024-03-04T15:40:30.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.314+0000 7fa9e6ce8700 1 -- start start 2024-03-04T15:40:30.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.314+0000 7fa9e6ce8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e0105c60 0x7fa9e0106080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:30.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.314+0000 7fa9e6ce8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e0106650 con 0x7fa9e0105c60 2024-03-04T15:40:30.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.314+0000 7fa9e4a84700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e0105c60 0x7fa9e0106080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:30.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.314+0000 7fa9e4a84700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e0105c60 0x7fa9e0106080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51724/0 (socket says 172.21.15.117:51724) 2024-03-04T15:40:30.411 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.314+0000 7fa9e4a84700 1 -- 172.21.15.117:0/637618441 learned_addr learned my addr 172.21.15.117:0/637618441 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:30.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9e4a84700 1 -- 172.21.15.117:0/637618441 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e0106e60 con 0x7fa9e0105c60 2024-03-04T15:40:30.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9e4a84700 1 --2- 172.21.15.117:0/637618441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e0105c60 0x7fa9e0106080 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0098a0 tx=0x7fa9cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3944fe42dcfc9 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:30.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9df7fe700 1 -- 172.21.15.117:0/637618441 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9cc004160 con 0x7fa9e0105c60 2024-03-04T15:40:30.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9df7fe700 1 -- 172.21.15.117:0/637618441 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fa9cc004300 con 0x7fa9e0105c60 2024-03-04T15:40:30.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9df7fe700 1 -- 172.21.15.117:0/637618441 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9cc0118f0 con 0x7fa9e0105c60 2024-03-04T15:40:30.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9e6ce8700 1 -- 172.21.15.117:0/637618441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e0105c60 msgr2=0x7fa9e0106080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:30.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9e6ce8700 1 --2- 172.21.15.117:0/637618441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e0105c60 0x7fa9e0106080 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc0098a0 tx=0x7fa9cc009bb0 comp rx=0 tx=0).stop 2024-03-04T15:40:30.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9e6ce8700 1 -- 172.21.15.117:0/637618441 shutdown_connections 2024-03-04T15:40:30.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9e6ce8700 1 --2- 172.21.15.117:0/637618441 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e0105c60 0x7fa9e0106080 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:30.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9e6ce8700 1 -- 172.21.15.117:0/637618441 >> 172.21.15.117:0/637618441 conn(0x7fa9e01011c0 msgr2=0x7fa9e0103640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:30.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9e6ce8700 1 -- 172.21.15.117:0/637618441 shutdown_connections 2024-03-04T15:40:30.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9e6ce8700 1 -- 172.21.15.117:0/637618441 wait complete. 2024-03-04T15:40:30.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9e6ce8700 1 Processor -- start 2024-03-04T15:40:30.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9e6ce8700 1 -- start start 2024-03-04T15:40:30.412 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9e6ce8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e0105c60 0x7fa9e0199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:30.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9e6ce8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9e010a090 con 0x7fa9e0105c60 2024-03-04T15:40:30.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9e4a84700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e0105c60 0x7fa9e0199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:30.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9e4a84700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e0105c60 0x7fa9e0199a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51736/0 (socket says 172.21.15.117:51736) 2024-03-04T15:40:30.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9e4a84700 1 -- 172.21.15.117:0/811056911 learned_addr learned my addr 172.21.15.117:0/811056911 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:30.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9e4a84700 1 -- 172.21.15.117:0/811056911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9e0199fb0 con 0x7fa9e0105c60 2024-03-04T15:40:30.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9e4a84700 1 --2- 172.21.15.117:0/811056911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e0105c60 0x7fa9e0199a70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009870 tx=0x7fa9cc011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:30.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9ddffb700 1 -- 172.21.15.117:0/811056911 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9cc0038b0 con 0x7fa9e0105c60 2024-03-04T15:40:30.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9ddffb700 1 -- 172.21.15.117:0/811056911 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fa9cc003a50 con 0x7fa9e0105c60 2024-03-04T15:40:30.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9e6ce8700 1 -- 172.21.15.117:0/811056911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9e019a240 con 0x7fa9e0105c60 2024-03-04T15:40:30.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9ddffb700 1 -- 172.21.15.117:0/811056911 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa9cc024410 con 0x7fa9e0105c60 2024-03-04T15:40:30.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.318+0000 7fa9e6ce8700 1 -- 172.21.15.117:0/811056911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9e019cf30 con 0x7fa9e0105c60 2024-03-04T15:40:30.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.322+0000 7fa9ddffb700 1 -- 172.21.15.117:0/811056911 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fa9cc022020 con 0x7fa9e0105c60 2024-03-04T15:40:30.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.322+0000 7fa9e6ce8700 1 -- 172.21.15.117:0/811056911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9e0063410 con 0x7fa9e0105c60 2024-03-04T15:40:30.413 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.322+0000 7fa9ddffb700 1 --2- 172.21.15.117:0/811056911 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fa9d0038c90 0x7fa9d003b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:30.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.322+0000 7fa9ddffb700 1 -- 172.21.15.117:0/811056911 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa9cc04ce50 con 0x7fa9e0105c60 2024-03-04T15:40:30.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.322+0000 7fa9dffff700 1 --2- 172.21.15.117:0/811056911 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fa9d0038c90 0x7fa9d003b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:30.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.322+0000 7fa9dffff700 1 --2- 172.21.15.117:0/811056911 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fa9d0038c90 0x7fa9d003b150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4006fd0 tx=0x7fa9d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:30.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.326+0000 7fa9ddffb700 1 -- 172.21.15.117:0/811056911 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa9cc01a7b0 con 0x7fa9e0105c60 2024-03-04T15:40:30.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.710+0000 7fa9e6ce8700 1 -- 172.21.15.117:0/811056911 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fa9e019d310 con 0x7fa9e0105c60 2024-03-04T15:40:30.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.714+0000 7fa9ddffb700 1 -- 172.21.15.117:0/811056911 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fa9cc02dc90 con 0x7fa9e0105c60 2024-03-04T15:40:30.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.714+0000 7fa9e6ce8700 1 -- 172.21.15.117:0/811056911 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fa9d0038c90 msgr2=0x7fa9d003b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:30.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.714+0000 7fa9e6ce8700 1 --2- 172.21.15.117:0/811056911 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fa9d0038c90 0x7fa9d003b150 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa9d4006fd0 tx=0x7fa9d4006e40 comp rx=0 tx=0).stop 2024-03-04T15:40:30.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.714+0000 7fa9e6ce8700 1 -- 172.21.15.117:0/811056911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e0105c60 msgr2=0x7fa9e0199a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:30.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.714+0000 7fa9e6ce8700 1 --2- 172.21.15.117:0/811056911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e0105c60 0x7fa9e0199a70 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa9cc009870 tx=0x7fa9cc011ef0 comp rx=0 tx=0).stop 2024-03-04T15:40:30.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.714+0000 7fa9e6ce8700 1 -- 172.21.15.117:0/811056911 shutdown_connections 2024-03-04T15:40:30.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.714+0000 7fa9e6ce8700 1 --2- 172.21.15.117:0/811056911 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fa9d0038c90 0x7fa9d003b150 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:30.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.714+0000 7fa9e6ce8700 1 --2- 172.21.15.117:0/811056911 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa9e0105c60 0x7fa9e0199a70 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:30.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.714+0000 7fa9e6ce8700 1 -- 172.21.15.117:0/811056911 >> 172.21.15.117:0/811056911 conn(0x7fa9e01011c0 msgr2=0x7fa9e0101d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:30.414 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.714+0000 7fa9e6ce8700 1 -- 172.21.15.117:0/811056911 shutdown_connections 2024-03-04T15:40:30.415 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:29.714+0000 7fa9e6ce8700 1 -- 172.21.15.117:0/811056911 wait complete. 2024-03-04T15:40:30.415 INFO:teuthology.orchestra.run.smithi117.stdout:firewalld does not appear to be present 2024-03-04T15:40:30.415 INFO:teuthology.orchestra.run.smithi117.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-03-04T15:40:30.415 INFO:teuthology.orchestra.run.smithi117.stdout:Ceph Dashboard is now available at: 2024-03-04T15:40:30.415 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-04T15:40:30.415 INFO:teuthology.orchestra.run.smithi117.stdout: URL: https://smithi117.front.sepia.ceph.com:8443/ 2024-03-04T15:40:30.415 INFO:teuthology.orchestra.run.smithi117.stdout: User: admin 2024-03-04T15:40:30.415 INFO:teuthology.orchestra.run.smithi117.stdout: Password: du5yg5ewuw 2024-03-04T15:40:30.415 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-04T15:40:30.415 INFO:teuthology.orchestra.run.smithi117.stdout:Saving cluster configuration to /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/config directory 2024-03-04T15:40:30.415 INFO:teuthology.orchestra.run.smithi117.stdout:Enabling autotune for osd_memory_target 2024-03-04T15:40:32.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:31 smithi117 bash[21480]: audit 2024-03-04T15:40:30.854230+0000 mon.a (mon.0) 91 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:40:32.243 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.326+0000 7f020fcca700 1 Processor -- start 2024-03-04T15:40:32.243 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.326+0000 7f020fcca700 1 -- start start 2024-03-04T15:40:32.243 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.326+0000 7f020fcca700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0208106730 0x7f0208108b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:32.243 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.326+0000 7f020fcca700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02080796d0 con 0x7f0208106730 2024-03-04T15:40:32.243 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020da66700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0208106730 0x7f0208108b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:32.244 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020da66700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0208106730 0x7f0208108b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51740/0 (socket says 172.21.15.117:51740) 2024-03-04T15:40:32.244 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020da66700 1 -- 172.21.15.117:0/3627286514 learned_addr learned my addr 172.21.15.117:0/3627286514 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:32.244 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020da66700 1 -- 172.21.15.117:0/3627286514 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0208109060 con 0x7f0208106730 2024-03-04T15:40:32.244 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020da66700 1 --2- 172.21.15.117:0/3627286514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0208106730 0x7f0208108b20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f02040098a0 tx=0x7f0204009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a833a3a70bae7cff server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:32.244 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020ca64700 1 -- 172.21.15.117:0/3627286514 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0204004160 con 0x7f0208106730 2024-03-04T15:40:32.244 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020ca64700 1 -- 172.21.15.117:0/3627286514 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f0204004300 con 0x7f0208106730 2024-03-04T15:40:32.244 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020ca64700 1 -- 172.21.15.117:0/3627286514 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02040118f0 con 0x7f0208106730 2024-03-04T15:40:32.244 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020fcca700 1 -- 172.21.15.117:0/3627286514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0208106730 msgr2=0x7f0208108b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:32.244 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020fcca700 1 --2- 172.21.15.117:0/3627286514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0208106730 0x7f0208108b20 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f02040098a0 tx=0x7f0204009bb0 comp rx=0 tx=0).stop 2024-03-04T15:40:32.244 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020fcca700 1 -- 172.21.15.117:0/3627286514 shutdown_connections 2024-03-04T15:40:32.244 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020fcca700 1 --2- 172.21.15.117:0/3627286514 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0208106730 0x7f0208108b20 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:32.244 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020fcca700 1 -- 172.21.15.117:0/3627286514 >> 172.21.15.117:0/3627286514 conn(0x7f02081000c0 msgr2=0x7f0208102540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:32.244 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020fcca700 1 -- 172.21.15.117:0/3627286514 shutdown_connections 2024-03-04T15:40:32.244 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020fcca700 1 -- 172.21.15.117:0/3627286514 wait complete. 2024-03-04T15:40:32.244 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020fcca700 1 Processor -- start 2024-03-04T15:40:32.245 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020fcca700 1 -- start start 2024-03-04T15:40:32.245 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020fcca700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0208106730 0x7f0208199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:32.245 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020fcca700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0208109f50 con 0x7f0208106730 2024-03-04T15:40:32.245 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020da66700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0208106730 0x7f0208199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:32.245 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020da66700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0208106730 0x7f0208199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51742/0 (socket says 172.21.15.117:51742) 2024-03-04T15:40:32.245 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020da66700 1 -- 172.21.15.117:0/4236965538 learned_addr learned my addr 172.21.15.117:0/4236965538 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:32.245 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020da66700 1 -- 172.21.15.117:0/4236965538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0208199ed0 con 0x7f0208106730 2024-03-04T15:40:32.245 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020da66700 1 --2- 172.21.15.117:0/4236965538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0208106730 0x7f0208199990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0204003f60 tx=0x7f0204011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:32.245 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f01feffd700 1 -- 172.21.15.117:0/4236965538 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f02040038b0 con 0x7f0208106730 2024-03-04T15:40:32.245 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f01feffd700 1 -- 172.21.15.117:0/4236965538 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f0204003a50 con 0x7f0208106730 2024-03-04T15:40:32.245 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f01feffd700 1 -- 172.21.15.117:0/4236965538 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0204024410 con 0x7f0208106730 2024-03-04T15:40:32.245 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020fcca700 1 -- 172.21.15.117:0/4236965538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020819a160 con 0x7f0208106730 2024-03-04T15:40:32.245 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.330+0000 7f020fcca700 1 -- 172.21.15.117:0/4236965538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020819ce50 con 0x7f0208106730 2024-03-04T15:40:32.245 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.334+0000 7f01feffd700 1 -- 172.21.15.117:0/4236965538 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f0204022020 con 0x7f0208106730 2024-03-04T15:40:32.245 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.334+0000 7f020fcca700 1 -- 172.21.15.117:0/4236965538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0208063410 con 0x7f0208106730 2024-03-04T15:40:32.245 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.334+0000 7f01feffd700 1 --2- 172.21.15.117:0/4236965538 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f01f4038cf0 0x7f01f403b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:32.246 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.334+0000 7f01feffd700 1 -- 172.21.15.117:0/4236965538 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f020404cc50 con 0x7f0208106730 2024-03-04T15:40:32.246 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.334+0000 7f020d265700 1 --2- 172.21.15.117:0/4236965538 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f01f4038cf0 0x7f01f403b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:32.246 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.334+0000 7f020d265700 1 --2- 172.21.15.117:0/4236965538 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f01f4038cf0 0x7f01f403b1b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f01f8006fd0 tx=0x7f01f8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:32.246 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.338+0000 7f01feffd700 1 -- 172.21.15.117:0/4236965538 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f020401a9f0 con 0x7f0208106730 2024-03-04T15:40:32.246 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.682+0000 7f020fcca700 1 -- 172.21.15.117:0/4236965538 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f020819d2a0 con 0x7f0208106730 2024-03-04T15:40:32.246 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.686+0000 7f01feffd700 1 -- 172.21.15.117:0/4236965538 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f020402d3f0 con 0x7f0208106730 2024-03-04T15:40:32.246 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.686+0000 7f020fcca700 1 -- 172.21.15.117:0/4236965538 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f01f4038cf0 msgr2=0x7f01f403b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:32.246 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.686+0000 7f020fcca700 1 --2- 172.21.15.117:0/4236965538 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f01f4038cf0 0x7f01f403b1b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f01f8006fd0 tx=0x7f01f8006e40 comp rx=0 tx=0).stop 2024-03-04T15:40:32.246 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.686+0000 7f020fcca700 1 -- 172.21.15.117:0/4236965538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0208106730 msgr2=0x7f0208199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:32.246 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.686+0000 7f020fcca700 1 --2- 172.21.15.117:0/4236965538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0208106730 0x7f0208199990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f0204003f60 tx=0x7f0204011ef0 comp rx=0 tx=0).stop 2024-03-04T15:40:32.246 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.686+0000 7f020fcca700 1 -- 172.21.15.117:0/4236965538 shutdown_connections 2024-03-04T15:40:32.246 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.686+0000 7f020fcca700 1 --2- 172.21.15.117:0/4236965538 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f01f4038cf0 0x7f01f403b1b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:32.246 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.686+0000 7f020fcca700 1 --2- 172.21.15.117:0/4236965538 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0208106730 0x7f0208199990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:32.246 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.686+0000 7f020fcca700 1 -- 172.21.15.117:0/4236965538 >> 172.21.15.117:0/4236965538 conn(0x7f02081000c0 msgr2=0x7f0208100ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:32.247 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.686+0000 7f020fcca700 1 -- 172.21.15.117:0/4236965538 shutdown_connections 2024-03-04T15:40:32.247 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:31.686+0000 7f020fcca700 1 -- 172.21.15.117:0/4236965538 wait complete. 2024-03-04T15:40:34.253 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.089+0000 7f07ae273700 1 Processor -- start 2024-03-04T15:40:34.253 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.089+0000 7f07ae273700 1 -- start start 2024-03-04T15:40:34.253 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.089+0000 7f07ae273700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a810d850 0x7f07a810dc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:34.253 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.089+0000 7f07ae273700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a810e240 con 0x7f07a810d850 2024-03-04T15:40:34.253 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.089+0000 7f07a7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a810d850 0x7f07a810dc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:34.253 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.089+0000 7f07a7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a810d850 0x7f07a810dc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51750/0 (socket says 172.21.15.117:51750) 2024-03-04T15:40:34.253 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.089+0000 7f07a7fff700 1 -- 172.21.15.117:0/2011559081 learned_addr learned my addr 172.21.15.117:0/2011559081 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:34.253 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.089+0000 7f07a7fff700 1 -- 172.21.15.117:0/2011559081 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a810c550 con 0x7f07a810d850 2024-03-04T15:40:34.253 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.089+0000 7f07a7fff700 1 --2- 172.21.15.117:0/2011559081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a810d850 0x7f07a810dc70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f07980098a0 tx=0x7f0798009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff2922f1cef6250 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:34.253 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07a6ffd700 1 -- 172.21.15.117:0/2011559081 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0798004160 con 0x7f07a810d850 2024-03-04T15:40:34.253 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07a6ffd700 1 -- 172.21.15.117:0/2011559081 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f0798004300 con 0x7f07a810d850 2024-03-04T15:40:34.253 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07a6ffd700 1 -- 172.21.15.117:0/2011559081 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07980118f0 con 0x7f07a810d850 2024-03-04T15:40:34.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07ae273700 1 -- 172.21.15.117:0/2011559081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a810d850 msgr2=0x7f07a810dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:34.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07ae273700 1 --2- 172.21.15.117:0/2011559081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a810d850 0x7f07a810dc70 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f07980098a0 tx=0x7f0798009bb0 comp rx=0 tx=0).stop 2024-03-04T15:40:34.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07ae273700 1 -- 172.21.15.117:0/2011559081 shutdown_connections 2024-03-04T15:40:34.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07ae273700 1 --2- 172.21.15.117:0/2011559081 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a810d850 0x7f07a810dc70 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:34.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07ae273700 1 -- 172.21.15.117:0/2011559081 >> 172.21.15.117:0/2011559081 conn(0x7f07a807b220 msgr2=0x7f07a807d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:34.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07ae273700 1 -- 172.21.15.117:0/2011559081 shutdown_connections 2024-03-04T15:40:34.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07ae273700 1 -- 172.21.15.117:0/2011559081 wait complete. 2024-03-04T15:40:34.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07ae273700 1 Processor -- start 2024-03-04T15:40:34.258 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07ae273700 1 -- start start 2024-03-04T15:40:34.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07ae273700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a810d850 0x7f07a81a0020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:34.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07ae273700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07a810e3c0 con 0x7f07a810d850 2024-03-04T15:40:34.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07a7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a810d850 0x7f07a81a0020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:34.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07a7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a810d850 0x7f07a81a0020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51764/0 (socket says 172.21.15.117:51764) 2024-03-04T15:40:34.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07a7fff700 1 -- 172.21.15.117:0/4203262705 learned_addr learned my addr 172.21.15.117:0/4203262705 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:34.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07a7fff700 1 -- 172.21.15.117:0/4203262705 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07a81a0560 con 0x7f07a810d850 2024-03-04T15:40:34.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07a7fff700 1 --2- 172.21.15.117:0/4203262705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a810d850 0x7f07a81a0020 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0798009870 tx=0x7f0798011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:34.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07a57fa700 1 -- 172.21.15.117:0/4203262705 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07980038b0 con 0x7f07a810d850 2024-03-04T15:40:34.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07a57fa700 1 -- 172.21.15.117:0/4203262705 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f0798003a50 con 0x7f07a810d850 2024-03-04T15:40:34.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07ae273700 1 -- 172.21.15.117:0/4203262705 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07a81a07f0 con 0x7f07a810d850 2024-03-04T15:40:34.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07a57fa700 1 -- 172.21.15.117:0/4203262705 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f07980245a0 con 0x7f07a810d850 2024-03-04T15:40:34.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07ae273700 1 -- 172.21.15.117:0/4203262705 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07a81a34e0 con 0x7f07a810d850 2024-03-04T15:40:34.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07a57fa700 1 -- 172.21.15.117:0/4203262705 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f0798022020 con 0x7f07a810d850 2024-03-04T15:40:34.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07ae273700 1 -- 172.21.15.117:0/4203262705 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07a8063410 con 0x7f07a810d850 2024-03-04T15:40:34.259 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.093+0000 7f07a57fa700 1 --2- 172.21.15.117:0/4203262705 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f0794038cf0 0x7f079403b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:34.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.097+0000 7f07a57fa700 1 -- 172.21.15.117:0/4203262705 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f079804cf80 con 0x7f07a810d850 2024-03-04T15:40:34.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.097+0000 7f07a77fe700 1 --2- 172.21.15.117:0/4203262705 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f0794038cf0 0x7f079403b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:34.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.097+0000 7f07a77fe700 1 --2- 172.21.15.117:0/4203262705 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f0794038cf0 0x7f079403b1b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0790006fd0 tx=0x7f0790006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:34.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.101+0000 7f07a57fa700 1 -- 172.21.15.117:0/4203262705 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f079802cdf0 con 0x7f07a810d850 2024-03-04T15:40:34.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.661+0000 7f07ae273700 1 -- 172.21.15.117:0/4203262705 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f07a81a38c0 con 0x7f07a810d850 2024-03-04T15:40:34.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.665+0000 7f07a57fa700 1 -- 172.21.15.117:0/4203262705 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v26)=0 set mgr/dashboard/cluster/status v26) v1 ==== 153+0+0 (secure 0 0 0) 0x7f079800b5a0 con 0x7f07a810d850 2024-03-04T15:40:34.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.665+0000 7f07ae273700 1 -- 172.21.15.117:0/4203262705 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f0794038cf0 msgr2=0x7f079403b1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:34.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.665+0000 7f07ae273700 1 --2- 172.21.15.117:0/4203262705 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f0794038cf0 0x7f079403b1b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f0790006fd0 tx=0x7f0790006e40 comp rx=0 tx=0).stop 2024-03-04T15:40:34.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.665+0000 7f07ae273700 1 -- 172.21.15.117:0/4203262705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a810d850 msgr2=0x7f07a81a0020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:34.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.665+0000 7f07ae273700 1 --2- 172.21.15.117:0/4203262705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a810d850 0x7f07a81a0020 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0798009870 tx=0x7f0798011ef0 comp rx=0 tx=0).stop 2024-03-04T15:40:34.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.665+0000 7f07ae273700 1 -- 172.21.15.117:0/4203262705 shutdown_connections 2024-03-04T15:40:34.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.665+0000 7f07ae273700 1 --2- 172.21.15.117:0/4203262705 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f0794038cf0 0x7f079403b1b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:34.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.665+0000 7f07ae273700 1 --2- 172.21.15.117:0/4203262705 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f07a810d850 0x7f07a81a0020 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:34.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.665+0000 7f07ae273700 1 -- 172.21.15.117:0/4203262705 >> 172.21.15.117:0/4203262705 conn(0x7f07a807b220 msgr2=0x7f07a807bd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:34.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.665+0000 7f07ae273700 1 -- 172.21.15.117:0/4203262705 shutdown_connections 2024-03-04T15:40:34.260 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr 2024-03-04T15:40:33.665+0000 7f07ae273700 1 -- 172.21.15.117:0/4203262705 wait complete. 2024-03-04T15:40:34.261 INFO:teuthology.orchestra.run.smithi117.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-03-04T15:40:34.261 INFO:teuthology.orchestra.run.smithi117.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-03-04T15:40:34.261 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-04T15:40:34.261 INFO:teuthology.orchestra.run.smithi117.stdout: sudo /usr/sbin/cephadm shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-03-04T15:40:34.261 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-04T15:40:34.261 INFO:teuthology.orchestra.run.smithi117.stdout:Or, if you are only running a single cluster on this host: 2024-03-04T15:40:34.261 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-04T15:40:34.261 INFO:teuthology.orchestra.run.smithi117.stdout: sudo /usr/sbin/cephadm shell 2024-03-04T15:40:34.261 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-04T15:40:34.261 INFO:teuthology.orchestra.run.smithi117.stdout:Please consider enabling telemetry to help improve Ceph: 2024-03-04T15:40:34.261 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-04T15:40:34.261 INFO:teuthology.orchestra.run.smithi117.stdout: ceph telemetry on 2024-03-04T15:40:34.261 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-04T15:40:34.261 INFO:teuthology.orchestra.run.smithi117.stdout:For more information see: 2024-03-04T15:40:34.261 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-04T15:40:34.261 INFO:teuthology.orchestra.run.smithi117.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-03-04T15:40:34.262 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-04T15:40:34.262 INFO:teuthology.orchestra.run.smithi117.stdout:Bootstrap complete. 2024-03-04T15:40:34.315 INFO:tasks.cephadm:Fetching config... 2024-03-04T15:40:34.315 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-04T15:40:34.315 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-03-04T15:40:34.321 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-03-04T15:40:34.321 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-04T15:40:34.321 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-03-04T15:40:34.369 INFO:tasks.cephadm:Fetching mon keyring... 2024-03-04T15:40:34.369 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-04T15:40:34.370 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/keyring of=/dev/stdout 2024-03-04T15:40:34.425 INFO:tasks.cephadm:Fetching pub ssh key... 2024-03-04T15:40:34.426 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-04T15:40:34.426 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-03-04T15:40:34.474 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-03-04T15:40:34.474 DEBUG:teuthology.orchestra.run.smithi117:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCopvF0TLtYEXloWdTRgEm4DhVivvyJ9bLfq/J1+mogDpYEXcGWJFOzuzwukvJbgN+cZpGs7o/ysA1AEPO6OmWRM91bwy/q7IMtr4TmhNJiYA+ZqVzg6krnmjWS7lwNsDwpdAcW2GRI+pHPBzTuiLk9x/X7bkRcZnkP6xzJr4g1zqDcgscwUPezy6g1q+AZPVOiiz4I7nl/y3X+btops+NpyukVl4UgwcLU4oWHc0zTyKQ30QPwubyEc6tfYSEQkwn1oTQAhgnZSd8yqKhcekyEtI+hJYjFlyuGQEfh5+wiZTINPQFJMSvbquADnt3NVgrJft7vDMcLrNcBSkhjVwso4ArO+lPqlO+TIfJLFLzRM6GU9pdVXj1YQ4SsGUCbJ0XMSNGklD7oIK9CCYCmQ4RVlX2ZJ2Zqwij1ufoaXSh29UcN42dJeYoJhFrGO/nDw5Ha/xLpxBkfaYkHTZCsiX8zm492nG4JUZniKr/93aLz1FY5TqYDugl0l8j2mVBmjF8= ceph-e40b5504-da3c-11ee-95c6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-04T15:40:34.538 INFO:teuthology.orchestra.run.smithi117.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCopvF0TLtYEXloWdTRgEm4DhVivvyJ9bLfq/J1+mogDpYEXcGWJFOzuzwukvJbgN+cZpGs7o/ysA1AEPO6OmWRM91bwy/q7IMtr4TmhNJiYA+ZqVzg6krnmjWS7lwNsDwpdAcW2GRI+pHPBzTuiLk9x/X7bkRcZnkP6xzJr4g1zqDcgscwUPezy6g1q+AZPVOiiz4I7nl/y3X+btops+NpyukVl4UgwcLU4oWHc0zTyKQ30QPwubyEc6tfYSEQkwn1oTQAhgnZSd8yqKhcekyEtI+hJYjFlyuGQEfh5+wiZTINPQFJMSvbquADnt3NVgrJft7vDMcLrNcBSkhjVwso4ArO+lPqlO+TIfJLFLzRM6GU9pdVXj1YQ4SsGUCbJ0XMSNGklD7oIK9CCYCmQ4RVlX2ZJ2Zqwij1ufoaXSh29UcN42dJeYoJhFrGO/nDw5Ha/xLpxBkfaYkHTZCsiX8zm492nG4JUZniKr/93aLz1FY5TqYDugl0l8j2mVBmjF8= ceph-e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T15:40:34.547 DEBUG:teuthology.orchestra.run.smithi151:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCopvF0TLtYEXloWdTRgEm4DhVivvyJ9bLfq/J1+mogDpYEXcGWJFOzuzwukvJbgN+cZpGs7o/ysA1AEPO6OmWRM91bwy/q7IMtr4TmhNJiYA+ZqVzg6krnmjWS7lwNsDwpdAcW2GRI+pHPBzTuiLk9x/X7bkRcZnkP6xzJr4g1zqDcgscwUPezy6g1q+AZPVOiiz4I7nl/y3X+btops+NpyukVl4UgwcLU4oWHc0zTyKQ30QPwubyEc6tfYSEQkwn1oTQAhgnZSd8yqKhcekyEtI+hJYjFlyuGQEfh5+wiZTINPQFJMSvbquADnt3NVgrJft7vDMcLrNcBSkhjVwso4ArO+lPqlO+TIfJLFLzRM6GU9pdVXj1YQ4SsGUCbJ0XMSNGklD7oIK9CCYCmQ4RVlX2ZJ2Zqwij1ufoaXSh29UcN42dJeYoJhFrGO/nDw5Ha/xLpxBkfaYkHTZCsiX8zm492nG4JUZniKr/93aLz1FY5TqYDugl0l8j2mVBmjF8= ceph-e40b5504-da3c-11ee-95c6-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-03-04T15:40:34.573 INFO:teuthology.orchestra.run.smithi151.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCopvF0TLtYEXloWdTRgEm4DhVivvyJ9bLfq/J1+mogDpYEXcGWJFOzuzwukvJbgN+cZpGs7o/ysA1AEPO6OmWRM91bwy/q7IMtr4TmhNJiYA+ZqVzg6krnmjWS7lwNsDwpdAcW2GRI+pHPBzTuiLk9x/X7bkRcZnkP6xzJr4g1zqDcgscwUPezy6g1q+AZPVOiiz4I7nl/y3X+btops+NpyukVl4UgwcLU4oWHc0zTyKQ30QPwubyEc6tfYSEQkwn1oTQAhgnZSd8yqKhcekyEtI+hJYjFlyuGQEfh5+wiZTINPQFJMSvbquADnt3NVgrJft7vDMcLrNcBSkhjVwso4ArO+lPqlO+TIfJLFLzRM6GU9pdVXj1YQ4SsGUCbJ0XMSNGklD7oIK9CCYCmQ4RVlX2ZJ2Zqwij1ufoaXSh29UcN42dJeYoJhFrGO/nDw5Ha/xLpxBkfaYkHTZCsiX8zm492nG4JUZniKr/93aLz1FY5TqYDugl0l8j2mVBmjF8= ceph-e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T15:40:34.582 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-03-04T15:40:35.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:34 smithi117 bash[21480]: audit 2024-03-04T15:40:33.669051+0000 mon.a (mon.0) 92 : audit 1 from='client.? 172.21.15.117:0/4203262705' entity='client.admin' 2024-03-04T15:40:39.246 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:40:42.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.245+0000 7fd4eb2d0700 1 -- 172.21.15.117:0/1589328178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e4102320 msgr2=0x7fd4e4104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:42.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.245+0000 7fd4eb2d0700 1 --2- 172.21.15.117:0/1589328178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e4102320 0x7fd4e4104710 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0009910 tx=0x7fd4e0009c20 comp rx=0 tx=0).stop 2024-03-04T15:40:42.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.245+0000 7fd4eb2d0700 1 -- 172.21.15.117:0/1589328178 shutdown_connections 2024-03-04T15:40:42.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.245+0000 7fd4eb2d0700 1 --2- 172.21.15.117:0/1589328178 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e4102320 0x7fd4e4104710 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:42.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.245+0000 7fd4eb2d0700 1 -- 172.21.15.117:0/1589328178 >> 172.21.15.117:0/1589328178 conn(0x7fd4e40fbcd0 msgr2=0x7fd4e40fe0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:42.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.245+0000 7fd4eb2d0700 1 -- 172.21.15.117:0/1589328178 shutdown_connections 2024-03-04T15:40:42.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.245+0000 7fd4eb2d0700 1 -- 172.21.15.117:0/1589328178 wait complete. 2024-03-04T15:40:42.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.245+0000 7fd4eb2d0700 1 Processor -- start 2024-03-04T15:40:42.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.245+0000 7fd4eb2d0700 1 -- start start 2024-03-04T15:40:42.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.245+0000 7fd4eb2d0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e4102320 0x7fd4e4196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:42.253 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.245+0000 7fd4eb2d0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4e4106d10 con 0x7fd4e4102320 2024-03-04T15:40:42.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.245+0000 7fd4e906c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e4102320 0x7fd4e4196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:42.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.245+0000 7fd4e906c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e4102320 0x7fd4e4196420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:54378/0 (socket says 172.21.15.117:54378) 2024-03-04T15:40:42.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.245+0000 7fd4e906c700 1 -- 172.21.15.117:0/3077044732 learned_addr learned my addr 172.21.15.117:0/3077044732 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:42.254 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.249+0000 7fd4e906c700 1 -- 172.21.15.117:0/3077044732 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4e4196960 con 0x7fd4e4102320 2024-03-04T15:40:42.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.249+0000 7fd4e906c700 1 --2- 172.21.15.117:0/3077044732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e4102320 0x7fd4e4196420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0005710 tx=0x7fd4e00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:42.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.249+0000 7fd4da7fc700 1 -- 172.21.15.117:0/3077044732 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4e0027070 con 0x7fd4e4102320 2024-03-04T15:40:42.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.249+0000 7fd4eb2d0700 1 -- 172.21.15.117:0/3077044732 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4e4196bf0 con 0x7fd4e4102320 2024-03-04T15:40:42.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.249+0000 7fd4da7fc700 1 -- 172.21.15.117:0/3077044732 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd4e000f460 con 0x7fd4e4102320 2024-03-04T15:40:42.255 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.249+0000 7fd4da7fc700 1 -- 172.21.15.117:0/3077044732 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd4e0017440 con 0x7fd4e4102320 2024-03-04T15:40:42.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.249+0000 7fd4eb2d0700 1 -- 172.21.15.117:0/3077044732 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4e41998e0 con 0x7fd4e4102320 2024-03-04T15:40:42.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.249+0000 7fd4da7fc700 1 -- 172.21.15.117:0/3077044732 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fd4e002a070 con 0x7fd4e4102320 2024-03-04T15:40:42.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.249+0000 7fd4da7fc700 1 --2- 172.21.15.117:0/3077044732 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fd4d00388a0 0x7fd4d003ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:42.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.249+0000 7fd4da7fc700 1 -- 172.21.15.117:0/3077044732 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd4e0052660 con 0x7fd4e4102320 2024-03-04T15:40:42.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.249+0000 7fd4e886b700 1 --2- 172.21.15.117:0/3077044732 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fd4d00388a0 0x7fd4d003ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:42.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.249+0000 7fd4eb2d0700 1 -- 172.21.15.117:0/3077044732 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4e4107490 con 0x7fd4e4102320 2024-03-04T15:40:42.256 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.249+0000 7fd4e886b700 1 --2- 172.21.15.117:0/3077044732 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fd4d00388a0 0x7fd4d003ad60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4006fd0 tx=0x7fd4d4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:42.260 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.253+0000 7fd4da7fc700 1 -- 172.21.15.117:0/3077044732 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd4e0013070 con 0x7fd4e4102320 2024-03-04T15:40:42.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.453+0000 7fd4eb2d0700 1 -- 172.21.15.117:0/3077044732 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fd4e4063410 con 0x7fd4e4102320 2024-03-04T15:40:42.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.461+0000 7fd4da7fc700 1 -- 172.21.15.117:0/3077044732 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fd4e000f610 con 0x7fd4e4102320 2024-03-04T15:40:42.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.465+0000 7fd4eb2d0700 1 -- 172.21.15.117:0/3077044732 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fd4d00388a0 msgr2=0x7fd4d003ad60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:42.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.465+0000 7fd4eb2d0700 1 --2- 172.21.15.117:0/3077044732 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fd4d00388a0 0x7fd4d003ad60 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd4d4006fd0 tx=0x7fd4d4006e40 comp rx=0 tx=0).stop 2024-03-04T15:40:42.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.465+0000 7fd4eb2d0700 1 -- 172.21.15.117:0/3077044732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e4102320 msgr2=0x7fd4e4196420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:42.469 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.465+0000 7fd4eb2d0700 1 --2- 172.21.15.117:0/3077044732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e4102320 0x7fd4e4196420 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd4e0005710 tx=0x7fd4e00057f0 comp rx=0 tx=0).stop 2024-03-04T15:40:42.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.465+0000 7fd4eb2d0700 1 -- 172.21.15.117:0/3077044732 shutdown_connections 2024-03-04T15:40:42.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.465+0000 7fd4eb2d0700 1 --2- 172.21.15.117:0/3077044732 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fd4d00388a0 0x7fd4d003ad60 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:42.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.465+0000 7fd4eb2d0700 1 --2- 172.21.15.117:0/3077044732 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd4e4102320 0x7fd4e4196420 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:42.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.465+0000 7fd4eb2d0700 1 -- 172.21.15.117:0/3077044732 >> 172.21.15.117:0/3077044732 conn(0x7fd4e40fbcd0 msgr2=0x7fd4e40fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:42.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.465+0000 7fd4eb2d0700 1 -- 172.21.15.117:0/3077044732 shutdown_connections 2024-03-04T15:40:42.470 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:42.465+0000 7fd4eb2d0700 1 -- 172.21.15.117:0/3077044732 wait complete. 2024-03-04T15:40:43.172 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-03-04T15:40:43.172 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-03-04T15:40:43.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:43 smithi117 bash[21480]: audit 2024-03-04T15:40:42.053179+0000 mon.a (mon.0) 93 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:40:43.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:43 smithi117 bash[21480]: audit 2024-03-04T15:40:42.059655+0000 mon.a (mon.0) 94 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:40:43.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:43 smithi117 bash[21480]: audit 2024-03-04T15:40:42.061362+0000 mon.a (mon.0) 95 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi117", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:40:43.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:43 smithi117 bash[21480]: audit 2024-03-04T15:40:42.070740+0000 mon.a (mon.0) 96 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:40:43.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:43 smithi117 bash[21480]: audit 2024-03-04T15:40:42.104497+0000 mon.a (mon.0) 97 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:40:43.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:43 smithi117 bash[21480]: audit 2024-03-04T15:40:42.110067+0000 mon.a (mon.0) 98 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:40:43.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:43 smithi117 bash[21480]: audit 2024-03-04T15:40:42.466329+0000 mon.a (mon.0) 99 : audit 1 from='client.? 172.21.15.117:0/3077044732' entity='client.admin' 2024-03-04T15:40:43.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:43 smithi117 bash[21480]: audit 2024-03-04T15:40:42.491998+0000 mon.a (mon.0) 100 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:40:43.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:43 smithi117 bash[21480]: audit 2024-03-04T15:40:42.496843+0000 mon.a (mon.0) 101 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:40:44.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:44 smithi117 bash[21480]: cluster 2024-03-04T15:40:42.705352+0000 mgr.y (mgr.14152) 9 : cluster 0 pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:40:46.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:46 smithi117 bash[21480]: cluster 2024-03-04T15:40:44.705754+0000 mgr.y (mgr.14152) 10 : cluster 0 pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:40:47.881 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:40:48.688 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:48 smithi117 bash[21480]: cluster 2024-03-04T15:40:46.706221+0000 mgr.y (mgr.14152) 11 : cluster 0 pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:40:50.345 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31c0b67700 1 -- 172.21.15.117:0/3308874776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31bc103a30 msgr2=0x7f31bc103e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:50.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31c0b67700 1 --2- 172.21.15.117:0/3308874776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31bc103a30 0x7f31bc103e50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f31ac009910 tx=0x7f31ac009c20 comp rx=0 tx=0).stop 2024-03-04T15:40:50.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31c0b67700 1 -- 172.21.15.117:0/3308874776 shutdown_connections 2024-03-04T15:40:50.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31c0b67700 1 --2- 172.21.15.117:0/3308874776 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31bc103a30 0x7f31bc103e50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:50.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31c0b67700 1 -- 172.21.15.117:0/3308874776 >> 172.21.15.117:0/3308874776 conn(0x7f31bc0fefd0 msgr2=0x7f31bc101410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:50.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31c0b67700 1 -- 172.21.15.117:0/3308874776 shutdown_connections 2024-03-04T15:40:50.346 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31c0b67700 1 -- 172.21.15.117:0/3308874776 wait complete. 2024-03-04T15:40:50.347 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31c0b67700 1 Processor -- start 2024-03-04T15:40:50.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31c0b67700 1 -- start start 2024-03-04T15:40:50.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31c0b67700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31bc103a30 0x7f31bc1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:50.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31c0b67700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f31bc108f30 con 0x7f31bc103a30 2024-03-04T15:40:50.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31ba59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31bc103a30 0x7f31bc1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:50.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31ba59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31bc103a30 0x7f31bc1986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:45278/0 (socket says 172.21.15.117:45278) 2024-03-04T15:40:50.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31ba59c700 1 -- 172.21.15.117:0/4055347998 learned_addr learned my addr 172.21.15.117:0/4055347998 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:50.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31ba59c700 1 -- 172.21.15.117:0/4055347998 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31bc198c00 con 0x7f31bc103a30 2024-03-04T15:40:50.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31ba59c700 1 --2- 172.21.15.117:0/4055347998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31bc103a30 0x7f31bc1986c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f31ac005950 tx=0x7f31ac0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:50.348 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31b37fe700 1 -- 172.21.15.117:0/4055347998 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31ac027070 con 0x7f31bc103a30 2024-03-04T15:40:50.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31b37fe700 1 -- 172.21.15.117:0/4055347998 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f31ac00f460 con 0x7f31bc103a30 2024-03-04T15:40:50.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31c0b67700 1 -- 172.21.15.117:0/4055347998 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f31bc198e90 con 0x7f31bc103a30 2024-03-04T15:40:50.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31b37fe700 1 -- 172.21.15.117:0/4055347998 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f31ac017440 con 0x7f31bc103a30 2024-03-04T15:40:50.349 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31c0b67700 1 -- 172.21.15.117:0/4055347998 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f31bc19bb80 con 0x7f31bc103a30 2024-03-04T15:40:50.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31c0b67700 1 -- 172.21.15.117:0/4055347998 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f31bc1096b0 con 0x7f31bc103a30 2024-03-04T15:40:50.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31b37fe700 1 -- 172.21.15.117:0/4055347998 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f31ac02a070 con 0x7f31bc103a30 2024-03-04T15:40:50.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31b37fe700 1 --2- 172.21.15.117:0/4055347998 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f31a8038ca0 0x7f31a803b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:50.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31b37fe700 1 -- 172.21.15.117:0/4055347998 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f31ac04cd40 con 0x7f31bc103a30 2024-03-04T15:40:50.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.341+0000 7f31b3fff700 1 --2- 172.21.15.117:0/4055347998 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f31a8038ca0 0x7f31a803b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:50.350 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.345+0000 7f31b3fff700 1 --2- 172.21.15.117:0/4055347998 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f31a8038ca0 0x7f31a803b160 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f31a4006fd0 tx=0x7f31a4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:50.353 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.349+0000 7f31b37fe700 1 -- 172.21.15.117:0/4055347998 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f31ac053080 con 0x7f31bc103a30 2024-03-04T15:40:50.568 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.561+0000 7f31c0b67700 1 -- 172.21.15.117:0/4055347998 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7f31bc062220 con 0x7f31a8038ca0 2024-03-04T15:40:50.577 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.573+0000 7f31b37fe700 1 -- 172.21.15.117:0/4055347998 <== mgr.14152 v2:172.21.15.117:6800/2865911351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f31bc062220 con 0x7f31a8038ca0 2024-03-04T15:40:50.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.573+0000 7f31c0b67700 1 -- 172.21.15.117:0/4055347998 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f31a8038ca0 msgr2=0x7f31a803b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:50.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.573+0000 7f31c0b67700 1 --2- 172.21.15.117:0/4055347998 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f31a8038ca0 0x7f31a803b160 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f31a4006fd0 tx=0x7f31a4006e40 comp rx=0 tx=0).stop 2024-03-04T15:40:50.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.573+0000 7f31c0b67700 1 -- 172.21.15.117:0/4055347998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31bc103a30 msgr2=0x7f31bc1986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:50.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.573+0000 7f31c0b67700 1 --2- 172.21.15.117:0/4055347998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31bc103a30 0x7f31bc1986c0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f31ac005950 tx=0x7f31ac0057c0 comp rx=0 tx=0).stop 2024-03-04T15:40:50.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.573+0000 7f31c0b67700 1 -- 172.21.15.117:0/4055347998 shutdown_connections 2024-03-04T15:40:50.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.573+0000 7f31c0b67700 1 --2- 172.21.15.117:0/4055347998 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f31a8038ca0 0x7f31a803b160 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:50.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.573+0000 7f31c0b67700 1 --2- 172.21.15.117:0/4055347998 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f31bc103a30 0x7f31bc1986c0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:50.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.573+0000 7f31c0b67700 1 -- 172.21.15.117:0/4055347998 >> 172.21.15.117:0/4055347998 conn(0x7f31bc0fefd0 msgr2=0x7f31bc0ffae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:50.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.573+0000 7f31c0b67700 1 -- 172.21.15.117:0/4055347998 shutdown_connections 2024-03-04T15:40:50.580 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:50.573+0000 7f31c0b67700 1 -- 172.21.15.117:0/4055347998 wait complete. 2024-03-04T15:40:50.813 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:50 smithi117 bash[21480]: cluster 2024-03-04T15:40:48.706667+0000 mgr.y (mgr.14152) 12 : cluster 0 pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:40:51.249 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi151 2024-03-04T15:40:51.250 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-04T15:40:51.250 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/etc/ceph/ceph.conf 2024-03-04T15:40:51.256 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-04T15:40:51.256 DEBUG:teuthology.orchestra.run.smithi151:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-04T15:40:51.303 INFO:tasks.cephadm:Adding host smithi151 to orchestrator... 2024-03-04T15:40:51.304 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph orch host add smithi151 2024-03-04T15:40:52.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:51 smithi117 bash[21480]: audit 2024-03-04T15:40:50.569429+0000 mgr.y (mgr.14152) 13 : audit 0 from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:40:52.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:51 smithi117 bash[21480]: audit 2024-03-04T15:40:50.576565+0000 mon.a (mon.0) 102 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:40:52.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:51 smithi117 bash[21480]: audit 2024-03-04T15:40:50.596900+0000 mon.a (mon.0) 103 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:40:52.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:51 smithi117 bash[21480]: audit 2024-03-04T15:40:50.598327+0000 mon.a (mon.0) 104 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:40:52.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:51 smithi117 bash[21480]: audit 2024-03-04T15:40:50.599011+0000 mon.a (mon.0) 105 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:40:52.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:51 smithi117 bash[21480]: cephadm 2024-03-04T15:40:50.599948+0000 mgr.y (mgr.14152) 14 : cephadm 1 Updating smithi117:/etc/ceph/ceph.conf 2024-03-04T15:40:52.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:51 smithi117 bash[21480]: audit 2024-03-04T15:40:51.060291+0000 mon.a (mon.0) 106 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:40:52.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:51 smithi117 bash[21480]: audit 2024-03-04T15:40:51.064012+0000 mon.a (mon.0) 107 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:40:52.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:51 smithi117 bash[21480]: audit 2024-03-04T15:40:51.067084+0000 mon.a (mon.0) 108 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:40:53.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:52 smithi117 bash[21480]: cluster 2024-03-04T15:40:50.707042+0000 mgr.y (mgr.14152) 15 : cluster 0 pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:40:53.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:52 smithi117 bash[21480]: cephadm 2024-03-04T15:40:50.713962+0000 mgr.y (mgr.14152) 16 : cephadm 1 Updating smithi117:/var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T15:40:53.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:52 smithi117 bash[21480]: cephadm 2024-03-04T15:40:50.828338+0000 mgr.y (mgr.14152) 17 : cephadm 1 Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-03-04T15:40:53.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:52 smithi117 bash[21480]: cephadm 2024-03-04T15:40:50.942433+0000 mgr.y (mgr.14152) 18 : cephadm 1 Updating smithi117:/var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T15:40:55.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:54 smithi117 bash[21480]: cluster 2024-03-04T15:40:52.707433+0000 mgr.y (mgr.14152) 19 : cluster 0 pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:40:56.015 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:40:56.857 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:56 smithi117 bash[21480]: cluster 2024-03-04T15:40:54.707844+0000 mgr.y (mgr.14152) 20 : cluster 0 pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:40:58.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb3723e3700 1 -- 172.21.15.117:0/3080577989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb36c1066e0 msgr2=0x7fb36c106ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:40:58.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb3723e3700 1 --2- 172.21.15.117:0/3080577989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb36c1066e0 0x7fb36c106ac0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb35c009910 tx=0x7fb35c009c20 comp rx=0 tx=0).stop 2024-03-04T15:40:58.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb3723e3700 1 -- 172.21.15.117:0/3080577989 shutdown_connections 2024-03-04T15:40:58.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb3723e3700 1 --2- 172.21.15.117:0/3080577989 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb36c1066e0 0x7fb36c106ac0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:40:58.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb3723e3700 1 -- 172.21.15.117:0/3080577989 >> 172.21.15.117:0/3080577989 conn(0x7fb36c0fc210 msgr2=0x7fb36c0fe630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:40:58.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb3723e3700 1 -- 172.21.15.117:0/3080577989 shutdown_connections 2024-03-04T15:40:58.431 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb3723e3700 1 -- 172.21.15.117:0/3080577989 wait complete. 2024-03-04T15:40:58.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb3723e3700 1 Processor -- start 2024-03-04T15:40:58.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb3723e3700 1 -- start start 2024-03-04T15:40:58.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb3723e3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb36c1066e0 0x7fb36c19c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:58.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb3723e3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb36c10a1a0 con 0x7fb36c1066e0 2024-03-04T15:40:58.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb36bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb36c1066e0 0x7fb36c19c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:58.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb36bfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb36c1066e0 0x7fb36c19c5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:37066/0 (socket says 172.21.15.117:37066) 2024-03-04T15:40:58.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb36bfff700 1 -- 172.21.15.117:0/4038192565 learned_addr learned my addr 172.21.15.117:0/4038192565 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:40:58.433 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb36bfff700 1 -- 172.21.15.117:0/4038192565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb36c19cae0 con 0x7fb36c1066e0 2024-03-04T15:40:58.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb36bfff700 1 --2- 172.21.15.117:0/4038192565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb36c1066e0 0x7fb36c19c5a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb35c006010 tx=0x7fb35c00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:58.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb3697fa700 1 -- 172.21.15.117:0/4038192565 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb35c027070 con 0x7fb36c1066e0 2024-03-04T15:40:58.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb3697fa700 1 -- 172.21.15.117:0/4038192565 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb35c022470 con 0x7fb36c1066e0 2024-03-04T15:40:58.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb3697fa700 1 -- 172.21.15.117:0/4038192565 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb35c017440 con 0x7fb36c1066e0 2024-03-04T15:40:58.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb3723e3700 1 -- 172.21.15.117:0/4038192565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb36c19cdd0 con 0x7fb36c1066e0 2024-03-04T15:40:58.434 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.425+0000 7fb3723e3700 1 -- 172.21.15.117:0/4038192565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb36c196840 con 0x7fb36c1066e0 2024-03-04T15:40:58.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.429+0000 7fb3723e3700 1 -- 172.21.15.117:0/4038192565 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb36c101e20 con 0x7fb36c1066e0 2024-03-04T15:40:58.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.429+0000 7fb3697fa700 1 -- 172.21.15.117:0/4038192565 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fb35c02a070 con 0x7fb36c1066e0 2024-03-04T15:40:58.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.429+0000 7fb3697fa700 1 --2- 172.21.15.117:0/4038192565 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fb354038ca0 0x7fb35403b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:40:58.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.429+0000 7fb3697fa700 1 -- 172.21.15.117:0/4038192565 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb35c01e030 con 0x7fb36c1066e0 2024-03-04T15:40:58.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.429+0000 7fb36b7fe700 1 --2- 172.21.15.117:0/4038192565 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fb354038ca0 0x7fb35403b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:40:58.435 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.429+0000 7fb36b7fe700 1 --2- 172.21.15.117:0/4038192565 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fb354038ca0 0x7fb35403b160 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb360006fd0 tx=0x7fb360006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:40:58.439 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.433+0000 7fb3697fa700 1 -- 172.21.15.117:0/4038192565 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fb35c013070 con 0x7fb36c1066e0 2024-03-04T15:40:58.647 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:40:58.641+0000 7fb3723e3700 1 -- 172.21.15.117:0/4038192565 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}) v1 -- 0x7fb36c063410 con 0x7fb354038ca0 2024-03-04T15:40:59.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:58 smithi117 bash[21480]: cluster 2024-03-04T15:40:56.708241+0000 mgr.y (mgr.14152) 21 : cluster 0 pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:00.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:40:59 smithi117 bash[21480]: audit 2024-03-04T15:40:58.648006+0000 mgr.y (mgr.14152) 22 : audit 0 from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi151", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:41:01.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:00 smithi117 bash[21480]: cluster 2024-03-04T15:40:58.708618+0000 mgr.y (mgr.14152) 23 : cluster 0 pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:01.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:00 smithi117 bash[21480]: cephadm 2024-03-04T15:40:59.317965+0000 mgr.y (mgr.14152) 24 : cephadm 1 Deploying cephadm binary to smithi151 2024-03-04T15:41:03.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:02 smithi117 bash[21480]: cluster 2024-03-04T15:41:00.709058+0000 mgr.y (mgr.14152) 25 : cluster 0 pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:04.375 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:04.369+0000 7fb3697fa700 1 -- 172.21.15.117:0/4038192565 <== mgr.14152 v2:172.21.15.117:6800/2865911351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fb36c063410 con 0x7fb354038ca0 2024-03-04T15:41:04.375 INFO:teuthology.orchestra.run.smithi117.stdout:Added host 'smithi151' with addr '172.21.15.151' 2024-03-04T15:41:04.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:04.373+0000 7fb3723e3700 1 -- 172.21.15.117:0/4038192565 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fb354038ca0 msgr2=0x7fb35403b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:04.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:04.373+0000 7fb3723e3700 1 --2- 172.21.15.117:0/4038192565 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fb354038ca0 0x7fb35403b160 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fb360006fd0 tx=0x7fb360006e40 comp rx=0 tx=0).stop 2024-03-04T15:41:04.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:04.373+0000 7fb3723e3700 1 -- 172.21.15.117:0/4038192565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb36c1066e0 msgr2=0x7fb36c19c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:04.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:04.373+0000 7fb3723e3700 1 --2- 172.21.15.117:0/4038192565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb36c1066e0 0x7fb36c19c5a0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb35c006010 tx=0x7fb35c00f740 comp rx=0 tx=0).stop 2024-03-04T15:41:04.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:04.373+0000 7fb3723e3700 1 -- 172.21.15.117:0/4038192565 shutdown_connections 2024-03-04T15:41:04.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:04.373+0000 7fb3723e3700 1 --2- 172.21.15.117:0/4038192565 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fb354038ca0 0x7fb35403b160 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:04.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:04.373+0000 7fb3723e3700 1 --2- 172.21.15.117:0/4038192565 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb36c1066e0 0x7fb36c19c5a0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:04.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:04.373+0000 7fb3723e3700 1 -- 172.21.15.117:0/4038192565 >> 172.21.15.117:0/4038192565 conn(0x7fb36c0fc210 msgr2=0x7fb36c108790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:41:04.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:04.373+0000 7fb3723e3700 1 -- 172.21.15.117:0/4038192565 shutdown_connections 2024-03-04T15:41:04.378 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:04.373+0000 7fb3723e3700 1 -- 172.21.15.117:0/4038192565 wait complete. 2024-03-04T15:41:04.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:04 smithi117 bash[21480]: cluster 2024-03-04T15:41:02.709432+0000 mgr.y (mgr.14152) 26 : cluster 0 pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:04.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:04 smithi117 bash[21480]: audit 2024-03-04T15:41:04.373025+0000 mon.a (mon.0) 109 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:04.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:04 smithi117 bash[21480]: audit 2024-03-04T15:41:04.409952+0000 mon.a (mon.0) 110 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:41:05.133 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph orch host ls --format=json 2024-03-04T15:41:06.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:05 smithi117 bash[21480]: cephadm 2024-03-04T15:41:04.373971+0000 mgr.y (mgr.14152) 27 : cephadm 1 Added host smithi151 2024-03-04T15:41:06.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:05 smithi117 bash[21480]: audit 2024-03-04T15:41:04.832781+0000 mon.a (mon.0) 111 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:07.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:06 smithi117 bash[21480]: cluster 2024-03-04T15:41:04.709864+0000 mgr.y (mgr.14152) 28 : cluster 0 pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:07.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:06 smithi117 bash[21480]: audit 2024-03-04T15:41:06.357834+0000 mon.a (mon.0) 112 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:09.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:08 smithi117 bash[21480]: cluster 2024-03-04T15:41:06.710313+0000 mgr.y (mgr.14152) 29 : cluster 0 pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:09.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:08 smithi117 bash[21480]: audit 2024-03-04T15:41:07.660079+0000 mon.a (mon.0) 113 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:09.838 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:41:10.884 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:10 smithi117 bash[21480]: cluster 2024-03-04T15:41:08.710694+0000 mgr.y (mgr.14152) 30 : cluster 0 pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:12.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.081+0000 7fece1d64700 1 -- 172.21.15.117:0/4125543900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc108040 msgr2=0x7fecdc108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:12.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.081+0000 7fece1d64700 1 --2- 172.21.15.117:0/4125543900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc108040 0x7fecdc108420 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7feccc009910 tx=0x7feccc009c20 comp rx=0 tx=0).stop 2024-03-04T15:41:12.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.081+0000 7fece1d64700 1 -- 172.21.15.117:0/4125543900 shutdown_connections 2024-03-04T15:41:12.084 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.081+0000 7fece1d64700 1 --2- 172.21.15.117:0/4125543900 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc108040 0x7fecdc108420 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:12.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.081+0000 7fece1d64700 1 -- 172.21.15.117:0/4125543900 >> 172.21.15.117:0/4125543900 conn(0x7fecdc077610 msgr2=0x7fecdc079a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:41:12.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.081+0000 7fece1d64700 1 -- 172.21.15.117:0/4125543900 shutdown_connections 2024-03-04T15:41:12.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.081+0000 7fece1d64700 1 -- 172.21.15.117:0/4125543900 wait complete. 2024-03-04T15:41:12.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.081+0000 7fece1d64700 1 Processor -- start 2024-03-04T15:41:12.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.081+0000 7fece1d64700 1 -- start start 2024-03-04T15:41:12.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.081+0000 7fece1d64700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc108040 0x7fecdc19ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:41:12.085 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.081+0000 7fece1d64700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecdc111870 con 0x7fecdc108040 2024-03-04T15:41:12.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.081+0000 7fecdb7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc108040 0x7fecdc19ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:41:12.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.081+0000 7fecdb7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc108040 0x7fecdc19ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:60124/0 (socket says 172.21.15.117:60124) 2024-03-04T15:41:12.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.081+0000 7fecdb7fe700 1 -- 172.21.15.117:0/4153283950 learned_addr learned my addr 172.21.15.117:0/4153283950 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:41:12.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.081+0000 7fecdb7fe700 1 -- 172.21.15.117:0/4153283950 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecdc19d390 con 0x7fecdc108040 2024-03-04T15:41:12.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.085+0000 7fecdb7fe700 1 --2- 172.21.15.117:0/4153283950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc108040 0x7fecdc19ce50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feccc006010 tx=0x7feccc00f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:41:12.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.085+0000 7fecd8ff9700 1 -- 172.21.15.117:0/4153283950 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feccc027070 con 0x7fecdc108040 2024-03-04T15:41:12.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.085+0000 7fecd8ff9700 1 -- 172.21.15.117:0/4153283950 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7feccc022470 con 0x7fecdc108040 2024-03-04T15:41:12.087 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.085+0000 7fece1d64700 1 -- 172.21.15.117:0/4153283950 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecdc19d680 con 0x7fecdc108040 2024-03-04T15:41:12.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.085+0000 7fecd8ff9700 1 -- 172.21.15.117:0/4153283950 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feccc017440 con 0x7fecdc108040 2024-03-04T15:41:12.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.085+0000 7fece1d64700 1 -- 172.21.15.117:0/4153283950 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecdc1a1720 con 0x7fecdc108040 2024-03-04T15:41:12.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.085+0000 7fecd8ff9700 1 -- 172.21.15.117:0/4153283950 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7feccc02a070 con 0x7fecdc108040 2024-03-04T15:41:12.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.085+0000 7fece1d64700 1 -- 172.21.15.117:0/4153283950 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecdc109510 con 0x7fecdc108040 2024-03-04T15:41:12.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.085+0000 7fecd8ff9700 1 --2- 172.21.15.117:0/4153283950 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fecc4038ca0 0x7fecc403b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:41:12.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.085+0000 7fecd8ff9700 1 -- 172.21.15.117:0/4153283950 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7feccc01e030 con 0x7fecdc108040 2024-03-04T15:41:12.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.085+0000 7fecdaffd700 1 --2- 172.21.15.117:0/4153283950 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fecc4038ca0 0x7fecc403b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:41:12.088 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.085+0000 7fecdaffd700 1 --2- 172.21.15.117:0/4153283950 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fecc4038ca0 0x7fecc403b160 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fecd0006fd0 tx=0x7fecd0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:41:12.091 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.089+0000 7fecd8ff9700 1 -- 172.21.15.117:0/4153283950 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feccc013070 con 0x7fecdc108040 2024-03-04T15:41:12.301 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.301+0000 7fece1d64700 1 -- 172.21.15.117:0/4153283950 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fecdc19e1b0 con 0x7fecc4038ca0 2024-03-04T15:41:12.303 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.301+0000 7fecd8ff9700 1 -- 172.21.15.117:0/4153283950 <== mgr.14152 v2:172.21.15.117:6800/2865911351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7fecdc19e1b0 con 0x7fecc4038ca0 2024-03-04T15:41:12.303 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-04T15:41:12.303 INFO:teuthology.orchestra.run.smithi117.stdout:[{"addr": "172.21.15.117", "hostname": "smithi117", "labels": [], "status": ""}, {"addr": "172.21.15.151", "hostname": "smithi151", "labels": [], "status": ""}] 2024-03-04T15:41:12.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.301+0000 7fece1d64700 1 -- 172.21.15.117:0/4153283950 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fecc4038ca0 msgr2=0x7fecc403b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:12.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.301+0000 7fece1d64700 1 --2- 172.21.15.117:0/4153283950 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fecc4038ca0 0x7fecc403b160 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fecd0006fd0 tx=0x7fecd0006e40 comp rx=0 tx=0).stop 2024-03-04T15:41:12.305 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.301+0000 7fece1d64700 1 -- 172.21.15.117:0/4153283950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc108040 msgr2=0x7fecdc19ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:12.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.305+0000 7fece1d64700 1 --2- 172.21.15.117:0/4153283950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc108040 0x7fecdc19ce50 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7feccc006010 tx=0x7feccc00f740 comp rx=0 tx=0).stop 2024-03-04T15:41:12.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.305+0000 7fece1d64700 1 -- 172.21.15.117:0/4153283950 shutdown_connections 2024-03-04T15:41:12.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.305+0000 7fece1d64700 1 --2- 172.21.15.117:0/4153283950 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fecc4038ca0 0x7fecc403b160 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:12.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.305+0000 7fece1d64700 1 --2- 172.21.15.117:0/4153283950 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fecdc108040 0x7fecdc19ce50 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:12.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.305+0000 7fece1d64700 1 -- 172.21.15.117:0/4153283950 >> 172.21.15.117:0/4153283950 conn(0x7fecdc077610 msgr2=0x7fecdc10fd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:41:12.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.305+0000 7fece1d64700 1 -- 172.21.15.117:0/4153283950 shutdown_connections 2024-03-04T15:41:12.306 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:12.305+0000 7fece1d64700 1 -- 172.21.15.117:0/4153283950 wait complete. 2024-03-04T15:41:12.884 INFO:tasks.cephadm:Setting crush tunables to default 2024-03-04T15:41:12.884 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph osd crush tunables default 2024-03-04T15:41:13.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:12 smithi117 bash[21480]: cluster 2024-03-04T15:41:10.710996+0000 mgr.y (mgr.14152) 31 : cluster 0 pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:14.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:13 smithi117 bash[21480]: audit 2024-03-04T15:41:12.302246+0000 mgr.y (mgr.14152) 32 : audit 0 from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T15:41:14.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:13 smithi117 bash[21480]: audit 2024-03-04T15:41:13.246274+0000 mon.a (mon.0) 114 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:14.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:13 smithi117 bash[21480]: audit 2024-03-04T15:41:13.252205+0000 mon.a (mon.0) 115 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:14.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:13 smithi117 bash[21480]: audit 2024-03-04T15:41:13.260828+0000 mon.a (mon.0) 116 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:14.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:13 smithi117 bash[21480]: audit 2024-03-04T15:41:13.268751+0000 mon.a (mon.0) 117 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:14.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:13 smithi117 bash[21480]: audit 2024-03-04T15:41:13.270293+0000 mon.a (mon.0) 118 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi151", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:41:14.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:13 smithi117 bash[21480]: audit 2024-03-04T15:41:13.272321+0000 mon.a (mon.0) 119 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:14.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:13 smithi117 bash[21480]: audit 2024-03-04T15:41:13.274139+0000 mon.a (mon.0) 120 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:41:15.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:14 smithi117 bash[21480]: cluster 2024-03-04T15:41:12.711362+0000 mgr.y (mgr.14152) 33 : cluster 0 pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:15.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:14 smithi117 bash[21480]: cephadm 2024-03-04T15:41:13.276718+0000 mgr.y (mgr.14152) 34 : cephadm 1 Updating smithi151:/etc/ceph/ceph.conf 2024-03-04T15:41:15.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:14 smithi117 bash[21480]: cephadm 2024-03-04T15:41:13.492797+0000 mgr.y (mgr.14152) 35 : cephadm 1 Updating smithi151:/var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T15:41:15.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:14 smithi117 bash[21480]: cephadm 2024-03-04T15:41:13.658109+0000 mgr.y (mgr.14152) 36 : cephadm 1 Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-03-04T15:41:15.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:14 smithi117 bash[21480]: audit 2024-03-04T15:41:13.993037+0000 mon.a (mon.0) 121 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:15.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:14 smithi117 bash[21480]: audit 2024-03-04T15:41:14.001191+0000 mon.a (mon.0) 122 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:15.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:14 smithi117 bash[21480]: audit 2024-03-04T15:41:14.009832+0000 mon.a (mon.0) 123 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:16.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:15 smithi117 bash[21480]: cephadm 2024-03-04T15:41:13.822733+0000 mgr.y (mgr.14152) 37 : cephadm 1 Updating smithi151:/var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T15:41:17.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:16 smithi117 bash[21480]: cluster 2024-03-04T15:41:14.711771+0000 mgr.y (mgr.14152) 38 : cluster 0 pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:17.587 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:41:19.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:18 smithi117 bash[21480]: cluster 2024-03-04T15:41:16.712181+0000 mgr.y (mgr.14152) 39 : cluster 0 pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:19.897 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a4c4d2700 1 -- 172.21.15.117:0/3034018281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a441099a0 msgr2=0x7f5a44109d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:19.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a4c4d2700 1 --2- 172.21.15.117:0/3034018281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a441099a0 0x7f5a44109d80 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f5a34009910 tx=0x7f5a34009c20 comp rx=0 tx=0).stop 2024-03-04T15:41:19.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a4c4d2700 1 -- 172.21.15.117:0/3034018281 shutdown_connections 2024-03-04T15:41:19.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a4c4d2700 1 --2- 172.21.15.117:0/3034018281 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a441099a0 0x7f5a44109d80 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:19.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a4c4d2700 1 -- 172.21.15.117:0/3034018281 >> 172.21.15.117:0/3034018281 conn(0x7f5a440ff4d0 msgr2=0x7f5a441018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:41:19.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a4c4d2700 1 -- 172.21.15.117:0/3034018281 shutdown_connections 2024-03-04T15:41:19.898 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a4c4d2700 1 -- 172.21.15.117:0/3034018281 wait complete. 2024-03-04T15:41:19.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a4c4d2700 1 Processor -- start 2024-03-04T15:41:19.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a4c4d2700 1 -- start start 2024-03-04T15:41:19.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a4c4d2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a441099a0 0x7f5a4419e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:41:19.900 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a4c4d2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a4410d460 con 0x7f5a441099a0 2024-03-04T15:41:19.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a4a26e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a441099a0 0x7f5a4419e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:41:19.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a4a26e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a441099a0 0x7f5a4419e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56596/0 (socket says 172.21.15.117:56596) 2024-03-04T15:41:19.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a4a26e700 1 -- 172.21.15.117:0/487465745 learned_addr learned my addr 172.21.15.117:0/487465745 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:41:19.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a4a26e700 1 -- 172.21.15.117:0/487465745 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a4419ebc0 con 0x7f5a441099a0 2024-03-04T15:41:19.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a4a26e700 1 --2- 172.21.15.117:0/487465745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a441099a0 0x7f5a4419e680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5a34006010 tx=0x7f5a3400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:41:19.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a437fe700 1 -- 172.21.15.117:0/487465745 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a34027070 con 0x7f5a441099a0 2024-03-04T15:41:19.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a437fe700 1 -- 172.21.15.117:0/487465745 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f5a34022470 con 0x7f5a441099a0 2024-03-04T15:41:19.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a4c4d2700 1 -- 172.21.15.117:0/487465745 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a4419eeb0 con 0x7f5a441099a0 2024-03-04T15:41:19.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a437fe700 1 -- 172.21.15.117:0/487465745 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5a34017440 con 0x7f5a441099a0 2024-03-04T15:41:19.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a4c4d2700 1 -- 172.21.15.117:0/487465745 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a44198800 con 0x7f5a441099a0 2024-03-04T15:41:19.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a437fe700 1 -- 172.21.15.117:0/487465745 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f5a3402a070 con 0x7f5a441099a0 2024-03-04T15:41:19.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a437fe700 1 --2- 172.21.15.117:0/487465745 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f5a2c038c40 0x7f5a2c03b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:41:19.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a437fe700 1 -- 172.21.15.117:0/487465745 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5a3401e030 con 0x7f5a441099a0 2024-03-04T15:41:19.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a4c4d2700 1 -- 172.21.15.117:0/487465745 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a441050e0 con 0x7f5a441099a0 2024-03-04T15:41:19.901 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a49a6d700 1 --2- 172.21.15.117:0/487465745 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f5a2c038c40 0x7f5a2c03b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:41:19.902 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.896+0000 7f5a49a6d700 1 --2- 172.21.15.117:0/487465745 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f5a2c038c40 0x7f5a2c03b100 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800ad30 tx=0x7f5a380093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:41:19.905 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:19.904+0000 7f5a437fe700 1 -- 172.21.15.117:0/487465745 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5a34013070 con 0x7f5a441099a0 2024-03-04T15:41:20.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:20.108+0000 7f5a4c4d2700 1 -- 172.21.15.117:0/487465745 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f5a44063410 con 0x7f5a441099a0 2024-03-04T15:41:20.679 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:20.676+0000 7f5a437fe700 1 -- 172.21.15.117:0/487465745 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f5a3402ac60 con 0x7f5a441099a0 2024-03-04T15:41:20.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:20.680+0000 7f5a4c4d2700 1 -- 172.21.15.117:0/487465745 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f5a2c038c40 msgr2=0x7f5a2c03b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:20.682 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:20.680+0000 7f5a4c4d2700 1 --2- 172.21.15.117:0/487465745 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f5a2c038c40 0x7f5a2c03b100 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f5a3800ad30 tx=0x7f5a380093f0 comp rx=0 tx=0).stop 2024-03-04T15:41:20.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:20.680+0000 7f5a4c4d2700 1 -- 172.21.15.117:0/487465745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a441099a0 msgr2=0x7f5a4419e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:20.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:20.680+0000 7f5a4c4d2700 1 --2- 172.21.15.117:0/487465745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a441099a0 0x7f5a4419e680 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f5a34006010 tx=0x7f5a3400f740 comp rx=0 tx=0).stop 2024-03-04T15:41:20.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:20.680+0000 7f5a4c4d2700 1 -- 172.21.15.117:0/487465745 shutdown_connections 2024-03-04T15:41:20.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:20.680+0000 7f5a4c4d2700 1 --2- 172.21.15.117:0/487465745 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f5a2c038c40 0x7f5a2c03b100 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:20.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:20.680+0000 7f5a4c4d2700 1 --2- 172.21.15.117:0/487465745 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5a441099a0 0x7f5a4419e680 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:20.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:20.680+0000 7f5a4c4d2700 1 -- 172.21.15.117:0/487465745 >> 172.21.15.117:0/487465745 conn(0x7f5a440ff4d0 msgr2=0x7f5a4410b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:41:20.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:20.680+0000 7f5a4c4d2700 1 -- 172.21.15.117:0/487465745 shutdown_connections 2024-03-04T15:41:20.683 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:20.680+0000 7f5a4c4d2700 1 -- 172.21.15.117:0/487465745 wait complete. 2024-03-04T15:41:20.683 INFO:teuthology.orchestra.run.smithi117.stderr:adjusted tunables profile to default 2024-03-04T15:41:21.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:20 smithi117 bash[21480]: cluster 2024-03-04T15:41:18.712574+0000 mgr.y (mgr.14152) 40 : cluster 0 pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:21.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:20 smithi117 bash[21480]: audit 2024-03-04T15:41:20.110462+0000 mon.a (mon.0) 124 : audit 1 from='client.? 172.21.15.117:0/487465745' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-03-04T15:41:21.327 INFO:tasks.cephadm:Adding mon.a on smithi117 2024-03-04T15:41:21.327 INFO:tasks.cephadm:Adding mon.c on smithi117 2024-03-04T15:41:21.327 INFO:tasks.cephadm:Adding mon.b on smithi151 2024-03-04T15:41:21.327 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph orch apply mon '3;smithi117:172.21.15.117=a;smithi117:[v2:172.21.15.117:3301,v1:172.21.15.117:6790]=c;smithi151:172.21.15.151=b' 2024-03-04T15:41:22.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:21 smithi117 bash[21480]: audit 2024-03-04T15:41:20.678833+0000 mon.a (mon.0) 125 : audit 1 from='client.? 172.21.15.117:0/487465745' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-03-04T15:41:22.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:21 smithi117 bash[21480]: cluster 2024-03-04T15:41:20.684704+0000 mon.a (mon.0) 126 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2024-03-04T15:41:22.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:21 smithi117 bash[21480]: cluster 2024-03-04T15:41:20.713002+0000 mgr.y (mgr.14152) 41 : cluster 0 pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:22.541 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T15:41:24.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:23 smithi117 bash[21480]: cluster 2024-03-04T15:41:22.713352+0000 mgr.y (mgr.14152) 42 : cluster 0 pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:24.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.798+0000 7fbf1a854700 1 -- 172.21.15.151:0/615722497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf14108040 msgr2=0x7fbf14108420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:24.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.798+0000 7fbf1a854700 1 --2- 172.21.15.151:0/615722497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf14108040 0x7fbf14108420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbf04009910 tx=0x7fbf04009c20 comp rx=0 tx=0).stop 2024-03-04T15:41:24.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.798+0000 7fbf1a854700 1 -- 172.21.15.151:0/615722497 shutdown_connections 2024-03-04T15:41:24.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.798+0000 7fbf1a854700 1 --2- 172.21.15.151:0/615722497 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf14108040 0x7fbf14108420 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:24.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.798+0000 7fbf1a854700 1 -- 172.21.15.151:0/615722497 >> 172.21.15.151:0/615722497 conn(0x7fbf14077610 msgr2=0x7fbf14079a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:41:24.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.798+0000 7fbf1a854700 1 -- 172.21.15.151:0/615722497 shutdown_connections 2024-03-04T15:41:24.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.798+0000 7fbf1a854700 1 -- 172.21.15.151:0/615722497 wait complete. 2024-03-04T15:41:24.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.798+0000 7fbf1a854700 1 Processor -- start 2024-03-04T15:41:24.802 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.798+0000 7fbf1a854700 1 -- start start 2024-03-04T15:41:24.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.798+0000 7fbf1a854700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf14108040 0x7fbf1419ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:41:24.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.798+0000 7fbf1a854700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf14111870 con 0x7fbf14108040 2024-03-04T15:41:24.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.798+0000 7fbf13fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf14108040 0x7fbf1419ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:41:24.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.798+0000 7fbf13fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf14108040 0x7fbf1419ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.151:56170/0 (socket says 172.21.15.151:56170) 2024-03-04T15:41:24.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.798+0000 7fbf13fff700 1 -- 172.21.15.151:0/3042462138 learned_addr learned my addr 172.21.15.151:0/3042462138 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-04T15:41:24.803 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.798+0000 7fbf13fff700 1 -- 172.21.15.151:0/3042462138 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1419d380 con 0x7fbf14108040 2024-03-04T15:41:24.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.802+0000 7fbf13fff700 1 --2- 172.21.15.151:0/3042462138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf14108040 0x7fbf1419ce40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400f690 tx=0x7fbf0400f770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:41:24.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.802+0000 7fbf117fa700 1 -- 172.21.15.151:0/3042462138 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf04027070 con 0x7fbf14108040 2024-03-04T15:41:24.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.802+0000 7fbf1a854700 1 -- 172.21.15.151:0/3042462138 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1419d670 con 0x7fbf14108040 2024-03-04T15:41:24.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.802+0000 7fbf1a854700 1 -- 172.21.15.151:0/3042462138 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf141a1710 con 0x7fbf14108040 2024-03-04T15:41:24.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.802+0000 7fbf117fa700 1 -- 172.21.15.151:0/3042462138 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fbf0400fb90 con 0x7fbf14108040 2024-03-04T15:41:24.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.802+0000 7fbf117fa700 1 -- 172.21.15.151:0/3042462138 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbf04017440 con 0x7fbf14108040 2024-03-04T15:41:24.804 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.802+0000 7fbf1a854700 1 -- 172.21.15.151:0/3042462138 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf14109510 con 0x7fbf14108040 2024-03-04T15:41:24.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.802+0000 7fbf117fa700 1 -- 172.21.15.151:0/3042462138 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fbf0402a070 con 0x7fbf14108040 2024-03-04T15:41:24.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.802+0000 7fbf117fa700 1 --2- 172.21.15.151:0/3042462138 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fbf00038c40 0x7fbf0003b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:41:24.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.802+0000 7fbf117fa700 1 -- 172.21.15.151:0/3042462138 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbf0401e030 con 0x7fbf14108040 2024-03-04T15:41:24.805 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.802+0000 7fbf137fe700 1 --2- 172.21.15.151:0/3042462138 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fbf00038c40 0x7fbf0003b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:41:24.806 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.802+0000 7fbf137fe700 1 --2- 172.21.15.151:0/3042462138 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fbf00038c40 0x7fbf0003b100 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00ad30 tx=0x7fbefc0093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:41:24.818 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:24.814+0000 7fbf117fa700 1 -- 172.21.15.151:0/3042462138 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbf04013070 con 0x7fbf14108040 2024-03-04T15:41:25.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:25.034+0000 7fbf1a854700 1 -- 172.21.15.151:0/3042462138 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi117:172.21.15.117=a;smithi117:[v2:172.21.15.117:3301,v1:172.21.15.117:6790]=c;smithi151:172.21.15.151=b", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf14063410 con 0x7fbf00038c40 2024-03-04T15:41:25.052 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:25.050+0000 7fbf117fa700 1 -- 172.21.15.151:0/3042462138 <== mgr.14152 v2:172.21.15.117:6800/2865911351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fbf14063410 con 0x7fbf00038c40 2024-03-04T15:41:25.053 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled mon update... 2024-03-04T15:41:25.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:25.050+0000 7fbf1a854700 1 -- 172.21.15.151:0/3042462138 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fbf00038c40 msgr2=0x7fbf0003b100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:25.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:25.050+0000 7fbf1a854700 1 --2- 172.21.15.151:0/3042462138 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fbf00038c40 0x7fbf0003b100 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00ad30 tx=0x7fbefc0093f0 comp rx=0 tx=0).stop 2024-03-04T15:41:25.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:25.050+0000 7fbf1a854700 1 -- 172.21.15.151:0/3042462138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf14108040 msgr2=0x7fbf1419ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:25.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:25.050+0000 7fbf1a854700 1 --2- 172.21.15.151:0/3042462138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf14108040 0x7fbf1419ce40 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbf0400f690 tx=0x7fbf0400f770 comp rx=0 tx=0).stop 2024-03-04T15:41:25.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:25.050+0000 7fbf1a854700 1 -- 172.21.15.151:0/3042462138 shutdown_connections 2024-03-04T15:41:25.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:25.050+0000 7fbf1a854700 1 --2- 172.21.15.151:0/3042462138 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fbf00038c40 0x7fbf0003b100 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:25.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:25.050+0000 7fbf1a854700 1 --2- 172.21.15.151:0/3042462138 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbf14108040 0x7fbf1419ce40 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:25.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:25.050+0000 7fbf1a854700 1 -- 172.21.15.151:0/3042462138 >> 172.21.15.151:0/3042462138 conn(0x7fbf14077610 msgr2=0x7fbf1410fd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:41:25.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:25.050+0000 7fbf1a854700 1 -- 172.21.15.151:0/3042462138 shutdown_connections 2024-03-04T15:41:25.055 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:25.050+0000 7fbf1a854700 1 -- 172.21.15.151:0/3042462138 wait complete. 2024-03-04T15:41:25.747 DEBUG:teuthology.orchestra.run.smithi117:mon.c> sudo journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mon.c.service 2024-03-04T15:41:25.750 DEBUG:teuthology.orchestra.run.smithi151:mon.b> sudo journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mon.b.service 2024-03-04T15:41:25.752 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-04T15:41:25.752 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T15:41:26.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:26 smithi117 bash[21480]: cluster 2024-03-04T15:41:24.713723+0000 mgr.y (mgr.14152) 43 : cluster 0 pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:26.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:26 smithi117 bash[21480]: audit 2024-03-04T15:41:25.039335+0000 mgr.y (mgr.14152) 44 : audit 0 from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi117:172.21.15.117=a;smithi117:[v2:172.21.15.117:3301,v1:172.21.15.117:6790]=c;smithi151:172.21.15.151=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:41:26.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:26 smithi117 bash[21480]: cephadm 2024-03-04T15:41:25.045474+0000 mgr.y (mgr.14152) 45 : cephadm 1 Saving service mon spec with placement smithi117:172.21.15.117=a;smithi117:[v2:172.21.15.117:3301,v1:172.21.15.117:6790]=c;smithi151:172.21.15.151=b;count:3 2024-03-04T15:41:26.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:26 smithi117 bash[21480]: audit 2024-03-04T15:41:25.051174+0000 mon.a (mon.0) 127 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:26.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:26 smithi117 bash[21480]: audit 2024-03-04T15:41:25.068346+0000 mon.a (mon.0) 128 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:41:26.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:26 smithi117 bash[21480]: audit 2024-03-04T15:41:25.071910+0000 mon.a (mon.0) 129 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:26.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:26 smithi117 bash[21480]: audit 2024-03-04T15:41:25.073542+0000 mon.a (mon.0) 130 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:41:26.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:26 smithi117 bash[21480]: audit 2024-03-04T15:41:25.081870+0000 mon.a (mon.0) 131 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:26.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:26 smithi117 bash[21480]: audit 2024-03-04T15:41:25.085931+0000 mon.a (mon.0) 132 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T15:41:26.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:26 smithi117 bash[21480]: audit 2024-03-04T15:41:25.087643+0000 mon.a (mon.0) 133 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:26.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:26 smithi117 bash[21480]: cephadm 2024-03-04T15:41:25.089811+0000 mgr.y (mgr.14152) 46 : cephadm 1 Deploying daemon mon.b on smithi151 2024-03-04T15:41:27.438 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T15:41:28.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:28 smithi117 bash[21480]: cluster 2024-03-04T15:41:26.714128+0000 mgr.y (mgr.14152) 47 : cluster 0 pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:30.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed654ce700 1 -- 172.21.15.151:0/1846732782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed601077c0 msgr2=0x7fed60107ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:30.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed654ce700 1 --2- 172.21.15.151:0/1846732782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed601077c0 0x7fed60107ba0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fed50009910 tx=0x7fed50009c20 comp rx=0 tx=0).stop 2024-03-04T15:41:30.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed654ce700 1 -- 172.21.15.151:0/1846732782 shutdown_connections 2024-03-04T15:41:30.160 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed654ce700 1 --2- 172.21.15.151:0/1846732782 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed601077c0 0x7fed60107ba0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:30.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed654ce700 1 -- 172.21.15.151:0/1846732782 >> 172.21.15.151:0/1846732782 conn(0x7fed600fd2d0 msgr2=0x7fed600ff6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:41:30.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed654ce700 1 -- 172.21.15.151:0/1846732782 shutdown_connections 2024-03-04T15:41:30.161 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed654ce700 1 -- 172.21.15.151:0/1846732782 wait complete. 2024-03-04T15:41:30.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed654ce700 1 Processor -- start 2024-03-04T15:41:30.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed654ce700 1 -- start start 2024-03-04T15:41:30.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed654ce700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed601077c0 0x7fed6019e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:41:30.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed654ce700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed6010b280 con 0x7fed601077c0 2024-03-04T15:41:30.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed5effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed601077c0 0x7fed6019e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:41:30.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed5effd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed601077c0 0x7fed6019e790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.151:34814/0 (socket says 172.21.15.151:34814) 2024-03-04T15:41:30.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed5effd700 1 -- 172.21.15.151:0/2518494838 learned_addr learned my addr 172.21.15.151:0/2518494838 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-04T15:41:30.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed5effd700 1 -- 172.21.15.151:0/2518494838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed6019ecd0 con 0x7fed601077c0 2024-03-04T15:41:30.162 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed5effd700 1 --2- 172.21.15.151:0/2518494838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed601077c0 0x7fed6019e790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fed50006010 tx=0x7fed5000f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:41:30.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed47fff700 1 -- 172.21.15.151:0/2518494838 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed50027070 con 0x7fed601077c0 2024-03-04T15:41:30.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed47fff700 1 -- 172.21.15.151:0/2518494838 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fed50022470 con 0x7fed601077c0 2024-03-04T15:41:30.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed654ce700 1 -- 172.21.15.151:0/2518494838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed6019efc0 con 0x7fed601077c0 2024-03-04T15:41:30.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed47fff700 1 -- 172.21.15.151:0/2518494838 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fed50017440 con 0x7fed601077c0 2024-03-04T15:41:30.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.158+0000 7fed654ce700 1 -- 172.21.15.151:0/2518494838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed60198950 con 0x7fed601077c0 2024-03-04T15:41:30.163 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.162+0000 7fed654ce700 1 -- 172.21.15.151:0/2518494838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed60102f00 con 0x7fed601077c0 2024-03-04T15:41:30.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.162+0000 7fed47fff700 1 -- 172.21.15.151:0/2518494838 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7fed5000fb60 con 0x7fed601077c0 2024-03-04T15:41:30.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.162+0000 7fed47fff700 1 --2- 172.21.15.151:0/2518494838 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fed48038ca0 0x7fed4803b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:41:30.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.162+0000 7fed47fff700 1 -- 172.21.15.151:0/2518494838 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fed5001e030 con 0x7fed601077c0 2024-03-04T15:41:30.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.162+0000 7fed5e7fc700 1 --2- 172.21.15.151:0/2518494838 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fed48038ca0 0x7fed4803b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:41:30.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.162+0000 7fed5e7fc700 1 --2- 172.21.15.151:0/2518494838 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fed48038ca0 0x7fed4803b160 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fed54006fd0 tx=0x7fed54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:41:30.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.170+0000 7fed47fff700 1 -- 172.21.15.151:0/2518494838 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fed50013070 con 0x7fed601077c0 2024-03-04T15:41:30.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:30 smithi117 bash[21480]: cluster 2024-03-04T15:41:28.714529+0000 mgr.y (mgr.14152) 48 : cluster 0 pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:30.461 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.458+0000 7fed654ce700 1 -- 172.21.15.151:0/2518494838 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fed60063410 con 0x7fed601077c0 2024-03-04T15:41:30.462 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.458+0000 7fed47fff700 1 -- 172.21.15.151:0/2518494838 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+749 (secure 0 0 0) 0x7fed5002be10 con 0x7fed601077c0 2024-03-04T15:41:30.462 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-04T15:41:30.462 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":1,"fsid":"e40b5504-da3c-11ee-95c6-87774f69a715","modified":"2024-03-04T15:39:11.403703Z","created":"2024-03-04T15:39:11.403703Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-03-04T15:41:30.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.462+0000 7fed654ce700 1 -- 172.21.15.151:0/2518494838 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fed48038ca0 msgr2=0x7fed4803b160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:30.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.462+0000 7fed654ce700 1 --2- 172.21.15.151:0/2518494838 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fed48038ca0 0x7fed4803b160 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fed54006fd0 tx=0x7fed54006e40 comp rx=0 tx=0).stop 2024-03-04T15:41:30.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.462+0000 7fed654ce700 1 -- 172.21.15.151:0/2518494838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed601077c0 msgr2=0x7fed6019e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:30.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.462+0000 7fed654ce700 1 --2- 172.21.15.151:0/2518494838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed601077c0 0x7fed6019e790 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fed50006010 tx=0x7fed5000f740 comp rx=0 tx=0).stop 2024-03-04T15:41:30.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.462+0000 7fed654ce700 1 -- 172.21.15.151:0/2518494838 shutdown_connections 2024-03-04T15:41:30.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.462+0000 7fed654ce700 1 --2- 172.21.15.151:0/2518494838 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fed48038ca0 0x7fed4803b160 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:30.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.462+0000 7fed654ce700 1 --2- 172.21.15.151:0/2518494838 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fed601077c0 0x7fed6019e790 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:30.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.462+0000 7fed654ce700 1 -- 172.21.15.151:0/2518494838 >> 172.21.15.151:0/2518494838 conn(0x7fed600fd2d0 msgr2=0x7fed600fd6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:41:30.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.462+0000 7fed654ce700 1 -- 172.21.15.151:0/2518494838 shutdown_connections 2024-03-04T15:41:30.465 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:30.462+0000 7fed654ce700 1 -- 172.21.15.151:0/2518494838 wait complete. 2024-03-04T15:41:30.466 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 1 2024-03-04T15:41:31.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:31 smithi117 bash[21480]: audit 2024-03-04T15:41:30.072168+0000 mon.a (mon.0) 134 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:31.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:31 smithi117 bash[21480]: audit 2024-03-04T15:41:30.078374+0000 mon.a (mon.0) 135 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:31.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:31 smithi117 bash[21480]: audit 2024-03-04T15:41:30.122109+0000 mon.a (mon.0) 136 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:31.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:31 smithi117 bash[21480]: audit 2024-03-04T15:41:30.123597+0000 mon.a (mon.0) 137 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T15:41:31.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:31 smithi117 bash[21480]: audit 2024-03-04T15:41:30.125355+0000 mon.a (mon.0) 138 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:31.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:31 smithi117 bash[21480]: cephadm 2024-03-04T15:41:30.127417+0000 mgr.y (mgr.14152) 49 : cephadm 1 Deploying daemon mon.c on smithi117 2024-03-04T15:41:31.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:31 smithi117 bash[21480]: audit 2024-03-04T15:41:30.461838+0000 mon.a (mon.0) 139 : audit 0 from='client.? 172.21.15.151:0/2518494838' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T15:41:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:31 smithi151 bash[23756]: debug 2024-03-04T15:41:31.493+0000 7f1dc5ffa700 1 mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-04T15:41:32.320 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-03-04T15:41:32.320 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph mon dump -f json 2024-03-04T15:41:33.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:33 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:41:33.821 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:41:33 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:41:34.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:33 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:41:34.322 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:41:33 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:41:35.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:34 smithi117 bash[28843]: debug 2024-03-04T15:41:34.868+0000 7f6be0696700 1 mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-03-04T15:41:36.018 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T15:41:36.913 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: cluster 2024-03-04T15:41:30.714944+0000 mgr.y (mgr.14152) 50 : cluster 0 pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:36.913 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: audit 2024-03-04T15:41:31.505157+0000 mon.a (mon.0) 141 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:41:36.913 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: audit 2024-03-04T15:41:31.505358+0000 mon.a (mon.0) 142 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:36.913 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: cluster 2024-03-04T15:41:31.505452+0000 mon.a (mon.0) 143 : cluster 1 mon.a calling monitor election 2024-03-04T15:41:36.913 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: audit 2024-03-04T15:41:32.502866+0000 mon.a (mon.0) 144 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:36.913 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: cluster 2024-03-04T15:41:32.715311+0000 mgr.y (mgr.14152) 51 : cluster 0 pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:36.913 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: audit 2024-03-04T15:41:33.503246+0000 mon.a (mon.0) 145 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:36.913 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: cluster 2024-03-04T15:41:33.504877+0000 mon.b (mon.1) 1 : cluster 1 mon.b calling monitor election 2024-03-04T15:41:36.913 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: audit 2024-03-04T15:41:34.503273+0000 mon.a (mon.0) 146 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:36.913 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: cluster 2024-03-04T15:41:34.715730+0000 mgr.y (mgr.14152) 52 : cluster 0 pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:36.914 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: audit 2024-03-04T15:41:34.876823+0000 mon.a (mon.0) 147 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:36.914 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: audit 2024-03-04T15:41:35.503580+0000 mon.a (mon.0) 148 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:36.914 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: audit 2024-03-04T15:41:35.877602+0000 mon.a (mon.0) 149 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:36.914 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: audit 2024-03-04T15:41:36.503810+0000 mon.a (mon.0) 150 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:36.914 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: cluster 2024-03-04T15:41:36.549184+0000 mon.a (mon.0) 151 : cluster 1 mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-04T15:41:36.914 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: cluster 2024-03-04T15:41:36.559703+0000 mon.a (mon.0) 152 : cluster 0 monmap e2: 2 mons at {a=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],b=[v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T15:41:36.914 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: cluster 2024-03-04T15:41:36.582091+0000 mon.a (mon.0) 153 : cluster 0 fsmap 2024-03-04T15:41:36.914 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: cluster 2024-03-04T15:41:36.582173+0000 mon.a (mon.0) 154 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2024-03-04T15:41:36.914 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: cluster 2024-03-04T15:41:36.582461+0000 mon.a (mon.0) 155 : cluster 0 mgrmap e13: y(active, since 73s) 2024-03-04T15:41:36.914 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: cluster 2024-03-04T15:41:36.582717+0000 mon.a (mon.0) 156 : cluster 1 overall HEALTH_OK 2024-03-04T15:41:36.914 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: audit 2024-03-04T15:41:36.589520+0000 mon.a (mon.0) 157 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:36.914 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: audit 2024-03-04T15:41:36.597192+0000 mon.a (mon.0) 158 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:36.914 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: audit 2024-03-04T15:41:36.615660+0000 mon.a (mon.0) 159 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:36.914 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:36 smithi151 bash[23756]: audit 2024-03-04T15:41:36.622826+0000 mon.a (mon.0) 160 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:37.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: cluster 2024-03-04T15:41:30.714944+0000 mgr.y (mgr.14152) 50 : cluster 0 pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:37.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: audit 2024-03-04T15:41:31.505157+0000 mon.a (mon.0) 141 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:41:37.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: audit 2024-03-04T15:41:31.505358+0000 mon.a (mon.0) 142 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:37.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: cluster 2024-03-04T15:41:31.505452+0000 mon.a (mon.0) 143 : cluster 1 mon.a calling monitor election 2024-03-04T15:41:37.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: audit 2024-03-04T15:41:32.502866+0000 mon.a (mon.0) 144 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:37.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: cluster 2024-03-04T15:41:32.715311+0000 mgr.y (mgr.14152) 51 : cluster 0 pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:37.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: audit 2024-03-04T15:41:33.503246+0000 mon.a (mon.0) 145 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:37.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: cluster 2024-03-04T15:41:33.504877+0000 mon.b (mon.1) 1 : cluster 1 mon.b calling monitor election 2024-03-04T15:41:37.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: audit 2024-03-04T15:41:34.503273+0000 mon.a (mon.0) 146 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:37.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: cluster 2024-03-04T15:41:34.715730+0000 mgr.y (mgr.14152) 52 : cluster 0 pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:37.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: audit 2024-03-04T15:41:34.876823+0000 mon.a (mon.0) 147 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:37.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: audit 2024-03-04T15:41:35.503580+0000 mon.a (mon.0) 148 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:37.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: audit 2024-03-04T15:41:35.877602+0000 mon.a (mon.0) 149 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:37.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: audit 2024-03-04T15:41:36.503810+0000 mon.a (mon.0) 150 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:37.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: cluster 2024-03-04T15:41:36.549184+0000 mon.a (mon.0) 151 : cluster 1 mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-04T15:41:37.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: cluster 2024-03-04T15:41:36.559703+0000 mon.a (mon.0) 152 : cluster 0 monmap e2: 2 mons at {a=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],b=[v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T15:41:37.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: cluster 2024-03-04T15:41:36.582091+0000 mon.a (mon.0) 153 : cluster 0 fsmap 2024-03-04T15:41:37.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: cluster 2024-03-04T15:41:36.582173+0000 mon.a (mon.0) 154 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2024-03-04T15:41:37.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: cluster 2024-03-04T15:41:36.582461+0000 mon.a (mon.0) 155 : cluster 0 mgrmap e13: y(active, since 73s) 2024-03-04T15:41:37.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: cluster 2024-03-04T15:41:36.582717+0000 mon.a (mon.0) 156 : cluster 1 overall HEALTH_OK 2024-03-04T15:41:37.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: audit 2024-03-04T15:41:36.589520+0000 mon.a (mon.0) 157 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:37.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: audit 2024-03-04T15:41:36.597192+0000 mon.a (mon.0) 158 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:37.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: audit 2024-03-04T15:41:36.615660+0000 mon.a (mon.0) 159 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:37.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:36 smithi117 bash[21480]: audit 2024-03-04T15:41:36.622826+0000 mon.a (mon.0) 160 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:37.821 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:41:37 smithi117 bash[21734]: debug 2024-03-04T15:41:37.500+0000 7f580047f700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-03-04T15:41:42.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.025+0000 7f5750f32700 1 -- 172.21.15.151:0/4199913128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57380035c0 msgr2=0x7f5738005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:42.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.025+0000 7f5750f32700 1 --2- 172.21.15.151:0/4199913128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57380035c0 0x7f5738005a50 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f574c1017f0 tx=0x7f573c009e80 comp rx=0 tx=0).stop 2024-03-04T15:41:42.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.025+0000 7f5750f32700 1 -- 172.21.15.151:0/4199913128 shutdown_connections 2024-03-04T15:41:42.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.025+0000 7f5750f32700 1 --2- 172.21.15.151:0/4199913128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57380035c0 0x7f5738005a50 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:42.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.025+0000 7f5750f32700 1 -- 172.21.15.151:0/4199913128 >> 172.21.15.151:0/4199913128 conn(0x7f574c0fd3b0 msgr2=0x7f574c0ff7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:41:42.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.025+0000 7f5750f32700 1 -- 172.21.15.151:0/4199913128 shutdown_connections 2024-03-04T15:41:42.031 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.025+0000 7f5750f32700 1 -- 172.21.15.151:0/4199913128 wait complete. 2024-03-04T15:41:42.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f5750f32700 1 Processor -- start 2024-03-04T15:41:42.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f5750f32700 1 -- start start 2024-03-04T15:41:42.034 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f5750f32700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57380035c0 0x7f574c19c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:41:42.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f5750f32700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f574c19cbe0 0x7f574c1968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:41:42.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f5750f32700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f574c196df0 0x7f574c1972a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:41:42.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f5750f32700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c108e30 con 0x7f57380035c0 2024-03-04T15:41:42.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f5750f32700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f574c108cb0 con 0x7f574c19cbe0 2024-03-04T15:41:42.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f5750f32700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f574c108fb0 con 0x7f574c196df0 2024-03-04T15:41:42.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f5749d9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f574c19cbe0 0x7f574c1968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:41:42.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f5749d9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f574c19cbe0 0x7f574c1968b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:45554/0 (socket says 172.21.15.151:45554) 2024-03-04T15:41:42.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f5749d9b700 1 -- 172.21.15.151:0/2098865800 learned_addr learned my addr 172.21.15.151:0/2098865800 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-04T15:41:42.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f574a59c700 1 --2- 172.21.15.151:0/2098865800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57380035c0 0x7f574c19c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:41:42.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f5749d9b700 1 -- 172.21.15.151:0/2098865800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f574c19cbe0 msgr2=0x7f574c1968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-03-04T15:41:42.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f5749d9b700 1 -- 172.21.15.151:0/2098865800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f574c19cbe0 msgr2=0x7f574c1968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-04T15:41:42.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f5749d9b700 1 --2- 172.21.15.151:0/2098865800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f574c19cbe0 0x7f574c1968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-04T15:41:42.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f574ad9d700 1 --2- 172.21.15.151:0/2098865800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f574c196df0 0x7f574c1972a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:41:42.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f5749d9b700 1 --2- 172.21.15.151:0/2098865800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f574c19cbe0 0x7f574c1968b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T15:41:42.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f574ad9d700 1 -- 172.21.15.151:0/2098865800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f574c196df0 msgr2=0x7f574c1972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-03-04T15:41:42.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f574ad9d700 1 -- 172.21.15.151:0/2098865800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f574c196df0 msgr2=0x7f574c1972a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-03-04T15:41:42.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f574ad9d700 1 --2- 172.21.15.151:0/2098865800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f574c196df0 0x7f574c1972a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-03-04T15:41:42.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f574ad9d700 1 --2- 172.21.15.151:0/2098865800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f574c196df0 0x7f574c1972a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T15:41:42.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f574a59c700 1 -- 172.21.15.151:0/2098865800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f574c196df0 msgr2=0x7f574c1972a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T15:41:42.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f574a59c700 1 --2- 172.21.15.151:0/2098865800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f574c196df0 0x7f574c1972a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:42.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f574a59c700 1 -- 172.21.15.151:0/2098865800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f574c19cbe0 msgr2=0x7f574c1968b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T15:41:42.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f574a59c700 1 --2- 172.21.15.151:0/2098865800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f574c19cbe0 0x7f574c1968b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:42.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f574a59c700 1 -- 172.21.15.151:0/2098865800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f574c1a3570 con 0x7f57380035c0 2024-03-04T15:41:42.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f574a59c700 1 --2- 172.21.15.151:0/2098865800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57380035c0 0x7f574c19c6a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f573c000f80 tx=0x7f573c021760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:41:42.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f57437fe700 1 -- 172.21.15.151:0/2098865800 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c003a00 con 0x7f57380035c0 2024-03-04T15:41:42.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f57437fe700 1 -- 172.21.15.151:0/2098865800 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f573c003ba0 con 0x7f57380035c0 2024-03-04T15:41:42.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f5750f32700 1 -- 172.21.15.151:0/2098865800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f574c1a3740 con 0x7f57380035c0 2024-03-04T15:41:42.037 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f57437fe700 1 -- 172.21.15.151:0/2098865800 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f573c004460 con 0x7f57380035c0 2024-03-04T15:41:42.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.029+0000 7f5750f32700 1 -- 172.21.15.151:0/2098865800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f574c1a3cd0 con 0x7f57380035c0 2024-03-04T15:41:42.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.033+0000 7f57437fe700 1 -- 172.21.15.151:0/2098865800 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f573c015070 con 0x7f57380035c0 2024-03-04T15:41:42.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.033+0000 7f5750f32700 1 -- 172.21.15.151:0/2098865800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f574c063410 con 0x7f57380035c0 2024-03-04T15:41:42.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.033+0000 7f57437fe700 1 --2- 172.21.15.151:0/2098865800 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f572c0390d0 0x7f572c03b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:41:42.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.033+0000 7f57437fe700 1 -- 172.21.15.151:0/2098865800 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f573c04e240 con 0x7f57380035c0 2024-03-04T15:41:42.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.033+0000 7f5749d9b700 1 --2- 172.21.15.151:0/2098865800 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f572c0390d0 0x7f572c03b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:41:42.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.033+0000 7f5749d9b700 1 --2- 172.21.15.151:0/2098865800 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f572c0390d0 0x7f572c03b590 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5734009180 tx=0x7f5734008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:41:42.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.037+0000 7f57437fe700 1 -- 172.21.15.151:0/2098865800 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f573c01d030 con 0x7f57380035c0 2024-03-04T15:41:42.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: cluster 2024-03-04T15:41:36.716150+0000 mgr.y (mgr.14152) 53 : cluster 0 pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:42.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: cluster 2024-03-04T15:41:36.882672+0000 mon.b (mon.1) 2 : cluster 1 mon.b calling monitor election 2024-03-04T15:41:42.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: audit 2024-03-04T15:41:36.883818+0000 mon.a (mon.0) 163 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:41:42.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: cluster 2024-03-04T15:41:36.884225+0000 mon.a (mon.0) 164 : cluster 1 mon.a calling monitor election 2024-03-04T15:41:42.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: audit 2024-03-04T15:41:36.902243+0000 mon.a (mon.0) 165 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:42.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: audit 2024-03-04T15:41:36.902454+0000 mon.a (mon.0) 166 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:42.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: audit 2024-03-04T15:41:37.878086+0000 mon.a (mon.0) 167 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:42.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: cluster 2024-03-04T15:41:38.716560+0000 mgr.y (mgr.14152) 54 : cluster 0 pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:42.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: audit 2024-03-04T15:41:38.878231+0000 mon.a (mon.0) 168 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:42.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: audit 2024-03-04T15:41:39.878447+0000 mon.a (mon.0) 169 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:42.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: cluster 2024-03-04T15:41:40.717014+0000 mgr.y (mgr.14152) 55 : cluster 0 pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:42.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: audit 2024-03-04T15:41:40.878510+0000 mon.a (mon.0) 170 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:42.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: audit 2024-03-04T15:41:41.878522+0000 mon.a (mon.0) 171 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:42.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: cluster 2024-03-04T15:41:41.902231+0000 mon.a (mon.0) 172 : cluster 1 mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-04T15:41:42.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: cluster 2024-03-04T15:41:41.908462+0000 mon.a (mon.0) 173 : cluster 0 monmap e3: 3 mons at {a=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],b=[v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0],c=[v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T15:41:42.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: cluster 2024-03-04T15:41:41.925637+0000 mon.a (mon.0) 174 : cluster 0 fsmap 2024-03-04T15:41:42.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: cluster 2024-03-04T15:41:41.925690+0000 mon.a (mon.0) 175 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2024-03-04T15:41:42.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: cluster 2024-03-04T15:41:41.925904+0000 mon.a (mon.0) 176 : cluster 0 mgrmap e13: y(active, since 79s) 2024-03-04T15:41:42.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: cluster 2024-03-04T15:41:41.926131+0000 mon.a (mon.0) 177 : cluster 3 Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-03-04T15:41:42.324 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: cluster 2024-03-04T15:41:41.931332+0000 mon.a (mon.0) 178 : cluster 3 Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-03-04T15:41:42.324 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: cluster 2024-03-04T15:41:41.931377+0000 mon.a (mon.0) 179 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-03-04T15:41:42.324 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: cluster 2024-03-04T15:41:41.931406+0000 mon.a (mon.0) 180 : cluster 3 mon.c (rank 2) addr [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] is down (out of quorum) 2024-03-04T15:41:42.324 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: audit 2024-03-04T15:41:41.935166+0000 mon.a (mon.0) 181 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:42.324 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: audit 2024-03-04T15:41:41.939154+0000 mon.a (mon.0) 182 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:42.324 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:41 smithi117 bash[21480]: audit 2024-03-04T15:41:41.944072+0000 mon.a (mon.0) 183 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:42.338 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.333+0000 7f5750f32700 1 -- 172.21.15.151:0/2098865800 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f574c1a3ff0 con 0x7f57380035c0 2024-03-04T15:41:42.339 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.333+0000 7f57437fe700 1 -- 172.21.15.151:0/2098865800 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1287 (secure 0 0 0) 0x7f573c021a00 con 0x7f57380035c0 2024-03-04T15:41:42.339 INFO:teuthology.orchestra.run.smithi151.stdout: 2024-03-04T15:41:42.339 INFO:teuthology.orchestra.run.smithi151.stdout:{"epoch":3,"fsid":"e40b5504-da3c-11ee-95c6-87774f69a715","modified":"2024-03-04T15:41:36.878455Z","created":"2024-03-04T15:39:11.403703Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3300","nonce":0},{"type":"v1","addr":"172.21.15.117:6789","nonce":0}]},"addr":"172.21.15.117:6789/0","public_addr":"172.21.15.117:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:3300","nonce":0},{"type":"v1","addr":"172.21.15.151:6789","nonce":0}]},"addr":"172.21.15.151:6789/0","public_addr":"172.21.15.151:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:3301","nonce":0},{"type":"v1","addr":"172.21.15.117:6790","nonce":0}]},"addr":"172.21.15.117:6790/0","public_addr":"172.21.15.117:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-03-04T15:41:42.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.337+0000 7f5750f32700 1 -- 172.21.15.151:0/2098865800 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f572c0390d0 msgr2=0x7f572c03b590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:42.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.337+0000 7f5750f32700 1 --2- 172.21.15.151:0/2098865800 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f572c0390d0 0x7f572c03b590 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5734009180 tx=0x7f5734008040 comp rx=0 tx=0).stop 2024-03-04T15:41:42.341 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.337+0000 7f5750f32700 1 -- 172.21.15.151:0/2098865800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57380035c0 msgr2=0x7f574c19c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:42.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.337+0000 7f5750f32700 1 --2- 172.21.15.151:0/2098865800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57380035c0 0x7f574c19c6a0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f573c000f80 tx=0x7f573c021760 comp rx=0 tx=0).stop 2024-03-04T15:41:42.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.337+0000 7f5750f32700 1 -- 172.21.15.151:0/2098865800 shutdown_connections 2024-03-04T15:41:42.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.337+0000 7f5750f32700 1 --2- 172.21.15.151:0/2098865800 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f57380035c0 0x7f574c19c6a0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:42.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.337+0000 7f5750f32700 1 --2- 172.21.15.151:0/2098865800 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f572c0390d0 0x7f572c03b590 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:42.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.337+0000 7f5750f32700 1 --2- 172.21.15.151:0/2098865800 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f574c196df0 0x7f574c1972a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:42.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.337+0000 7f5750f32700 1 --2- 172.21.15.151:0/2098865800 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f574c19cbe0 0x7f574c1968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:42.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.337+0000 7f5750f32700 1 -- 172.21.15.151:0/2098865800 >> 172.21.15.151:0/2098865800 conn(0x7f574c0fd3b0 msgr2=0x7f574c1053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:41:42.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.337+0000 7f5750f32700 1 -- 172.21.15.151:0/2098865800 shutdown_connections 2024-03-04T15:41:42.342 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:42.337+0000 7f5750f32700 1 -- 172.21.15.151:0/2098865800 wait complete. 2024-03-04T15:41:42.343 INFO:teuthology.orchestra.run.smithi151.stderr:dumped monmap epoch 3 2024-03-04T15:41:42.354 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: cluster 2024-03-04T15:41:36.716150+0000 mgr.y (mgr.14152) 53 : cluster 0 pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:42.354 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: cluster 2024-03-04T15:41:36.882672+0000 mon.b (mon.1) 2 : cluster 1 mon.b calling monitor election 2024-03-04T15:41:42.354 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: audit 2024-03-04T15:41:36.883818+0000 mon.a (mon.0) 163 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:41:42.354 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: cluster 2024-03-04T15:41:36.884225+0000 mon.a (mon.0) 164 : cluster 1 mon.a calling monitor election 2024-03-04T15:41:42.354 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: audit 2024-03-04T15:41:36.902243+0000 mon.a (mon.0) 165 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:42.354 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: audit 2024-03-04T15:41:36.902454+0000 mon.a (mon.0) 166 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:42.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: audit 2024-03-04T15:41:37.878086+0000 mon.a (mon.0) 167 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:42.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: cluster 2024-03-04T15:41:38.716560+0000 mgr.y (mgr.14152) 54 : cluster 0 pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:42.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: audit 2024-03-04T15:41:38.878231+0000 mon.a (mon.0) 168 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:42.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: audit 2024-03-04T15:41:39.878447+0000 mon.a (mon.0) 169 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:42.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: cluster 2024-03-04T15:41:40.717014+0000 mgr.y (mgr.14152) 55 : cluster 0 pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:42.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: audit 2024-03-04T15:41:40.878510+0000 mon.a (mon.0) 170 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:42.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: audit 2024-03-04T15:41:41.878522+0000 mon.a (mon.0) 171 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:42.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: cluster 2024-03-04T15:41:41.902231+0000 mon.a (mon.0) 172 : cluster 1 mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-04T15:41:42.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: cluster 2024-03-04T15:41:41.908462+0000 mon.a (mon.0) 173 : cluster 0 monmap e3: 3 mons at {a=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],b=[v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0],c=[v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T15:41:42.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: cluster 2024-03-04T15:41:41.925637+0000 mon.a (mon.0) 174 : cluster 0 fsmap 2024-03-04T15:41:42.355 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: cluster 2024-03-04T15:41:41.925690+0000 mon.a (mon.0) 175 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2024-03-04T15:41:42.356 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: cluster 2024-03-04T15:41:41.925904+0000 mon.a (mon.0) 176 : cluster 0 mgrmap e13: y(active, since 79s) 2024-03-04T15:41:42.356 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: cluster 2024-03-04T15:41:41.926131+0000 mon.a (mon.0) 177 : cluster 3 Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-03-04T15:41:42.356 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: cluster 2024-03-04T15:41:41.931332+0000 mon.a (mon.0) 178 : cluster 3 Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-03-04T15:41:42.356 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: cluster 2024-03-04T15:41:41.931377+0000 mon.a (mon.0) 179 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-03-04T15:41:42.356 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: cluster 2024-03-04T15:41:41.931406+0000 mon.a (mon.0) 180 : cluster 3 mon.c (rank 2) addr [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] is down (out of quorum) 2024-03-04T15:41:42.356 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: audit 2024-03-04T15:41:41.935166+0000 mon.a (mon.0) 181 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:42.356 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: audit 2024-03-04T15:41:41.939154+0000 mon.a (mon.0) 182 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:42.356 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:41 smithi151 bash[23756]: audit 2024-03-04T15:41:41.944072+0000 mon.a (mon.0) 183 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:43.084 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-03-04T15:41:43.085 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph config generate-minimal-conf 2024-03-04T15:41:43.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[21480]: audit 2024-03-04T15:41:42.009017+0000 mon.a (mon.0) 184 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:43.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[21480]: audit 2024-03-04T15:41:42.012630+0000 mon.a (mon.0) 185 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:43.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[21480]: audit 2024-03-04T15:41:42.338428+0000 mon.a (mon.0) 186 : audit 0 from='client.? 172.21.15.151:0/2098865800' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T15:41:43.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[21480]: audit 2024-03-04T15:41:42.585610+0000 mon.a (mon.0) 187 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:43.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[21480]: audit 2024-03-04T15:41:42.587340+0000 mon.a (mon.0) 188 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:41:43.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[21480]: cephadm 2024-03-04T15:41:42.589805+0000 mgr.y (mgr.14152) 56 : cephadm 1 Updating smithi117:/etc/ceph/ceph.conf 2024-03-04T15:41:43.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[21480]: cephadm 2024-03-04T15:41:42.590209+0000 mgr.y (mgr.14152) 57 : cephadm 1 Updating smithi151:/etc/ceph/ceph.conf 2024-03-04T15:41:43.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[21480]: audit 2024-03-04T15:41:42.878605+0000 mon.a (mon.0) 189 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:43.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[21480]: audit 2024-03-04T15:41:42.925775+0000 mon.a (mon.0) 190 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:43.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[21480]: audit 2024-03-04T15:41:42.931556+0000 mon.a (mon.0) 191 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:43.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[21480]: audit 2024-03-04T15:41:42.976975+0000 mon.a (mon.0) 192 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:43.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[21480]: audit 2024-03-04T15:41:42.983924+0000 mon.a (mon.0) 193 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:43.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[21480]: audit 2024-03-04T15:41:42.991109+0000 mon.a (mon.0) 194 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:43.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:43 smithi151 bash[23756]: audit 2024-03-04T15:41:42.009017+0000 mon.a (mon.0) 184 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:43.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:43 smithi151 bash[23756]: audit 2024-03-04T15:41:42.012630+0000 mon.a (mon.0) 185 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:43.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:43 smithi151 bash[23756]: audit 2024-03-04T15:41:42.338428+0000 mon.a (mon.0) 186 : audit 0 from='client.? 172.21.15.151:0/2098865800' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T15:41:43.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:43 smithi151 bash[23756]: audit 2024-03-04T15:41:42.585610+0000 mon.a (mon.0) 187 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:43.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:43 smithi151 bash[23756]: audit 2024-03-04T15:41:42.587340+0000 mon.a (mon.0) 188 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:41:43.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:43 smithi151 bash[23756]: cephadm 2024-03-04T15:41:42.589805+0000 mgr.y (mgr.14152) 56 : cephadm 1 Updating smithi117:/etc/ceph/ceph.conf 2024-03-04T15:41:43.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:43 smithi151 bash[23756]: cephadm 2024-03-04T15:41:42.590209+0000 mgr.y (mgr.14152) 57 : cephadm 1 Updating smithi151:/etc/ceph/ceph.conf 2024-03-04T15:41:43.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:43 smithi151 bash[23756]: audit 2024-03-04T15:41:42.878605+0000 mon.a (mon.0) 189 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:43.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:43 smithi151 bash[23756]: audit 2024-03-04T15:41:42.925775+0000 mon.a (mon.0) 190 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:43.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:43 smithi151 bash[23756]: audit 2024-03-04T15:41:42.931556+0000 mon.a (mon.0) 191 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:43.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:43 smithi151 bash[23756]: audit 2024-03-04T15:41:42.976975+0000 mon.a (mon.0) 192 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:43.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:43 smithi151 bash[23756]: audit 2024-03-04T15:41:42.983924+0000 mon.a (mon.0) 193 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:43.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:43 smithi151 bash[23756]: audit 2024-03-04T15:41:42.991109+0000 mon.a (mon.0) 194 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:44.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[21480]: cluster 2024-03-04T15:41:38.880209+0000 mon.c (mon.2) 1 : cluster 1 mon.c calling monitor election 2024-03-04T15:41:44.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[21480]: cluster 2024-03-04T15:41:43.910244+0000 mon.c (mon.2) 2 : cluster 1 mon.c calling monitor election 2024-03-04T15:41:44.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[21480]: cluster 2024-03-04T15:41:43.910979+0000 mon.b (mon.1) 3 : cluster 1 mon.b calling monitor election 2024-03-04T15:41:44.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[21480]: cluster 2024-03-04T15:41:43.911236+0000 mon.a (mon.0) 203 : cluster 1 mon.a calling monitor election 2024-03-04T15:41:44.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[21480]: cluster 2024-03-04T15:41:43.934094+0000 mon.a (mon.0) 204 : cluster 1 mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-04T15:41:44.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[21480]: cluster 2024-03-04T15:41:43.942273+0000 mon.a (mon.0) 205 : cluster 0 monmap e3: 3 mons at {a=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],b=[v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0],c=[v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T15:41:44.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[21480]: cluster 2024-03-04T15:41:43.943262+0000 mon.a (mon.0) 206 : cluster 0 fsmap 2024-03-04T15:41:44.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[21480]: cluster 2024-03-04T15:41:43.943332+0000 mon.a (mon.0) 207 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2024-03-04T15:41:44.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[21480]: cluster 2024-03-04T15:41:43.943625+0000 mon.a (mon.0) 208 : cluster 0 mgrmap e13: y(active, since 81s) 2024-03-04T15:41:44.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[21480]: cluster 2024-03-04T15:41:43.943924+0000 mon.a (mon.0) 209 : cluster 1 Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-03-04T15:41:44.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[21480]: cluster 2024-03-04T15:41:43.943969+0000 mon.a (mon.0) 210 : cluster 1 Cluster is now healthy 2024-03-04T15:41:44.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[21480]: cluster 2024-03-04T15:41:43.949563+0000 mon.a (mon.0) 211 : cluster 1 overall HEALTH_OK 2024-03-04T15:41:44.323 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:30.714944+0000 mgr.y (mgr.14152) 50 : cluster 0 pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:44.323 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:31.505157+0000 mon.a (mon.0) 141 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:41:44.323 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:31.505358+0000 mon.a (mon.0) 142 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:44.323 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:31.505452+0000 mon.a (mon.0) 143 : cluster 1 mon.a calling monitor election 2024-03-04T15:41:44.323 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:32.502866+0000 mon.a (mon.0) 144 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:44.323 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:32.715311+0000 mgr.y (mgr.14152) 51 : cluster 0 pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:44.324 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:33.503246+0000 mon.a (mon.0) 145 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:44.324 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:33.504877+0000 mon.b (mon.1) 1 : cluster 1 mon.b calling monitor election 2024-03-04T15:41:44.324 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:34.503273+0000 mon.a (mon.0) 146 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:44.324 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:34.715730+0000 mgr.y (mgr.14152) 52 : cluster 0 pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:44.324 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:34.876823+0000 mon.a (mon.0) 147 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:44.324 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:35.503580+0000 mon.a (mon.0) 148 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:44.324 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:35.877602+0000 mon.a (mon.0) 149 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:44.324 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:36.503810+0000 mon.a (mon.0) 150 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:44.324 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:36.549184+0000 mon.a (mon.0) 151 : cluster 1 mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-04T15:41:44.324 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:36.559703+0000 mon.a (mon.0) 152 : cluster 0 monmap e2: 2 mons at {a=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],b=[v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T15:41:44.324 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:36.582091+0000 mon.a (mon.0) 153 : cluster 0 fsmap 2024-03-04T15:41:44.325 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:36.582173+0000 mon.a (mon.0) 154 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2024-03-04T15:41:44.325 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:36.582461+0000 mon.a (mon.0) 155 : cluster 0 mgrmap e13: y(active, since 73s) 2024-03-04T15:41:44.325 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:36.582717+0000 mon.a (mon.0) 156 : cluster 1 overall HEALTH_OK 2024-03-04T15:41:44.325 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:36.589520+0000 mon.a (mon.0) 157 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:44.325 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:36.597192+0000 mon.a (mon.0) 158 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:44.325 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:36.615660+0000 mon.a (mon.0) 159 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:44.325 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:36.622826+0000 mon.a (mon.0) 160 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:44.325 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:36.716150+0000 mgr.y (mgr.14152) 53 : cluster 0 pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:44.325 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:36.882672+0000 mon.b (mon.1) 2 : cluster 1 mon.b calling monitor election 2024-03-04T15:41:44.325 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:36.883818+0000 mon.a (mon.0) 163 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:41:44.325 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:36.884225+0000 mon.a (mon.0) 164 : cluster 1 mon.a calling monitor election 2024-03-04T15:41:44.325 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:36.902243+0000 mon.a (mon.0) 165 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:41:44.325 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:36.902454+0000 mon.a (mon.0) 166 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:44.325 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:37.878086+0000 mon.a (mon.0) 167 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:44.325 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:38.716560+0000 mgr.y (mgr.14152) 54 : cluster 0 pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:44.325 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:38.878231+0000 mon.a (mon.0) 168 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:44.326 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:39.878447+0000 mon.a (mon.0) 169 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:44.326 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:40.717014+0000 mgr.y (mgr.14152) 55 : cluster 0 pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:44.326 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:40.878510+0000 mon.a (mon.0) 170 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:44.326 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:41.878522+0000 mon.a (mon.0) 171 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:44.326 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:41.902231+0000 mon.a (mon.0) 172 : cluster 1 mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-03-04T15:41:44.326 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:41.908462+0000 mon.a (mon.0) 173 : cluster 0 monmap e3: 3 mons at {a=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],b=[v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0],c=[v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T15:41:44.326 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:41.925637+0000 mon.a (mon.0) 174 : cluster 0 fsmap 2024-03-04T15:41:44.326 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:41.925690+0000 mon.a (mon.0) 175 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2024-03-04T15:41:44.326 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:41.925904+0000 mon.a (mon.0) 176 : cluster 0 mgrmap e13: y(active, since 79s) 2024-03-04T15:41:44.326 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:41.926131+0000 mon.a (mon.0) 177 : cluster 3 Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-03-04T15:41:44.326 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:41.931332+0000 mon.a (mon.0) 178 : cluster 3 Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-03-04T15:41:44.327 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:41.931377+0000 mon.a (mon.0) 179 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-03-04T15:41:44.327 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cluster 2024-03-04T15:41:41.931406+0000 mon.a (mon.0) 180 : cluster 3 mon.c (rank 2) addr [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] is down (out of quorum) 2024-03-04T15:41:44.327 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:41.935166+0000 mon.a (mon.0) 181 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:44.327 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:41.939154+0000 mon.a (mon.0) 182 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:44.327 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:41.944072+0000 mon.a (mon.0) 183 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:44.327 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:42.009017+0000 mon.a (mon.0) 184 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:44.327 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:42.012630+0000 mon.a (mon.0) 185 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:44.327 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:42.338428+0000 mon.a (mon.0) 186 : audit 0 from='client.? 172.21.15.151:0/2098865800' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-03-04T15:41:44.327 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:42.585610+0000 mon.a (mon.0) 187 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:44.327 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:42.587340+0000 mon.a (mon.0) 188 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:41:44.327 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cephadm 2024-03-04T15:41:42.589805+0000 mgr.y (mgr.14152) 56 : cephadm 1 Updating smithi117:/etc/ceph/ceph.conf 2024-03-04T15:41:44.327 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: cephadm 2024-03-04T15:41:42.590209+0000 mgr.y (mgr.14152) 57 : cephadm 1 Updating smithi151:/etc/ceph/ceph.conf 2024-03-04T15:41:44.327 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:42.878605+0000 mon.a (mon.0) 189 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:44.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:42.925775+0000 mon.a (mon.0) 190 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:44.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:42.931556+0000 mon.a (mon.0) 191 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:44.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:42.976975+0000 mon.a (mon.0) 192 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:44.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:42.983924+0000 mon.a (mon.0) 193 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:44.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:43 smithi117 bash[28843]: audit 2024-03-04T15:41:42.991109+0000 mon.a (mon.0) 194 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:44.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[28843]: cluster 2024-03-04T15:41:38.880209+0000 mon.c (mon.2) 1 : cluster 1 mon.c calling monitor election 2024-03-04T15:41:44.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[28843]: cluster 2024-03-04T15:41:43.910244+0000 mon.c (mon.2) 2 : cluster 1 mon.c calling monitor election 2024-03-04T15:41:44.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[28843]: cluster 2024-03-04T15:41:43.910979+0000 mon.b (mon.1) 3 : cluster 1 mon.b calling monitor election 2024-03-04T15:41:44.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[28843]: cluster 2024-03-04T15:41:43.911236+0000 mon.a (mon.0) 203 : cluster 1 mon.a calling monitor election 2024-03-04T15:41:44.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[28843]: cluster 2024-03-04T15:41:43.934094+0000 mon.a (mon.0) 204 : cluster 1 mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-04T15:41:44.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[28843]: cluster 2024-03-04T15:41:43.942273+0000 mon.a (mon.0) 205 : cluster 0 monmap e3: 3 mons at {a=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],b=[v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0],c=[v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T15:41:44.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[28843]: cluster 2024-03-04T15:41:43.943262+0000 mon.a (mon.0) 206 : cluster 0 fsmap 2024-03-04T15:41:44.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[28843]: cluster 2024-03-04T15:41:43.943332+0000 mon.a (mon.0) 207 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2024-03-04T15:41:44.328 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[28843]: cluster 2024-03-04T15:41:43.943625+0000 mon.a (mon.0) 208 : cluster 0 mgrmap e13: y(active, since 81s) 2024-03-04T15:41:44.329 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[28843]: cluster 2024-03-04T15:41:43.943924+0000 mon.a (mon.0) 209 : cluster 1 Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-03-04T15:41:44.329 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[28843]: cluster 2024-03-04T15:41:43.943969+0000 mon.a (mon.0) 210 : cluster 1 Cluster is now healthy 2024-03-04T15:41:44.329 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:44 smithi117 bash[28843]: cluster 2024-03-04T15:41:43.949563+0000 mon.a (mon.0) 211 : cluster 1 overall HEALTH_OK 2024-03-04T15:41:44.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:44 smithi151 bash[23756]: cluster 2024-03-04T15:41:38.880209+0000 mon.c (mon.2) 1 : cluster 1 mon.c calling monitor election 2024-03-04T15:41:44.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:44 smithi151 bash[23756]: cluster 2024-03-04T15:41:43.910244+0000 mon.c (mon.2) 2 : cluster 1 mon.c calling monitor election 2024-03-04T15:41:44.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:44 smithi151 bash[23756]: cluster 2024-03-04T15:41:43.910979+0000 mon.b (mon.1) 3 : cluster 1 mon.b calling monitor election 2024-03-04T15:41:44.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:44 smithi151 bash[23756]: cluster 2024-03-04T15:41:43.911236+0000 mon.a (mon.0) 203 : cluster 1 mon.a calling monitor election 2024-03-04T15:41:44.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:44 smithi151 bash[23756]: cluster 2024-03-04T15:41:43.934094+0000 mon.a (mon.0) 204 : cluster 1 mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-03-04T15:41:44.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:44 smithi151 bash[23756]: cluster 2024-03-04T15:41:43.942273+0000 mon.a (mon.0) 205 : cluster 0 monmap e3: 3 mons at {a=[v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0],b=[v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0],c=[v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-03-04T15:41:44.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:44 smithi151 bash[23756]: cluster 2024-03-04T15:41:43.943262+0000 mon.a (mon.0) 206 : cluster 0 fsmap 2024-03-04T15:41:44.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:44 smithi151 bash[23756]: cluster 2024-03-04T15:41:43.943332+0000 mon.a (mon.0) 207 : cluster 0 osdmap e4: 0 total, 0 up, 0 in 2024-03-04T15:41:44.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:44 smithi151 bash[23756]: cluster 2024-03-04T15:41:43.943625+0000 mon.a (mon.0) 208 : cluster 0 mgrmap e13: y(active, since 81s) 2024-03-04T15:41:44.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:44 smithi151 bash[23756]: cluster 2024-03-04T15:41:43.943924+0000 mon.a (mon.0) 209 : cluster 1 Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-03-04T15:41:44.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:44 smithi151 bash[23756]: cluster 2024-03-04T15:41:43.943969+0000 mon.a (mon.0) 210 : cluster 1 Cluster is now healthy 2024-03-04T15:41:44.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:44 smithi151 bash[23756]: cluster 2024-03-04T15:41:43.949563+0000 mon.a (mon.0) 211 : cluster 1 overall HEALTH_OK 2024-03-04T15:41:46.118 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:45 smithi117 bash[21480]: cluster 2024-03-04T15:41:44.717664+0000 mgr.y (mgr.14152) 63 : cluster 0 pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:46.119 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:45 smithi117 bash[21480]: audit 2024-03-04T15:41:44.879300+0000 mon.a (mon.0) 212 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:46.119 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:45 smithi117 bash[21480]: audit 2024-03-04T15:41:45.064338+0000 mon.a (mon.0) 213 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:46.119 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:45 smithi117 bash[21480]: audit 2024-03-04T15:41:45.069764+0000 mon.a (mon.0) 214 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:46.119 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:45 smithi117 bash[21480]: cephadm 2024-03-04T15:41:45.103498+0000 mgr.y (mgr.14152) 64 : cephadm 1 Reconfiguring mon.c (monmap changed)... 2024-03-04T15:41:46.119 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:45 smithi117 bash[21480]: audit 2024-03-04T15:41:45.104155+0000 mon.a (mon.0) 215 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T15:41:46.119 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:45 smithi117 bash[21480]: audit 2024-03-04T15:41:45.105708+0000 mon.a (mon.0) 216 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T15:41:46.120 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:45 smithi117 bash[21480]: audit 2024-03-04T15:41:45.107098+0000 mon.a (mon.0) 217 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:46.120 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:45 smithi117 bash[21480]: cephadm 2024-03-04T15:41:45.108605+0000 mgr.y (mgr.14152) 65 : cephadm 1 Reconfiguring daemon mon.c on smithi117 2024-03-04T15:41:46.120 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:45 smithi117 bash[28843]: cluster 2024-03-04T15:41:44.717664+0000 mgr.y (mgr.14152) 63 : cluster 0 pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:46.120 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:45 smithi117 bash[28843]: audit 2024-03-04T15:41:44.879300+0000 mon.a (mon.0) 212 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:46.120 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:45 smithi117 bash[28843]: audit 2024-03-04T15:41:45.064338+0000 mon.a (mon.0) 213 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:46.120 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:45 smithi117 bash[28843]: audit 2024-03-04T15:41:45.069764+0000 mon.a (mon.0) 214 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:46.120 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:45 smithi117 bash[28843]: cephadm 2024-03-04T15:41:45.103498+0000 mgr.y (mgr.14152) 64 : cephadm 1 Reconfiguring mon.c (monmap changed)... 2024-03-04T15:41:46.120 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:45 smithi117 bash[28843]: audit 2024-03-04T15:41:45.104155+0000 mon.a (mon.0) 215 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T15:41:46.121 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:45 smithi117 bash[28843]: audit 2024-03-04T15:41:45.105708+0000 mon.a (mon.0) 216 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T15:41:46.121 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:45 smithi117 bash[28843]: audit 2024-03-04T15:41:45.107098+0000 mon.a (mon.0) 217 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:46.121 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:45 smithi117 bash[28843]: cephadm 2024-03-04T15:41:45.108605+0000 mgr.y (mgr.14152) 65 : cephadm 1 Reconfiguring daemon mon.c on smithi117 2024-03-04T15:41:46.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:45 smithi151 bash[23756]: cluster 2024-03-04T15:41:44.717664+0000 mgr.y (mgr.14152) 63 : cluster 0 pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:46.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:45 smithi151 bash[23756]: audit 2024-03-04T15:41:44.879300+0000 mon.a (mon.0) 212 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:41:46.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:45 smithi151 bash[23756]: audit 2024-03-04T15:41:45.064338+0000 mon.a (mon.0) 213 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:46.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:45 smithi151 bash[23756]: audit 2024-03-04T15:41:45.069764+0000 mon.a (mon.0) 214 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:46.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:45 smithi151 bash[23756]: cephadm 2024-03-04T15:41:45.103498+0000 mgr.y (mgr.14152) 64 : cephadm 1 Reconfiguring mon.c (monmap changed)... 2024-03-04T15:41:46.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:45 smithi151 bash[23756]: audit 2024-03-04T15:41:45.104155+0000 mon.a (mon.0) 215 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T15:41:46.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:45 smithi151 bash[23756]: audit 2024-03-04T15:41:45.105708+0000 mon.a (mon.0) 216 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T15:41:46.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:45 smithi151 bash[23756]: audit 2024-03-04T15:41:45.107098+0000 mon.a (mon.0) 217 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:46.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:45 smithi151 bash[23756]: cephadm 2024-03-04T15:41:45.108605+0000 mgr.y (mgr.14152) 65 : cephadm 1 Reconfiguring daemon mon.c on smithi117 2024-03-04T15:41:47.794 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:41:48.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:47 smithi117 bash[21480]: cluster 2024-03-04T15:41:46.718077+0000 mgr.y (mgr.14152) 66 : cluster 0 pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:48.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:47 smithi117 bash[21480]: audit 2024-03-04T15:41:46.799557+0000 mon.a (mon.0) 218 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:48.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:47 smithi117 bash[21480]: audit 2024-03-04T15:41:46.807496+0000 mon.a (mon.0) 219 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:48.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:47 smithi117 bash[21480]: cephadm 2024-03-04T15:41:46.826236+0000 mgr.y (mgr.14152) 67 : cephadm 1 Reconfiguring mon.b (monmap changed)... 2024-03-04T15:41:48.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:47 smithi117 bash[21480]: audit 2024-03-04T15:41:46.826840+0000 mon.a (mon.0) 220 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T15:41:48.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:47 smithi117 bash[21480]: audit 2024-03-04T15:41:46.828327+0000 mon.a (mon.0) 221 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T15:41:48.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:47 smithi117 bash[21480]: audit 2024-03-04T15:41:46.829581+0000 mon.a (mon.0) 222 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:48.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:47 smithi117 bash[21480]: cephadm 2024-03-04T15:41:46.831430+0000 mgr.y (mgr.14152) 68 : cephadm 1 Reconfiguring daemon mon.b on smithi151 2024-03-04T15:41:48.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:47 smithi117 bash[28843]: cluster 2024-03-04T15:41:46.718077+0000 mgr.y (mgr.14152) 66 : cluster 0 pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:48.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:47 smithi117 bash[28843]: audit 2024-03-04T15:41:46.799557+0000 mon.a (mon.0) 218 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:48.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:47 smithi117 bash[28843]: audit 2024-03-04T15:41:46.807496+0000 mon.a (mon.0) 219 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:48.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:47 smithi117 bash[28843]: cephadm 2024-03-04T15:41:46.826236+0000 mgr.y (mgr.14152) 67 : cephadm 1 Reconfiguring mon.b (monmap changed)... 2024-03-04T15:41:48.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:47 smithi117 bash[28843]: audit 2024-03-04T15:41:46.826840+0000 mon.a (mon.0) 220 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T15:41:48.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:47 smithi117 bash[28843]: audit 2024-03-04T15:41:46.828327+0000 mon.a (mon.0) 221 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T15:41:48.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:47 smithi117 bash[28843]: audit 2024-03-04T15:41:46.829581+0000 mon.a (mon.0) 222 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:48.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:47 smithi117 bash[28843]: cephadm 2024-03-04T15:41:46.831430+0000 mgr.y (mgr.14152) 68 : cephadm 1 Reconfiguring daemon mon.b on smithi151 2024-03-04T15:41:48.079 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:47 smithi151 bash[23756]: cluster 2024-03-04T15:41:46.718077+0000 mgr.y (mgr.14152) 66 : cluster 0 pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:48.080 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:47 smithi151 bash[23756]: audit 2024-03-04T15:41:46.799557+0000 mon.a (mon.0) 218 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:48.080 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:47 smithi151 bash[23756]: audit 2024-03-04T15:41:46.807496+0000 mon.a (mon.0) 219 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:48.080 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:47 smithi151 bash[23756]: cephadm 2024-03-04T15:41:46.826236+0000 mgr.y (mgr.14152) 67 : cephadm 1 Reconfiguring mon.b (monmap changed)... 2024-03-04T15:41:48.080 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:47 smithi151 bash[23756]: audit 2024-03-04T15:41:46.826840+0000 mon.a (mon.0) 220 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-03-04T15:41:48.080 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:47 smithi151 bash[23756]: audit 2024-03-04T15:41:46.828327+0000 mon.a (mon.0) 221 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-03-04T15:41:48.080 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:47 smithi151 bash[23756]: audit 2024-03-04T15:41:46.829581+0000 mon.a (mon.0) 222 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:48.080 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:47 smithi151 bash[23756]: cephadm 2024-03-04T15:41:46.831430+0000 mgr.y (mgr.14152) 68 : cephadm 1 Reconfiguring daemon mon.b on smithi151 2024-03-04T15:41:49.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:49 smithi151 bash[23756]: audit 2024-03-04T15:41:48.653133+0000 mon.a (mon.0) 223 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:49.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:49 smithi151 bash[23756]: audit 2024-03-04T15:41:48.658588+0000 mon.a (mon.0) 224 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:49.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:49 smithi151 bash[23756]: audit 2024-03-04T15:41:48.701920+0000 mon.a (mon.0) 225 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:41:49.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:49 smithi151 bash[23756]: audit 2024-03-04T15:41:48.703914+0000 mon.a (mon.0) 226 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:49.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:49 smithi151 bash[23756]: audit 2024-03-04T15:41:48.704728+0000 mon.a (mon.0) 227 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:41:49.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:49 smithi151 bash[23756]: audit 2024-03-04T15:41:48.710901+0000 mon.a (mon.0) 228 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:49.967 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:49 smithi117 bash[21480]: audit 2024-03-04T15:41:48.653133+0000 mon.a (mon.0) 223 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:49.967 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:49 smithi117 bash[21480]: audit 2024-03-04T15:41:48.658588+0000 mon.a (mon.0) 224 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:49.967 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:49 smithi117 bash[21480]: audit 2024-03-04T15:41:48.701920+0000 mon.a (mon.0) 225 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:41:49.967 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:49 smithi117 bash[21480]: audit 2024-03-04T15:41:48.703914+0000 mon.a (mon.0) 226 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:49.967 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:49 smithi117 bash[21480]: audit 2024-03-04T15:41:48.704728+0000 mon.a (mon.0) 227 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:41:49.967 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:49 smithi117 bash[21480]: audit 2024-03-04T15:41:48.710901+0000 mon.a (mon.0) 228 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:49.968 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:49 smithi117 bash[28843]: audit 2024-03-04T15:41:48.653133+0000 mon.a (mon.0) 223 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:49.968 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:49 smithi117 bash[28843]: audit 2024-03-04T15:41:48.658588+0000 mon.a (mon.0) 224 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:49.968 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:49 smithi117 bash[28843]: audit 2024-03-04T15:41:48.701920+0000 mon.a (mon.0) 225 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:41:49.968 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:49 smithi117 bash[28843]: audit 2024-03-04T15:41:48.703914+0000 mon.a (mon.0) 226 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:49.968 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:49 smithi117 bash[28843]: audit 2024-03-04T15:41:48.704728+0000 mon.a (mon.0) 227 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:41:49.968 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:49 smithi117 bash[28843]: audit 2024-03-04T15:41:48.710901+0000 mon.a (mon.0) 228 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:50.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63fd768700 1 -- 172.21.15.117:0/2716221990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f8074930 msgr2=0x7f63f810e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:50.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63fd768700 1 --2- 172.21.15.117:0/2716221990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f8074930 0x7f63f810e330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f63e00098f0 tx=0x7f63e0009c00 comp rx=0 tx=0).stop 2024-03-04T15:41:50.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63fd768700 1 -- 172.21.15.117:0/2716221990 shutdown_connections 2024-03-04T15:41:50.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63fd768700 1 --2- 172.21.15.117:0/2716221990 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f810e870 0x7f63f8110c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:50.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63fd768700 1 --2- 172.21.15.117:0/2716221990 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63f8074010 0x7f63f80743f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:50.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63fd768700 1 --2- 172.21.15.117:0/2716221990 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f8074930 0x7f63f810e330 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:50.106 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63fd768700 1 -- 172.21.15.117:0/2716221990 >> 172.21.15.117:0/2716221990 conn(0x7f63f8078eb0 msgr2=0x7f63f80792c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:41:50.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63fd768700 1 -- 172.21.15.117:0/2716221990 shutdown_connections 2024-03-04T15:41:50.107 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63fd768700 1 -- 172.21.15.117:0/2716221990 wait complete. 2024-03-04T15:41:50.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63fd768700 1 Processor -- start 2024-03-04T15:41:50.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63fd768700 1 -- start start 2024-03-04T15:41:50.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63fd768700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f8074010 0x7f63f8199e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:41:50.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63fd768700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63f8074930 0x7f63f819a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:41:50.108 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63fd768700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f810e870 0x7f63f819e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:41:50.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63fd768700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f8113820 con 0x7f63f810e870 2024-03-04T15:41:50.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63fd768700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f81136a0 con 0x7f63f8074930 2024-03-04T15:41:50.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63fd768700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f81139a0 con 0x7f63f8074010 2024-03-04T15:41:50.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63f77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f810e870 0x7f63f819e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:41:50.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63f6ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f8074010 0x7f63f8199e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:41:50.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63f77fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f810e870 0x7f63f819e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:58618/0 (socket says 172.21.15.117:58618) 2024-03-04T15:41:50.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63f77fe700 1 -- 172.21.15.117:0/1599768506 learned_addr learned my addr 172.21.15.117:0/1599768506 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:41:50.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63ee7fc700 1 --2- 172.21.15.117:0/1599768506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63f8074930 0x7f63f819a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:41:50.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63f77fe700 1 -- 172.21.15.117:0/1599768506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f8074010 msgr2=0x7f63f8199e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:50.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63f77fe700 1 --2- 172.21.15.117:0/1599768506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f8074010 0x7f63f8199e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:50.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63f77fe700 1 -- 172.21.15.117:0/1599768506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63f8074930 msgr2=0x7f63f819a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:50.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63f77fe700 1 --2- 172.21.15.117:0/1599768506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63f8074930 0x7f63f819a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:50.109 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63f77fe700 1 -- 172.21.15.117:0/1599768506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f819ee10 con 0x7f63f810e870 2024-03-04T15:41:50.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63f6ffd700 1 --2- 172.21.15.117:0/1599768506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f8074010 0x7f63f8199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:41:50.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63f77fe700 1 --2- 172.21.15.117:0/1599768506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f810e870 0x7f63f819e6d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f63f000c3d0 tx=0x7f63f000c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:41:50.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63f4ff9700 1 -- 172.21.15.117:0/1599768506 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f00090d0 con 0x7f63f810e870 2024-03-04T15:41:50.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63f4ff9700 1 -- 172.21.15.117:0/1599768506 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f63f0009270 con 0x7f63f810e870 2024-03-04T15:41:50.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63fd768700 1 -- 172.21.15.117:0/1599768506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f819f100 con 0x7f63f810e870 2024-03-04T15:41:50.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63f4ff9700 1 -- 172.21.15.117:0/1599768506 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f0004730 con 0x7f63f810e870 2024-03-04T15:41:50.110 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.104+0000 7f63fd768700 1 -- 172.21.15.117:0/1599768506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f81a6980 con 0x7f63f810e870 2024-03-04T15:41:50.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.108+0000 7f63f4ff9700 1 -- 172.21.15.117:0/1599768506 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7f63f0012070 con 0x7f63f810e870 2024-03-04T15:41:50.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.108+0000 7f63fd768700 1 -- 172.21.15.117:0/1599768506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f804faa0 con 0x7f63f810e870 2024-03-04T15:41:50.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.108+0000 7f63f4ff9700 1 --2- 172.21.15.117:0/1599768506 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f63e4038e80 0x7f63e403b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:41:50.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.108+0000 7f63f4ff9700 1 -- 172.21.15.117:0/1599768506 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f63f004b880 con 0x7f63f810e870 2024-03-04T15:41:50.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.108+0000 7f63f6ffd700 1 --2- 172.21.15.117:0/1599768506 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f63e4038e80 0x7f63e403b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:41:50.111 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.108+0000 7f63f6ffd700 1 --2- 172.21.15.117:0/1599768506 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f63e4038e80 0x7f63e403b340 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f63e8005950 tx=0x7f63e800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:41:50.114 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.112+0000 7f63f4ff9700 1 -- 172.21.15.117:0/1599768506 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f63f0010040 con 0x7f63f810e870 2024-03-04T15:41:50.313 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.312+0000 7f63fd768700 1 -- 172.21.15.117:0/1599768506 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f63f819b280 con 0x7f63f810e870 2024-03-04T15:41:50.314 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.312+0000 7f63f4ff9700 1 -- 172.21.15.117:0/1599768506 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7f63f00194f0 con 0x7f63f810e870 2024-03-04T15:41:50.314 INFO:teuthology.orchestra.run.smithi117.stdout:# minimal ceph.conf for e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T15:41:50.314 INFO:teuthology.orchestra.run.smithi117.stdout:[global] 2024-03-04T15:41:50.314 INFO:teuthology.orchestra.run.smithi117.stdout: fsid = e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T15:41:50.314 INFO:teuthology.orchestra.run.smithi117.stdout: mon_host = [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] 2024-03-04T15:41:50.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.312+0000 7f63fd768700 1 -- 172.21.15.117:0/1599768506 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f63e4038e80 msgr2=0x7f63e403b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:50.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.312+0000 7f63fd768700 1 --2- 172.21.15.117:0/1599768506 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f63e4038e80 0x7f63e403b340 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f63e8005950 tx=0x7f63e800b410 comp rx=0 tx=0).stop 2024-03-04T15:41:50.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.312+0000 7f63fd768700 1 -- 172.21.15.117:0/1599768506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f810e870 msgr2=0x7f63f819e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:50.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.312+0000 7f63fd768700 1 --2- 172.21.15.117:0/1599768506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f810e870 0x7f63f819e6d0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f63f000c3d0 tx=0x7f63f000c6e0 comp rx=0 tx=0).stop 2024-03-04T15:41:50.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.312+0000 7f63fd768700 1 -- 172.21.15.117:0/1599768506 shutdown_connections 2024-03-04T15:41:50.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.312+0000 7f63fd768700 1 --2- 172.21.15.117:0/1599768506 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f63f810e870 0x7f63f819e6d0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:50.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.312+0000 7f63fd768700 1 --2- 172.21.15.117:0/1599768506 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f63e4038e80 0x7f63e403b340 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:50.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.312+0000 7f63fd768700 1 --2- 172.21.15.117:0/1599768506 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f63f8074010 0x7f63f8199e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:50.317 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.312+0000 7f63fd768700 1 --2- 172.21.15.117:0/1599768506 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f63f8074930 0x7f63f819a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:50.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.312+0000 7f63fd768700 1 -- 172.21.15.117:0/1599768506 >> 172.21.15.117:0/1599768506 conn(0x7f63f8078eb0 msgr2=0x7f63f80ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:41:50.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.316+0000 7f63fd768700 1 -- 172.21.15.117:0/1599768506 shutdown_connections 2024-03-04T15:41:50.318 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:41:50.316+0000 7f63fd768700 1 -- 172.21.15.117:0/1599768506 wait complete. 2024-03-04T15:41:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:50 smithi151 bash[23756]: cluster 2024-03-04T15:41:48.718377+0000 mgr.y (mgr.14152) 69 : cluster 0 pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:50 smithi151 bash[23756]: audit 2024-03-04T15:41:50.313975+0000 mon.a (mon.0) 229 : audit 0 from='client.? 172.21.15.117:0/1599768506' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:50.979 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:50 smithi117 bash[21480]: cluster 2024-03-04T15:41:48.718377+0000 mgr.y (mgr.14152) 69 : cluster 0 pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:50.979 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:50 smithi117 bash[21480]: audit 2024-03-04T15:41:50.313975+0000 mon.a (mon.0) 229 : audit 0 from='client.? 172.21.15.117:0/1599768506' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:50.979 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:50 smithi117 bash[28843]: cluster 2024-03-04T15:41:48.718377+0000 mgr.y (mgr.14152) 69 : cluster 0 pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:50.980 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:50 smithi117 bash[28843]: audit 2024-03-04T15:41:50.313975+0000 mon.a (mon.0) 229 : audit 0 from='client.? 172.21.15.117:0/1599768506' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:50.980 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-03-04T15:41:50.981 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-04T15:41:50.981 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.conf 2024-03-04T15:41:50.996 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-04T15:41:50.996 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-04T15:41:51.052 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-04T15:41:51.052 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.conf 2024-03-04T15:41:51.067 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-04T15:41:51.067 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-03-04T15:41:51.124 INFO:tasks.cephadm:Adding mgr.y on smithi117 2024-03-04T15:41:51.124 INFO:tasks.cephadm:Adding mgr.x on smithi151 2024-03-04T15:41:51.124 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph orch apply mgr '2;smithi117=y;smithi151=x' 2024-03-04T15:41:52.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:52 smithi151 bash[23756]: cluster 2024-03-04T15:41:50.718736+0000 mgr.y (mgr.14152) 70 : cluster 0 pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:53.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:52 smithi117 bash[21480]: cluster 2024-03-04T15:41:50.718736+0000 mgr.y (mgr.14152) 70 : cluster 0 pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:53.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:52 smithi117 bash[28843]: cluster 2024-03-04T15:41:50.718736+0000 mgr.y (mgr.14152) 70 : cluster 0 pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:54.865 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T15:41:54.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:54 smithi151 bash[23756]: cluster 2024-03-04T15:41:52.719174+0000 mgr.y (mgr.14152) 71 : cluster 0 pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:55.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:54 smithi117 bash[21480]: cluster 2024-03-04T15:41:52.719174+0000 mgr.y (mgr.14152) 71 : cluster 0 pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:55.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:54 smithi117 bash[28843]: cluster 2024-03-04T15:41:52.719174+0000 mgr.y (mgr.14152) 71 : cluster 0 pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:56.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:56 smithi151 bash[23756]: cluster 2024-03-04T15:41:54.719572+0000 mgr.y (mgr.14152) 72 : cluster 0 pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:57.035 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe719e0700 1 -- 172.21.15.151:0/6919114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe6c102900 msgr2=0x7efe6c104cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:57.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe719e0700 1 --2- 172.21.15.151:0/6919114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe6c102900 0x7efe6c104cf0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7efe5c009990 tx=0x7efe5c009ca0 comp rx=0 tx=0).stop 2024-03-04T15:41:57.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe719e0700 1 -- 172.21.15.151:0/6919114 shutdown_connections 2024-03-04T15:41:57.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe719e0700 1 --2- 172.21.15.151:0/6919114 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe6c107b60 0x7efe6c109fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:57.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe719e0700 1 --2- 172.21.15.151:0/6919114 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe6c102900 0x7efe6c104cf0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:57.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe719e0700 1 --2- 172.21.15.151:0/6919114 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe6c105230 0x7efe6c107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:57.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe719e0700 1 -- 172.21.15.151:0/6919114 >> 172.21.15.151:0/6919114 conn(0x7efe6c0fc310 msgr2=0x7efe6c0fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:41:57.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe719e0700 1 -- 172.21.15.151:0/6919114 shutdown_connections 2024-03-04T15:41:57.036 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe719e0700 1 -- 172.21.15.151:0/6919114 wait complete. 2024-03-04T15:41:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe719e0700 1 Processor -- start 2024-03-04T15:41:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe719e0700 1 -- start start 2024-03-04T15:41:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe719e0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe6c102900 0x7efe6c1974c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:41:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe719e0700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe6c105230 0x7efe6c197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:41:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe719e0700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe6c107b60 0x7efe6c19ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:41:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe719e0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c10cec0 con 0x7efe6c102900 2024-03-04T15:41:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe719e0700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c10cd40 con 0x7efe6c107b60 2024-03-04T15:41:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe719e0700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe6c10d040 con 0x7efe6c105230 2024-03-04T15:41:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe6b7fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe6c107b60 0x7efe6c19ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:41:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe6b7fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe6c107b60 0x7efe6c19ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44648/0 (socket says 172.21.15.151:44648) 2024-03-04T15:41:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe6b7fe700 1 -- 172.21.15.151:0/3977276128 learned_addr learned my addr 172.21.15.151:0/3977276128 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-04T15:41:57.038 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe6affd700 1 --2- 172.21.15.151:0/3977276128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe6c102900 0x7efe6c1974c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:41:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe6a7fc700 1 --2- 172.21.15.151:0/3977276128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe6c105230 0x7efe6c197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:41:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe6b7fe700 1 -- 172.21.15.151:0/3977276128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe6c105230 msgr2=0x7efe6c197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe6b7fe700 1 --2- 172.21.15.151:0/3977276128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe6c105230 0x7efe6c197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe6b7fe700 1 -- 172.21.15.151:0/3977276128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe6c102900 msgr2=0x7efe6c1974c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe6b7fe700 1 --2- 172.21.15.151:0/3977276128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe6c102900 0x7efe6c1974c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe6b7fe700 1 -- 172.21.15.151:0/3977276128 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe6c1006f0 con 0x7efe6c107b60 2024-03-04T15:41:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe6affd700 1 --2- 172.21.15.151:0/3977276128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe6c102900 0x7efe6c1974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T15:41:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe6b7fe700 1 --2- 172.21.15.151:0/3977276128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe6c107b60 0x7efe6c19ea80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efe64007d40 tx=0x7efe6400a340 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:41:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe709de700 1 -- 172.21.15.151:0/3977276128 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe64016020 con 0x7efe6c107b60 2024-03-04T15:41:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe709de700 1 -- 172.21.15.151:0/3977276128 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7efe6400ad40 con 0x7efe6c107b60 2024-03-04T15:41:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe709de700 1 -- 172.21.15.151:0/3977276128 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe640115e0 con 0x7efe6c107b60 2024-03-04T15:41:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe719e0700 1 -- 172.21.15.151:0/3977276128 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe6c1009e0 con 0x7efe6c107b60 2024-03-04T15:41:57.039 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe719e0700 1 -- 172.21.15.151:0/3977276128 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe6c100ed0 con 0x7efe6c107b60 2024-03-04T15:41:57.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe709de700 1 -- 172.21.15.151:0/3977276128 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 13) v1 ==== 45984+0+0 (secure 0 0 0) 0x7efe64019020 con 0x7efe6c107b60 2024-03-04T15:41:57.040 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe719e0700 1 -- 172.21.15.151:0/3977276128 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe6c04faa0 con 0x7efe6c107b60 2024-03-04T15:41:57.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe709de700 1 --2- 172.21.15.151:0/3977276128 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7efe58038e80 0x7efe5803b340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:41:57.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe709de700 1 -- 172.21.15.151:0/3977276128 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efe6404be00 con 0x7efe6c107b60 2024-03-04T15:41:57.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.033+0000 7efe6affd700 1 --2- 172.21.15.151:0/3977276128 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7efe58038e80 0x7efe5803b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:41:57.041 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.037+0000 7efe6affd700 1 --2- 172.21.15.151:0/3977276128 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7efe58038e80 0x7efe5803b340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00b5c0 tx=0x7efe5c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:41:57.044 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.041+0000 7efe709de700 1 -- 172.21.15.151:0/3977276128 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7efe64020020 con 0x7efe6c107b60 2024-03-04T15:41:57.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:56 smithi117 bash[21480]: cluster 2024-03-04T15:41:54.719572+0000 mgr.y (mgr.14152) 72 : cluster 0 pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:57.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:56 smithi117 bash[28843]: cluster 2024-03-04T15:41:54.719572+0000 mgr.y (mgr.14152) 72 : cluster 0 pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:57.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.253+0000 7efe719e0700 1 -- 172.21.15.151:0/3977276128 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi117=y;smithi151=x", "target": ["mon-mgr", ""]}) v1 -- 0x7efe6c062220 con 0x7efe58038e80 2024-03-04T15:41:57.271 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.265+0000 7efe709de700 1 -- 172.21.15.151:0/3977276128 <== mgr.14152 v2:172.21.15.117:6800/2865911351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7efe6c062220 con 0x7efe58038e80 2024-03-04T15:41:57.271 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled mgr update... 2024-03-04T15:41:57.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.269+0000 7efe719e0700 1 -- 172.21.15.151:0/3977276128 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7efe58038e80 msgr2=0x7efe5803b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:57.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.269+0000 7efe719e0700 1 --2- 172.21.15.151:0/3977276128 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7efe58038e80 0x7efe5803b340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efe5c00b5c0 tx=0x7efe5c01a040 comp rx=0 tx=0).stop 2024-03-04T15:41:57.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.269+0000 7efe719e0700 1 -- 172.21.15.151:0/3977276128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe6c107b60 msgr2=0x7efe6c19ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:41:57.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.269+0000 7efe719e0700 1 --2- 172.21.15.151:0/3977276128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe6c107b60 0x7efe6c19ea80 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7efe64007d40 tx=0x7efe6400a340 comp rx=0 tx=0).stop 2024-03-04T15:41:57.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.269+0000 7efe719e0700 1 -- 172.21.15.151:0/3977276128 shutdown_connections 2024-03-04T15:41:57.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.269+0000 7efe719e0700 1 --2- 172.21.15.151:0/3977276128 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7efe6c102900 0x7efe6c1974c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:57.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.269+0000 7efe719e0700 1 --2- 172.21.15.151:0/3977276128 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7efe58038e80 0x7efe5803b340 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:57.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.269+0000 7efe719e0700 1 --2- 172.21.15.151:0/3977276128 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7efe6c107b60 0x7efe6c19ea80 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:57.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.269+0000 7efe719e0700 1 --2- 172.21.15.151:0/3977276128 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7efe6c105230 0x7efe6c197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:41:57.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.269+0000 7efe719e0700 1 -- 172.21.15.151:0/3977276128 >> 172.21.15.151:0/3977276128 conn(0x7efe6c0fc310 msgr2=0x7efe6c103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:41:57.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.269+0000 7efe719e0700 1 -- 172.21.15.151:0/3977276128 shutdown_connections 2024-03-04T15:41:57.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:41:57.269+0000 7efe719e0700 1 -- 172.21.15.151:0/3977276128 wait complete. 2024-03-04T15:41:57.828 DEBUG:teuthology.orchestra.run.smithi151:mgr.x> sudo journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mgr.x.service 2024-03-04T15:41:57.830 INFO:tasks.cephadm:Deploying OSDs... 2024-03-04T15:41:57.830 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-04T15:41:57.830 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/scratch_devs of=/dev/stdout 2024-03-04T15:41:57.836 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-04T15:41:57.837 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_1 2024-03-04T15:41:57.886 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-04T15:41:57.886 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T15:41:57.886 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 874 Links: 1 2024-03-04T15:41:57.886 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T15:41:57.886 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-03-04 15:30:41.789216196 +0000 2024-03-04T15:41:57.886 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-03-04 15:30:41.533218480 +0000 2024-03-04T15:41:57.887 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-03-04 15:30:41.533218480 +0000 2024-03-04T15:41:57.887 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2024-03-04T15:41:57.887 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-04T15:41:57.941 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-03-04T15:41:57.941 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-03-04T15:41:57.941 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000451902 s, 1.1 MB/s 2024-03-04T15:41:57.942 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-04T15:41:57.991 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_2 2024-03-04T15:41:58.038 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-04T15:41:58.038 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T15:41:58.038 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 890 Links: 1 2024-03-04T15:41:58.038 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T15:41:58.038 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-03-04 15:30:42.041213941 +0000 2024-03-04T15:41:58.038 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-03-04 15:30:42.037213978 +0000 2024-03-04T15:41:58.038 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-03-04 15:30:42.037213978 +0000 2024-03-04T15:41:58.038 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2024-03-04T15:41:58.038 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-04T15:41:58.092 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-03-04T15:41:58.092 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-03-04T15:41:58.092 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000419012 s, 1.2 MB/s 2024-03-04T15:41:58.093 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-04T15:41:58.147 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_3 2024-03-04T15:41:58.194 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-04T15:41:58.194 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T15:41:58.194 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 904 Links: 1 2024-03-04T15:41:58.194 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T15:41:58.194 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-03-04 15:30:42.797207189 +0000 2024-03-04T15:41:58.194 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-03-04 15:30:42.533209548 +0000 2024-03-04T15:41:58.194 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-03-04 15:30:42.533209548 +0000 2024-03-04T15:41:58.194 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2024-03-04T15:41:58.194 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-04T15:41:58.248 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-03-04T15:41:58.248 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-03-04T15:41:58.248 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000412922 s, 1.2 MB/s 2024-03-04T15:41:58.249 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-04T15:41:58.299 DEBUG:teuthology.orchestra.run.smithi117:> stat /dev/vg_nvme/lv_4 2024-03-04T15:41:58.345 INFO:teuthology.orchestra.run.smithi117.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-04T15:41:58.345 INFO:teuthology.orchestra.run.smithi117.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T15:41:58.345 INFO:teuthology.orchestra.run.smithi117.stdout:Device: 5h/5d Inode: 926 Links: 1 2024-03-04T15:41:58.345 INFO:teuthology.orchestra.run.smithi117.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T15:41:58.346 INFO:teuthology.orchestra.run.smithi117.stdout:Access: 2024-03-04 15:30:43.329202436 +0000 2024-03-04T15:41:58.346 INFO:teuthology.orchestra.run.smithi117.stdout:Modify: 2024-03-04 15:30:43.057204866 +0000 2024-03-04T15:41:58.346 INFO:teuthology.orchestra.run.smithi117.stdout:Change: 2024-03-04 15:30:43.057204866 +0000 2024-03-04T15:41:58.346 INFO:teuthology.orchestra.run.smithi117.stdout: Birth: - 2024-03-04T15:41:58.346 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-04T15:41:58.400 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records in 2024-03-04T15:41:58.400 INFO:teuthology.orchestra.run.smithi117.stderr:1+0 records out 2024-03-04T15:41:58.400 INFO:teuthology.orchestra.run.smithi117.stderr:512 bytes copied, 0.000387828 s, 1.3 MB/s 2024-03-04T15:41:58.401 DEBUG:teuthology.orchestra.run.smithi117:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-04T15:41:58.451 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-04T15:41:58.451 DEBUG:teuthology.orchestra.run.smithi151:> dd if=/scratch_devs of=/dev/stdout 2024-03-04T15:41:58.457 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-03-04T15:41:58.458 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_1 2024-03-04T15:41:58.508 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-03-04T15:41:58.508 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T15:41:58.508 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 881 Links: 1 2024-03-04T15:41:58.508 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T15:41:58.508 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-03-04 15:30:37.176922126 +0000 2024-03-04T15:41:58.508 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-03-04 15:30:36.912924481 +0000 2024-03-04T15:41:58.509 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-03-04 15:30:36.912924481 +0000 2024-03-04T15:41:58.509 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: - 2024-03-04T15:41:58.509 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-03-04T15:41:58.561 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:58 smithi151 bash[23756]: cluster 2024-03-04T15:41:56.719977+0000 mgr.y (mgr.14152) 73 : cluster 0 pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:58.561 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:58 smithi151 bash[23756]: audit 2024-03-04T15:41:57.259668+0000 mgr.y (mgr.14152) 74 : audit 0 from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi117=y;smithi151=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:41:58.561 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:58 smithi151 bash[23756]: cephadm 2024-03-04T15:41:57.263266+0000 mgr.y (mgr.14152) 75 : cephadm 1 Saving service mgr spec with placement smithi117=y;smithi151=x;count:2 2024-03-04T15:41:58.561 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:58 smithi151 bash[23756]: audit 2024-03-04T15:41:57.269790+0000 mon.a (mon.0) 230 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:58.561 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:58 smithi151 bash[23756]: audit 2024-03-04T15:41:57.314280+0000 mon.a (mon.0) 231 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:41:58.561 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:58 smithi151 bash[23756]: audit 2024-03-04T15:41:57.318008+0000 mon.a (mon.0) 232 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:58.561 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:58 smithi151 bash[23756]: audit 2024-03-04T15:41:57.319890+0000 mon.a (mon.0) 233 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:41:58.561 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:58 smithi151 bash[23756]: audit 2024-03-04T15:41:57.330912+0000 mon.a (mon.0) 234 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:58.561 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:58 smithi151 bash[23756]: audit 2024-03-04T15:41:57.334102+0000 mon.a (mon.0) 235 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T15:41:58.562 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:58 smithi151 bash[23756]: audit 2024-03-04T15:41:57.336737+0000 mon.a (mon.0) 236 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-04T15:41:58.562 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:58 smithi151 bash[23756]: audit 2024-03-04T15:41:57.341506+0000 mon.a (mon.0) 237 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T15:41:58.562 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:58 smithi151 bash[23756]: audit 2024-03-04T15:41:57.343300+0000 mon.a (mon.0) 238 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:58.562 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:58 smithi151 bash[23756]: cephadm 2024-03-04T15:41:57.345284+0000 mgr.y (mgr.14152) 76 : cephadm 1 Deploying daemon mgr.x on smithi151 2024-03-04T15:41:58.563 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-03-04T15:41:58.563 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-03-04T15:41:58.563 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000463326 s, 1.1 MB/s 2024-03-04T15:41:58.564 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-03-04T15:41:58.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[21480]: cluster 2024-03-04T15:41:56.719977+0000 mgr.y (mgr.14152) 73 : cluster 0 pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:58.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[21480]: audit 2024-03-04T15:41:57.259668+0000 mgr.y (mgr.14152) 74 : audit 0 from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi117=y;smithi151=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:41:58.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[21480]: cephadm 2024-03-04T15:41:57.263266+0000 mgr.y (mgr.14152) 75 : cephadm 1 Saving service mgr spec with placement smithi117=y;smithi151=x;count:2 2024-03-04T15:41:58.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[21480]: audit 2024-03-04T15:41:57.269790+0000 mon.a (mon.0) 230 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:58.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[21480]: audit 2024-03-04T15:41:57.314280+0000 mon.a (mon.0) 231 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:41:58.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[21480]: audit 2024-03-04T15:41:57.318008+0000 mon.a (mon.0) 232 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:58.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[21480]: audit 2024-03-04T15:41:57.319890+0000 mon.a (mon.0) 233 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:41:58.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[21480]: audit 2024-03-04T15:41:57.330912+0000 mon.a (mon.0) 234 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:58.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[21480]: audit 2024-03-04T15:41:57.334102+0000 mon.a (mon.0) 235 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T15:41:58.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[21480]: audit 2024-03-04T15:41:57.336737+0000 mon.a (mon.0) 236 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-04T15:41:58.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[21480]: audit 2024-03-04T15:41:57.341506+0000 mon.a (mon.0) 237 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T15:41:58.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[21480]: audit 2024-03-04T15:41:57.343300+0000 mon.a (mon.0) 238 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:58.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[21480]: cephadm 2024-03-04T15:41:57.345284+0000 mgr.y (mgr.14152) 76 : cephadm 1 Deploying daemon mgr.x on smithi151 2024-03-04T15:41:58.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[28843]: cluster 2024-03-04T15:41:56.719977+0000 mgr.y (mgr.14152) 73 : cluster 0 pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:41:58.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[28843]: audit 2024-03-04T15:41:57.259668+0000 mgr.y (mgr.14152) 74 : audit 0 from='client.24109 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi117=y;smithi151=x", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:41:58.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[28843]: cephadm 2024-03-04T15:41:57.263266+0000 mgr.y (mgr.14152) 75 : cephadm 1 Saving service mgr spec with placement smithi117=y;smithi151=x;count:2 2024-03-04T15:41:58.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[28843]: audit 2024-03-04T15:41:57.269790+0000 mon.a (mon.0) 230 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:58.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[28843]: audit 2024-03-04T15:41:57.314280+0000 mon.a (mon.0) 231 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:41:58.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[28843]: audit 2024-03-04T15:41:57.318008+0000 mon.a (mon.0) 232 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:58.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[28843]: audit 2024-03-04T15:41:57.319890+0000 mon.a (mon.0) 233 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:41:58.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[28843]: audit 2024-03-04T15:41:57.330912+0000 mon.a (mon.0) 234 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:41:58.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[28843]: audit 2024-03-04T15:41:57.334102+0000 mon.a (mon.0) 235 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T15:41:58.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[28843]: audit 2024-03-04T15:41:57.336737+0000 mon.a (mon.0) 236 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-03-04T15:41:58.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[28843]: audit 2024-03-04T15:41:57.341506+0000 mon.a (mon.0) 237 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T15:41:58.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[28843]: audit 2024-03-04T15:41:57.343300+0000 mon.a (mon.0) 238 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:41:58.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:41:58 smithi117 bash[28843]: cephadm 2024-03-04T15:41:57.345284+0000 mgr.y (mgr.14152) 76 : cephadm 1 Deploying daemon mgr.x on smithi151 2024-03-04T15:41:58.613 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_2 2024-03-04T15:41:58.659 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-03-04T15:41:58.659 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T15:41:58.659 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 904 Links: 1 2024-03-04T15:41:58.659 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T15:41:58.659 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-03-04 15:30:37.644917951 +0000 2024-03-04T15:41:58.659 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-03-04 15:30:37.412920022 +0000 2024-03-04T15:41:58.659 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-03-04 15:30:37.412920022 +0000 2024-03-04T15:41:58.660 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: - 2024-03-04T15:41:58.660 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-03-04T15:41:58.712 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-03-04T15:41:58.712 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-03-04T15:41:58.712 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.00030578 s, 1.7 MB/s 2024-03-04T15:41:58.713 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-03-04T15:41:58.764 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_3 2024-03-04T15:41:58.811 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-03-04T15:41:58.811 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T15:41:58.811 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 923 Links: 1 2024-03-04T15:41:58.811 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T15:41:58.811 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-03-04 15:30:38.168913277 +0000 2024-03-04T15:41:58.811 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-03-04 15:30:37.896915704 +0000 2024-03-04T15:41:58.812 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-03-04 15:30:37.896915704 +0000 2024-03-04T15:41:58.812 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: - 2024-03-04T15:41:58.812 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-03-04T15:41:58.864 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-03-04T15:41:58.864 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-03-04T15:41:58.864 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000293284 s, 1.7 MB/s 2024-03-04T15:41:58.865 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-03-04T15:41:58.913 DEBUG:teuthology.orchestra.run.smithi151:> stat /dev/vg_nvme/lv_4 2024-03-04T15:41:58.959 INFO:teuthology.orchestra.run.smithi151.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-03-04T15:41:58.959 INFO:teuthology.orchestra.run.smithi151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-03-04T15:41:58.959 INFO:teuthology.orchestra.run.smithi151.stdout:Device: 5h/5d Inode: 934 Links: 1 2024-03-04T15:41:58.959 INFO:teuthology.orchestra.run.smithi151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-03-04T15:41:58.959 INFO:teuthology.orchestra.run.smithi151.stdout:Access: 2024-03-04 15:30:38.412911101 +0000 2024-03-04T15:41:58.959 INFO:teuthology.orchestra.run.smithi151.stdout:Modify: 2024-03-04 15:30:38.408911137 +0000 2024-03-04T15:41:58.960 INFO:teuthology.orchestra.run.smithi151.stdout:Change: 2024-03-04 15:30:38.408911137 +0000 2024-03-04T15:41:58.960 INFO:teuthology.orchestra.run.smithi151.stdout: Birth: - 2024-03-04T15:41:58.960 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-03-04T15:41:59.012 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records in 2024-03-04T15:41:59.012 INFO:teuthology.orchestra.run.smithi151.stderr:1+0 records out 2024-03-04T15:41:59.012 INFO:teuthology.orchestra.run.smithi151.stderr:512 bytes copied, 0.000255286 s, 2.0 MB/s 2024-03-04T15:41:59.013 DEBUG:teuthology.orchestra.run.smithi151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-03-04T15:41:59.060 INFO:tasks.cephadm:Deploying osd.0 on smithi117 with /dev/vg_nvme/lv_4... 2024-03-04T15:41:59.061 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-04T15:41:59.819 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:59 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:41:59.820 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:41:59 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:41:59.820 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:41:59 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:41:59.820 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:41:59 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:42:00.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:41:59 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:42:00.182 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:41:59 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:42:00.182 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:41:59 smithi151 systemd[1]: Started Ceph mgr.x for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T15:42:00.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:00 smithi117 bash[21480]: cluster 2024-03-04T15:41:58.720400+0000 mgr.y (mgr.14152) 77 : cluster 0 pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:00.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:00 smithi117 bash[21480]: audit 2024-03-04T15:41:59.965085+0000 mon.a (mon.0) 239 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:00.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:00 smithi117 bash[21480]: audit 2024-03-04T15:41:59.972735+0000 mon.a (mon.0) 240 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:00.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:00 smithi117 bash[21480]: audit 2024-03-04T15:42:00.023199+0000 mon.a (mon.0) 241 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:00.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:00 smithi117 bash[21480]: audit 2024-03-04T15:42:00.030918+0000 mon.a (mon.0) 242 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:00.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:00 smithi117 bash[21480]: audit 2024-03-04T15:42:00.099190+0000 mon.a (mon.0) 243 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:42:00.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:00 smithi117 bash[28843]: cluster 2024-03-04T15:41:58.720400+0000 mgr.y (mgr.14152) 77 : cluster 0 pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:00.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:00 smithi117 bash[28843]: audit 2024-03-04T15:41:59.965085+0000 mon.a (mon.0) 239 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:00.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:00 smithi117 bash[28843]: audit 2024-03-04T15:41:59.972735+0000 mon.a (mon.0) 240 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:00.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:00 smithi117 bash[28843]: audit 2024-03-04T15:42:00.023199+0000 mon.a (mon.0) 241 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:00.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:00 smithi117 bash[28843]: audit 2024-03-04T15:42:00.030918+0000 mon.a (mon.0) 242 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:00.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:00 smithi117 bash[28843]: audit 2024-03-04T15:42:00.099190+0000 mon.a (mon.0) 243 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:42:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:00 smithi151 bash[23756]: cluster 2024-03-04T15:41:58.720400+0000 mgr.y (mgr.14152) 77 : cluster 0 pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:00 smithi151 bash[23756]: audit 2024-03-04T15:41:59.965085+0000 mon.a (mon.0) 239 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:00 smithi151 bash[23756]: audit 2024-03-04T15:41:59.972735+0000 mon.a (mon.0) 240 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:00.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:00 smithi151 bash[23756]: audit 2024-03-04T15:42:00.023199+0000 mon.a (mon.0) 241 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:00.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:00 smithi151 bash[23756]: audit 2024-03-04T15:42:00.030918+0000 mon.a (mon.0) 242 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:00.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:00 smithi151 bash[23756]: audit 2024-03-04T15:42:00.099190+0000 mon.a (mon.0) 243 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:42:00.948 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:00 smithi151 bash[24709]: debug 2024-03-04T15:42:00.797+0000 7fc0000a6700 1 -- 172.21.15.151:0/3539526718 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x559b394d25a0 con 0x559b3a256800 2024-03-04T15:42:00.948 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:00 smithi151 bash[24709]: debug 2024-03-04T15:42:00.945+0000 7fc00a2a8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T15:42:02.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:02 smithi117 bash[21480]: cluster 2024-03-04T15:42:00.720872+0000 mgr.y (mgr.14152) 78 : cluster 0 pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:02.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:02 smithi117 bash[21480]: audit 2024-03-04T15:42:01.943056+0000 mon.a (mon.0) 244 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:02.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:02 smithi117 bash[28843]: cluster 2024-03-04T15:42:00.720872+0000 mgr.y (mgr.14152) 78 : cluster 0 pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:02.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:02 smithi117 bash[28843]: audit 2024-03-04T15:42:01.943056+0000 mon.a (mon.0) 244 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:02.603 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:02 smithi151 bash[24709]: debug 2024-03-04T15:42:02.501+0000 7fc00a2a8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T15:42:02.603 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:02 smithi151 bash[23756]: cluster 2024-03-04T15:42:00.720872+0000 mgr.y (mgr.14152) 78 : cluster 0 pgmap v43: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:02.603 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:02 smithi151 bash[23756]: audit 2024-03-04T15:42:01.943056+0000 mon.a (mon.0) 244 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:02.932 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:02 smithi151 bash[24709]: debug 2024-03-04T15:42:02.597+0000 7fc00a2a8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T15:42:02.932 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:02 smithi151 bash[24709]: debug 2024-03-04T15:42:02.845+0000 7fc00a2a8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T15:42:03.780 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:42:03.932 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:03 smithi151 bash[24709]: debug 2024-03-04T15:42:03.649+0000 7fc00a2a8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T15:42:03.932 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:03 smithi151 bash[24709]: debug 2024-03-04T15:42:03.761+0000 7fc00a2a8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T15:42:03.932 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:03 smithi151 bash[24709]: debug 2024-03-04T15:42:03.861+0000 7fc00a2a8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T15:42:04.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:04 smithi117 bash[21480]: cluster 2024-03-04T15:42:02.721391+0000 mgr.y (mgr.14152) 79 : cluster 0 pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:04.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:04 smithi117 bash[28843]: cluster 2024-03-04T15:42:02.721391+0000 mgr.y (mgr.14152) 79 : cluster 0 pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:04.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:04 smithi151 bash[23756]: cluster 2024-03-04T15:42:02.721391+0000 mgr.y (mgr.14152) 79 : cluster 0 pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:04.682 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:04 smithi151 bash[24709]: debug 2024-03-04T15:42:04.213+0000 7fc00a2a8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T15:42:04.682 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:04 smithi151 bash[24709]: debug 2024-03-04T15:42:04.325+0000 7fc00a2a8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T15:42:04.682 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:04 smithi151 bash[24709]: debug 2024-03-04T15:42:04.425+0000 7fc00a2a8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T15:42:05.182 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:04 smithi151 bash[24709]: debug 2024-03-04T15:42:04.777+0000 7fc00a2a8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T15:42:05.182 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:05 smithi151 bash[24709]: debug 2024-03-04T15:42:05.017+0000 7fc00a2a8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T15:42:05.669 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:05 smithi151 bash[24709]: debug 2024-03-04T15:42:05.281+0000 7fc00a2a8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T15:42:05.688 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:05 smithi151 bash[24709]: debug 2024-03-04T15:42:05.393+0000 7fc00a2a8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T15:42:05.932 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:05 smithi151 bash[24709]: debug 2024-03-04T15:42:05.665+0000 7fc00a2a8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T15:42:05.933 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:05 smithi151 bash[24709]: debug 2024-03-04T15:42:05.901+0000 7fc00a2a8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T15:42:06.554 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[21480]: cluster 2024-03-04T15:42:04.721697+0000 mgr.y (mgr.14152) 80 : cluster 0 pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:06.554 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[21480]: audit 2024-03-04T15:42:05.293444+0000 mon.a (mon.0) 245 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:06.554 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[21480]: audit 2024-03-04T15:42:05.302338+0000 mon.a (mon.0) 246 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:06.554 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[21480]: audit 2024-03-04T15:42:05.304194+0000 mon.a (mon.0) 247 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:42:06.554 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[21480]: audit 2024-03-04T15:42:05.306160+0000 mon.a (mon.0) 248 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:42:06.555 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[21480]: audit 2024-03-04T15:42:05.315374+0000 mon.a (mon.0) 249 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:06.555 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[21480]: cephadm 2024-03-04T15:42:05.384571+0000 mgr.y (mgr.14152) 81 : cephadm 1 Reconfiguring mgr.y (unknown last config time)... 2024-03-04T15:42:06.555 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[21480]: audit 2024-03-04T15:42:05.385175+0000 mon.a (mon.0) 250 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T15:42:06.555 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[21480]: audit 2024-03-04T15:42:05.386506+0000 mon.a (mon.0) 251 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T15:42:06.555 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[21480]: audit 2024-03-04T15:42:05.387533+0000 mon.a (mon.0) 252 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:42:06.555 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[21480]: cephadm 2024-03-04T15:42:05.388738+0000 mgr.y (mgr.14152) 82 : cephadm 1 Reconfiguring daemon mgr.y on smithi117 2024-03-04T15:42:06.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[28843]: cluster 2024-03-04T15:42:04.721697+0000 mgr.y (mgr.14152) 80 : cluster 0 pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:06.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[28843]: audit 2024-03-04T15:42:05.293444+0000 mon.a (mon.0) 245 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:06.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[28843]: audit 2024-03-04T15:42:05.302338+0000 mon.a (mon.0) 246 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:06.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[28843]: audit 2024-03-04T15:42:05.304194+0000 mon.a (mon.0) 247 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:42:06.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[28843]: audit 2024-03-04T15:42:05.306160+0000 mon.a (mon.0) 248 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:42:06.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[28843]: audit 2024-03-04T15:42:05.315374+0000 mon.a (mon.0) 249 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:06.555 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[28843]: cephadm 2024-03-04T15:42:05.384571+0000 mgr.y (mgr.14152) 81 : cephadm 1 Reconfiguring mgr.y (unknown last config time)... 2024-03-04T15:42:06.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[28843]: audit 2024-03-04T15:42:05.385175+0000 mon.a (mon.0) 250 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T15:42:06.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[28843]: audit 2024-03-04T15:42:05.386506+0000 mon.a (mon.0) 251 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T15:42:06.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[28843]: audit 2024-03-04T15:42:05.387533+0000 mon.a (mon.0) 252 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:42:06.556 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:06 smithi117 bash[28843]: cephadm 2024-03-04T15:42:05.388738+0000 mgr.y (mgr.14152) 82 : cephadm 1 Reconfiguring daemon mgr.y on smithi117 2024-03-04T15:42:06.672 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:06 smithi151 bash[24709]: debug 2024-03-04T15:42:06.309+0000 7fc00a2a8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T15:42:06.672 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:06 smithi151 bash[24709]: debug 2024-03-04T15:42:06.425+0000 7fc00a2a8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T15:42:06.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:06 smithi151 bash[23756]: cluster 2024-03-04T15:42:04.721697+0000 mgr.y (mgr.14152) 80 : cluster 0 pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:06.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:06 smithi151 bash[23756]: audit 2024-03-04T15:42:05.293444+0000 mon.a (mon.0) 245 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:06.672 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:06 smithi151 bash[23756]: audit 2024-03-04T15:42:05.302338+0000 mon.a (mon.0) 246 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:06.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:06 smithi151 bash[23756]: audit 2024-03-04T15:42:05.304194+0000 mon.a (mon.0) 247 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:42:06.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:06 smithi151 bash[23756]: audit 2024-03-04T15:42:05.306160+0000 mon.a (mon.0) 248 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:42:06.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:06 smithi151 bash[23756]: audit 2024-03-04T15:42:05.315374+0000 mon.a (mon.0) 249 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:06.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:06 smithi151 bash[23756]: cephadm 2024-03-04T15:42:05.384571+0000 mgr.y (mgr.14152) 81 : cephadm 1 Reconfiguring mgr.y (unknown last config time)... 2024-03-04T15:42:06.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:06 smithi151 bash[23756]: audit 2024-03-04T15:42:05.385175+0000 mon.a (mon.0) 250 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-03-04T15:42:06.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:06 smithi151 bash[23756]: audit 2024-03-04T15:42:05.386506+0000 mon.a (mon.0) 251 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-03-04T15:42:06.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:06 smithi151 bash[23756]: audit 2024-03-04T15:42:05.387533+0000 mon.a (mon.0) 252 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:42:06.673 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:06 smithi151 bash[23756]: cephadm 2024-03-04T15:42:05.388738+0000 mgr.y (mgr.14152) 82 : cephadm 1 Reconfiguring daemon mgr.y on smithi117 2024-03-04T15:42:06.932 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:06 smithi151 bash[24709]: debug 2024-03-04T15:42:06.669+0000 7fc00a2a8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T15:42:07.225 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-04T15:42:07.226 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-04T15:42:07.226 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-03-04T15:42:07.226 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-03-04T15:42:07.226 INFO:teuthology.orchestra.run.smithi117.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0151479 s, 692 MB/s 2024-03-04T15:42:07.226 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 2024-03-04T15:42:07.226 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-03-04T15:42:08.019 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi117:vg_nvme/lv_4 2024-03-04T15:42:08.109 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:07 smithi151 bash[24709]: debug 2024-03-04T15:42:07.689+0000 7fc00a2a8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T15:42:08.110 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:07 smithi151 bash[24709]: debug 2024-03-04T15:42:07.809+0000 7fc00a2a8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T15:42:08.432 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:08 smithi151 bash[24709]: debug 2024-03-04T15:42:08.105+0000 7fc00a2a8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T15:42:08.432 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:08 smithi151 bash[24709]: debug 2024-03-04T15:42:08.221+0000 7fc00a2a8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T15:42:08.932 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:08 smithi151 bash[24709]: debug 2024-03-04T15:42:08.669+0000 7fc00a2a8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T15:42:08.932 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:42:08 smithi151 bash[24709]: debug 2024-03-04T15:42:08.785+0000 7fc00a2a8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T15:42:08.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:08 smithi151 bash[23756]: cluster 2024-03-04T15:42:06.722028+0000 mgr.y (mgr.14152) 83 : cluster 0 pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:08.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:08 smithi151 bash[23756]: audit 2024-03-04T15:42:07.655952+0000 mon.a (mon.0) 253 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:08.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:08 smithi151 bash[23756]: audit 2024-03-04T15:42:07.663080+0000 mon.a (mon.0) 254 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:08.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:08 smithi151 bash[23756]: audit 2024-03-04T15:42:07.709333+0000 mon.a (mon.0) 255 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:42:08.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:08 smithi151 bash[23756]: audit 2024-03-04T15:42:08.284254+0000 mon.a (mon.0) 256 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:42:08.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:08 smithi151 bash[23756]: audit 2024-03-04T15:42:08.285966+0000 mon.a (mon.0) 257 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:42:08.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:08 smithi151 bash[23756]: audit 2024-03-04T15:42:08.294637+0000 mon.a (mon.0) 258 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:09.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:08 smithi117 bash[21480]: cluster 2024-03-04T15:42:06.722028+0000 mgr.y (mgr.14152) 83 : cluster 0 pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:09.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:08 smithi117 bash[21480]: audit 2024-03-04T15:42:07.655952+0000 mon.a (mon.0) 253 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:09.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:08 smithi117 bash[21480]: audit 2024-03-04T15:42:07.663080+0000 mon.a (mon.0) 254 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:09.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:08 smithi117 bash[21480]: audit 2024-03-04T15:42:07.709333+0000 mon.a (mon.0) 255 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:42:09.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:08 smithi117 bash[21480]: audit 2024-03-04T15:42:08.284254+0000 mon.a (mon.0) 256 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:42:09.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:08 smithi117 bash[21480]: audit 2024-03-04T15:42:08.285966+0000 mon.a (mon.0) 257 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:42:09.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:08 smithi117 bash[21480]: audit 2024-03-04T15:42:08.294637+0000 mon.a (mon.0) 258 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:09.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:08 smithi117 bash[28843]: cluster 2024-03-04T15:42:06.722028+0000 mgr.y (mgr.14152) 83 : cluster 0 pgmap v46: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:09.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:08 smithi117 bash[28843]: audit 2024-03-04T15:42:07.655952+0000 mon.a (mon.0) 253 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:09.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:08 smithi117 bash[28843]: audit 2024-03-04T15:42:07.663080+0000 mon.a (mon.0) 254 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:09.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:08 smithi117 bash[28843]: audit 2024-03-04T15:42:07.709333+0000 mon.a (mon.0) 255 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:42:09.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:08 smithi117 bash[28843]: audit 2024-03-04T15:42:08.284254+0000 mon.a (mon.0) 256 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:42:09.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:08 smithi117 bash[28843]: audit 2024-03-04T15:42:08.285966+0000 mon.a (mon.0) 257 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:42:09.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:08 smithi117 bash[28843]: audit 2024-03-04T15:42:08.294637+0000 mon.a (mon.0) 258 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:09.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:09 smithi151 bash[23756]: cluster 2024-03-04T15:42:08.793247+0000 mon.a (mon.0) 259 : cluster 0 Standby manager daemon x started 2024-03-04T15:42:09.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:09 smithi151 bash[23756]: audit 2024-03-04T15:42:08.793902+0000 mon.b (mon.1) 4 : audit 0 from='mgr.? 172.21.15.151:0/1800103875' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-04T15:42:09.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:09 smithi151 bash[23756]: audit 2024-03-04T15:42:08.794450+0000 mon.b (mon.1) 5 : audit 0 from='mgr.? 172.21.15.151:0/1800103875' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T15:42:09.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:09 smithi151 bash[23756]: audit 2024-03-04T15:42:08.795623+0000 mon.b (mon.1) 6 : audit 0 from='mgr.? 172.21.15.151:0/1800103875' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-04T15:42:09.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:09 smithi151 bash[23756]: audit 2024-03-04T15:42:08.796189+0000 mon.b (mon.1) 7 : audit 0 from='mgr.? 172.21.15.151:0/1800103875' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T15:42:10.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:09 smithi117 bash[21480]: cluster 2024-03-04T15:42:08.793247+0000 mon.a (mon.0) 259 : cluster 0 Standby manager daemon x started 2024-03-04T15:42:10.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:09 smithi117 bash[21480]: audit 2024-03-04T15:42:08.793902+0000 mon.b (mon.1) 4 : audit 0 from='mgr.? 172.21.15.151:0/1800103875' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-04T15:42:10.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:09 smithi117 bash[21480]: audit 2024-03-04T15:42:08.794450+0000 mon.b (mon.1) 5 : audit 0 from='mgr.? 172.21.15.151:0/1800103875' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T15:42:10.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:09 smithi117 bash[21480]: audit 2024-03-04T15:42:08.795623+0000 mon.b (mon.1) 6 : audit 0 from='mgr.? 172.21.15.151:0/1800103875' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-04T15:42:10.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:09 smithi117 bash[21480]: audit 2024-03-04T15:42:08.796189+0000 mon.b (mon.1) 7 : audit 0 from='mgr.? 172.21.15.151:0/1800103875' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T15:42:10.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:09 smithi117 bash[28843]: cluster 2024-03-04T15:42:08.793247+0000 mon.a (mon.0) 259 : cluster 0 Standby manager daemon x started 2024-03-04T15:42:10.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:09 smithi117 bash[28843]: audit 2024-03-04T15:42:08.793902+0000 mon.b (mon.1) 4 : audit 0 from='mgr.? 172.21.15.151:0/1800103875' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-04T15:42:10.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:09 smithi117 bash[28843]: audit 2024-03-04T15:42:08.794450+0000 mon.b (mon.1) 5 : audit 0 from='mgr.? 172.21.15.151:0/1800103875' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T15:42:10.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:09 smithi117 bash[28843]: audit 2024-03-04T15:42:08.795623+0000 mon.b (mon.1) 6 : audit 0 from='mgr.? 172.21.15.151:0/1800103875' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-04T15:42:10.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:09 smithi117 bash[28843]: audit 2024-03-04T15:42:08.796189+0000 mon.b (mon.1) 7 : audit 0 from='mgr.? 172.21.15.151:0/1800103875' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T15:42:10.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:10 smithi151 bash[23756]: cluster 2024-03-04T15:42:08.722419+0000 mgr.y (mgr.14152) 84 : cluster 0 pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:10.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:10 smithi151 bash[23756]: cluster 2024-03-04T15:42:09.676713+0000 mon.a (mon.0) 260 : cluster 0 mgrmap e14: y(active, since 106s), standbys: x 2024-03-04T15:42:10.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:10 smithi151 bash[23756]: audit 2024-03-04T15:42:09.676978+0000 mon.a (mon.0) 261 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-04T15:42:11.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:10 smithi117 bash[21480]: cluster 2024-03-04T15:42:08.722419+0000 mgr.y (mgr.14152) 84 : cluster 0 pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:11.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:10 smithi117 bash[21480]: cluster 2024-03-04T15:42:09.676713+0000 mon.a (mon.0) 260 : cluster 0 mgrmap e14: y(active, since 106s), standbys: x 2024-03-04T15:42:11.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:10 smithi117 bash[21480]: audit 2024-03-04T15:42:09.676978+0000 mon.a (mon.0) 261 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-04T15:42:11.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:10 smithi117 bash[28843]: cluster 2024-03-04T15:42:08.722419+0000 mgr.y (mgr.14152) 84 : cluster 0 pgmap v47: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:11.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:10 smithi117 bash[28843]: cluster 2024-03-04T15:42:09.676713+0000 mon.a (mon.0) 260 : cluster 0 mgrmap e14: y(active, since 106s), standbys: x 2024-03-04T15:42:11.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:10 smithi117 bash[28843]: audit 2024-03-04T15:42:09.676978+0000 mon.a (mon.0) 261 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-04T15:42:12.737 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:42:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:12 smithi151 bash[23756]: cluster 2024-03-04T15:42:10.722809+0000 mgr.y (mgr.14152) 85 : cluster 0 pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:13.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:12 smithi117 bash[28843]: cluster 2024-03-04T15:42:10.722809+0000 mgr.y (mgr.14152) 85 : cluster 0 pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:13.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:12 smithi117 bash[21480]: cluster 2024-03-04T15:42:10.722809+0000 mgr.y (mgr.14152) 85 : cluster 0 pgmap v48: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:14.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:14 smithi151 bash[23756]: cluster 2024-03-04T15:42:12.723213+0000 mgr.y (mgr.14152) 86 : cluster 0 pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:14.946 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:14 smithi117 bash[21480]: cluster 2024-03-04T15:42:12.723213+0000 mgr.y (mgr.14152) 86 : cluster 0 pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:14.946 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:14 smithi117 bash[28843]: cluster 2024-03-04T15:42:12.723213+0000 mgr.y (mgr.14152) 86 : cluster 0 pgmap v49: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:15.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff141042700 1 -- 172.21.15.117:0/386912295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff13c102fd0 msgr2=0x7ff13c103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:42:15.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff141042700 1 --2- 172.21.15.117:0/386912295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff13c102fd0 0x7ff13c103450 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff130009990 tx=0x7ff130009ca0 comp rx=0 tx=0).stop 2024-03-04T15:42:15.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff141042700 1 -- 172.21.15.117:0/386912295 shutdown_connections 2024-03-04T15:42:15.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff141042700 1 --2- 172.21.15.117:0/386912295 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff13c103990 0x7ff13c10a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:42:15.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff141042700 1 --2- 172.21.15.117:0/386912295 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff13c101e70 0x7ff13c102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:42:15.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff141042700 1 --2- 172.21.15.117:0/386912295 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff13c102fd0 0x7ff13c103450 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:42:15.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff141042700 1 -- 172.21.15.117:0/386912295 >> 172.21.15.117:0/386912295 conn(0x7ff13c0fd410 msgr2=0x7ff13c0ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:42:15.123 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff141042700 1 -- 172.21.15.117:0/386912295 shutdown_connections 2024-03-04T15:42:15.124 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff141042700 1 -- 172.21.15.117:0/386912295 wait complete. 2024-03-04T15:42:15.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff141042700 1 Processor -- start 2024-03-04T15:42:15.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff141042700 1 -- start start 2024-03-04T15:42:15.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff141042700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff13c101e70 0x7ff13c1975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:42:15.125 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff141042700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff13c102fd0 0x7ff13c197af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:42:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff141042700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff13c103990 0x7ff13c19eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:42:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff141042700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c10ceb0 con 0x7ff13c103990 2024-03-04T15:42:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff141042700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c10cd30 con 0x7ff13c102fd0 2024-03-04T15:42:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff141042700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff13c10d030 con 0x7ff13c101e70 2024-03-04T15:42:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff13ad9d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff13c101e70 0x7ff13c1975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:42:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff13ad9d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff13c101e70 0x7ff13c1975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:40156/0 (socket says 172.21.15.117:40156) 2024-03-04T15:42:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff13ad9d700 1 -- 172.21.15.117:0/3321444526 learned_addr learned my addr 172.21.15.117:0/3321444526 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:42:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff13b59e700 1 --2- 172.21.15.117:0/3321444526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff13c103990 0x7ff13c19eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:42:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff13a59c700 1 --2- 172.21.15.117:0/3321444526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff13c102fd0 0x7ff13c197af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:42:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff13ad9d700 1 -- 172.21.15.117:0/3321444526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff13c102fd0 msgr2=0x7ff13c197af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:42:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff13ad9d700 1 --2- 172.21.15.117:0/3321444526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff13c102fd0 0x7ff13c197af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:42:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff13ad9d700 1 -- 172.21.15.117:0/3321444526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff13c103990 msgr2=0x7ff13c19eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:42:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff13ad9d700 1 --2- 172.21.15.117:0/3321444526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff13c103990 0x7ff13c19eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:42:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff13ad9d700 1 -- 172.21.15.117:0/3321444526 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff13c19f210 con 0x7ff13c101e70 2024-03-04T15:42:15.126 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff13b59e700 1 --2- 172.21.15.117:0/3321444526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff13c103990 0x7ff13c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:42:15.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff13ad9d700 1 --2- 172.21.15.117:0/3321444526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff13c101e70 0x7ff13c1975b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00e9d0 tx=0x7ff12c00ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:42:15.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff123fff700 1 -- 172.21.15.117:0/3321444526 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c00cbd0 con 0x7ff13c101e70 2024-03-04T15:42:15.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff123fff700 1 -- 172.21.15.117:0/3321444526 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff12c00cd70 con 0x7ff13c101e70 2024-03-04T15:42:15.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff141042700 1 -- 172.21.15.117:0/3321444526 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff13c19f500 con 0x7ff13c101e70 2024-03-04T15:42:15.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff123fff700 1 -- 172.21.15.117:0/3321444526 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff12c018680 con 0x7ff13c101e70 2024-03-04T15:42:15.127 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.119+0000 7ff141042700 1 -- 172.21.15.117:0/3321444526 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff13c19fa40 con 0x7ff13c101e70 2024-03-04T15:42:15.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.123+0000 7ff141042700 1 -- 172.21.15.117:0/3321444526 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13c04faa0 con 0x7ff13c101e70 2024-03-04T15:42:15.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.123+0000 7ff123fff700 1 -- 172.21.15.117:0/3321444526 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 14) v1 ==== 91341+0+0 (secure 0 0 0) 0x7ff12c010430 con 0x7ff13c101e70 2024-03-04T15:42:15.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.123+0000 7ff123fff700 1 --2- 172.21.15.117:0/3321444526 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7ff12406d600 0x7ff12406fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:42:15.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.123+0000 7ff123fff700 1 -- 172.21.15.117:0/3321444526 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ff12c08c160 con 0x7ff13c101e70 2024-03-04T15:42:15.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.123+0000 7ff13a59c700 1 --2- 172.21.15.117:0/3321444526 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7ff12406d600 0x7ff12406fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:42:15.128 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.123+0000 7ff13a59c700 1 --2- 172.21.15.117:0/3321444526 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7ff12406d600 0x7ff12406fac0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff130006010 tx=0x7ff13001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:42:15.132 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.127+0000 7ff123fff700 1 -- 172.21.15.117:0/3321444526 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff12c05b380 con 0x7ff13c101e70 2024-03-04T15:42:15.341 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:15.335+0000 7ff141042700 1 -- 172.21.15.117:0/3321444526 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7ff13c02db10 con 0x7ff12406d600 2024-03-04T15:42:16.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:15 smithi117 bash[21480]: audit 2024-03-04T15:42:15.345130+0000 mon.a (mon.0) 262 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:42:16.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:15 smithi117 bash[21480]: audit 2024-03-04T15:42:15.350749+0000 mon.a (mon.0) 263 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:42:16.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:15 smithi117 bash[21480]: audit 2024-03-04T15:42:15.351849+0000 mon.a (mon.0) 264 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:42:16.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:15 smithi117 bash[28843]: audit 2024-03-04T15:42:15.345130+0000 mon.a (mon.0) 262 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:42:16.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:15 smithi117 bash[28843]: audit 2024-03-04T15:42:15.350749+0000 mon.a (mon.0) 263 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:42:16.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:15 smithi117 bash[28843]: audit 2024-03-04T15:42:15.351849+0000 mon.a (mon.0) 264 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:42:16.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:15 smithi151 bash[23756]: audit 2024-03-04T15:42:15.345130+0000 mon.a (mon.0) 262 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:42:16.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:15 smithi151 bash[23756]: audit 2024-03-04T15:42:15.350749+0000 mon.a (mon.0) 263 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:42:16.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:15 smithi151 bash[23756]: audit 2024-03-04T15:42:15.351849+0000 mon.a (mon.0) 264 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:42:17.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:16 smithi117 bash[21480]: cluster 2024-03-04T15:42:14.723627+0000 mgr.y (mgr.14152) 87 : cluster 0 pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:17.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:16 smithi117 bash[21480]: audit 2024-03-04T15:42:15.342398+0000 mgr.y (mgr.14152) 88 : audit 0 from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:42:17.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:16 smithi117 bash[28843]: cluster 2024-03-04T15:42:14.723627+0000 mgr.y (mgr.14152) 87 : cluster 0 pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:17.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:16 smithi117 bash[28843]: audit 2024-03-04T15:42:15.342398+0000 mgr.y (mgr.14152) 88 : audit 0 from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:42:17.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:16 smithi151 bash[23756]: cluster 2024-03-04T15:42:14.723627+0000 mgr.y (mgr.14152) 87 : cluster 0 pgmap v50: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:17.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:16 smithi151 bash[23756]: audit 2024-03-04T15:42:15.342398+0000 mgr.y (mgr.14152) 88 : audit 0 from='client.24116 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:42:19.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:18 smithi117 bash[21480]: cluster 2024-03-04T15:42:16.724050+0000 mgr.y (mgr.14152) 89 : cluster 0 pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:19.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:18 smithi117 bash[28843]: cluster 2024-03-04T15:42:16.724050+0000 mgr.y (mgr.14152) 89 : cluster 0 pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:19.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:18 smithi151 bash[23756]: cluster 2024-03-04T15:42:16.724050+0000 mgr.y (mgr.14152) 89 : cluster 0 pgmap v51: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:21.009 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:20 smithi117 bash[21480]: cluster 2024-03-04T15:42:18.724475+0000 mgr.y (mgr.14152) 90 : cluster 0 pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:21.009 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:20 smithi117 bash[28843]: cluster 2024-03-04T15:42:18.724475+0000 mgr.y (mgr.14152) 90 : cluster 0 pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:21.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:20 smithi151 bash[23756]: cluster 2024-03-04T15:42:18.724475+0000 mgr.y (mgr.14152) 90 : cluster 0 pgmap v52: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:23.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:22 smithi117 bash[21480]: cluster 2024-03-04T15:42:20.724840+0000 mgr.y (mgr.14152) 91 : cluster 0 pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:23.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:22 smithi117 bash[28843]: cluster 2024-03-04T15:42:20.724840+0000 mgr.y (mgr.14152) 91 : cluster 0 pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:23.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:22 smithi151 bash[23756]: cluster 2024-03-04T15:42:20.724840+0000 mgr.y (mgr.14152) 91 : cluster 0 pgmap v53: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:24.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:23 smithi117 bash[21480]: cluster 2024-03-04T15:42:22.725199+0000 mgr.y (mgr.14152) 92 : cluster 0 pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:24.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:23 smithi117 bash[21480]: audit 2024-03-04T15:42:23.339678+0000 mon.c (mon.2) 3 : audit 1 from='client.? 172.21.15.117:0/4271088755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf1f40a0-bf1a-4681-84de-fbf6fee19de7"}]: dispatch 2024-03-04T15:42:24.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:23 smithi117 bash[21480]: audit 2024-03-04T15:42:23.340394+0000 mon.a (mon.0) 265 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf1f40a0-bf1a-4681-84de-fbf6fee19de7"}]: dispatch 2024-03-04T15:42:24.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:23 smithi117 bash[21480]: audit 2024-03-04T15:42:23.342480+0000 mon.a (mon.0) 266 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf1f40a0-bf1a-4681-84de-fbf6fee19de7"}]': finished 2024-03-04T15:42:24.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:23 smithi117 bash[21480]: cluster 2024-03-04T15:42:23.344861+0000 mon.a (mon.0) 267 : cluster 0 osdmap e5: 1 total, 0 up, 1 in 2024-03-04T15:42:24.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:23 smithi117 bash[21480]: audit 2024-03-04T15:42:23.345097+0000 mon.a (mon.0) 268 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:42:24.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:23 smithi117 bash[28843]: cluster 2024-03-04T15:42:22.725199+0000 mgr.y (mgr.14152) 92 : cluster 0 pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:24.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:23 smithi117 bash[28843]: audit 2024-03-04T15:42:23.339678+0000 mon.c (mon.2) 3 : audit 1 from='client.? 172.21.15.117:0/4271088755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf1f40a0-bf1a-4681-84de-fbf6fee19de7"}]: dispatch 2024-03-04T15:42:24.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:23 smithi117 bash[28843]: audit 2024-03-04T15:42:23.340394+0000 mon.a (mon.0) 265 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf1f40a0-bf1a-4681-84de-fbf6fee19de7"}]: dispatch 2024-03-04T15:42:24.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:23 smithi117 bash[28843]: audit 2024-03-04T15:42:23.342480+0000 mon.a (mon.0) 266 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf1f40a0-bf1a-4681-84de-fbf6fee19de7"}]': finished 2024-03-04T15:42:24.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:23 smithi117 bash[28843]: cluster 2024-03-04T15:42:23.344861+0000 mon.a (mon.0) 267 : cluster 0 osdmap e5: 1 total, 0 up, 1 in 2024-03-04T15:42:24.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:23 smithi117 bash[28843]: audit 2024-03-04T15:42:23.345097+0000 mon.a (mon.0) 268 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:42:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:23 smithi151 bash[23756]: cluster 2024-03-04T15:42:22.725199+0000 mgr.y (mgr.14152) 92 : cluster 0 pgmap v54: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:23 smithi151 bash[23756]: audit 2024-03-04T15:42:23.339678+0000 mon.c (mon.2) 3 : audit 1 from='client.? 172.21.15.117:0/4271088755' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf1f40a0-bf1a-4681-84de-fbf6fee19de7"}]: dispatch 2024-03-04T15:42:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:23 smithi151 bash[23756]: audit 2024-03-04T15:42:23.340394+0000 mon.a (mon.0) 265 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "cf1f40a0-bf1a-4681-84de-fbf6fee19de7"}]: dispatch 2024-03-04T15:42:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:23 smithi151 bash[23756]: audit 2024-03-04T15:42:23.342480+0000 mon.a (mon.0) 266 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "cf1f40a0-bf1a-4681-84de-fbf6fee19de7"}]': finished 2024-03-04T15:42:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:23 smithi151 bash[23756]: cluster 2024-03-04T15:42:23.344861+0000 mon.a (mon.0) 267 : cluster 0 osdmap e5: 1 total, 0 up, 1 in 2024-03-04T15:42:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:23 smithi151 bash[23756]: audit 2024-03-04T15:42:23.345097+0000 mon.a (mon.0) 268 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:42:25.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:24 smithi117 bash[21480]: audit 2024-03-04T15:42:24.083546+0000 mon.a (mon.0) 269 : audit 0 from='client.? 172.21.15.117:0/3804805389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:42:25.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:24 smithi117 bash[28843]: audit 2024-03-04T15:42:24.083546+0000 mon.a (mon.0) 269 : audit 0 from='client.? 172.21.15.117:0/3804805389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:42:25.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:24 smithi151 bash[23756]: audit 2024-03-04T15:42:24.083546+0000 mon.a (mon.0) 269 : audit 0 from='client.? 172.21.15.117:0/3804805389' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:42:26.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:25 smithi117 bash[21480]: cluster 2024-03-04T15:42:24.725446+0000 mgr.y (mgr.14152) 93 : cluster 0 pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:26.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:25 smithi117 bash[28843]: cluster 2024-03-04T15:42:24.725446+0000 mgr.y (mgr.14152) 93 : cluster 0 pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:25 smithi151 bash[23756]: cluster 2024-03-04T15:42:24.725446+0000 mgr.y (mgr.14152) 93 : cluster 0 pgmap v56: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:28.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:27 smithi117 bash[21480]: cluster 2024-03-04T15:42:26.725829+0000 mgr.y (mgr.14152) 94 : cluster 0 pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:28.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:27 smithi117 bash[28843]: cluster 2024-03-04T15:42:26.725829+0000 mgr.y (mgr.14152) 94 : cluster 0 pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:28.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:27 smithi151 bash[23756]: cluster 2024-03-04T15:42:26.725829+0000 mgr.y (mgr.14152) 94 : cluster 0 pgmap v57: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:30.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:29 smithi117 bash[21480]: cluster 2024-03-04T15:42:28.726250+0000 mgr.y (mgr.14152) 95 : cluster 0 pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:30.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:29 smithi117 bash[28843]: cluster 2024-03-04T15:42:28.726250+0000 mgr.y (mgr.14152) 95 : cluster 0 pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:30.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:29 smithi151 bash[23756]: cluster 2024-03-04T15:42:28.726250+0000 mgr.y (mgr.14152) 95 : cluster 0 pgmap v58: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:32.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:31 smithi117 bash[21480]: cluster 2024-03-04T15:42:30.726741+0000 mgr.y (mgr.14152) 96 : cluster 0 pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:32.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:31 smithi117 bash[28843]: cluster 2024-03-04T15:42:30.726741+0000 mgr.y (mgr.14152) 96 : cluster 0 pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:32.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:31 smithi151 bash[23756]: cluster 2024-03-04T15:42:30.726741+0000 mgr.y (mgr.14152) 96 : cluster 0 pgmap v59: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:34.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:33 smithi117 bash[21480]: cluster 2024-03-04T15:42:32.727182+0000 mgr.y (mgr.14152) 97 : cluster 0 pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:34.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:33 smithi117 bash[28843]: cluster 2024-03-04T15:42:32.727182+0000 mgr.y (mgr.14152) 97 : cluster 0 pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:34.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:33 smithi151 bash[23756]: cluster 2024-03-04T15:42:32.727182+0000 mgr.y (mgr.14152) 97 : cluster 0 pgmap v60: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:36.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:35 smithi117 bash[21480]: cluster 2024-03-04T15:42:34.727563+0000 mgr.y (mgr.14152) 98 : cluster 0 pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:36.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:35 smithi117 bash[21480]: audit 2024-03-04T15:42:35.680876+0000 mon.a (mon.0) 270 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-04T15:42:36.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:35 smithi117 bash[21480]: audit 2024-03-04T15:42:35.682171+0000 mon.a (mon.0) 271 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:42:36.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:35 smithi117 bash[21480]: cephadm 2024-03-04T15:42:35.683390+0000 mgr.y (mgr.14152) 99 : cephadm 1 Deploying daemon osd.0 on smithi117 2024-03-04T15:42:36.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:35 smithi117 bash[28843]: cluster 2024-03-04T15:42:34.727563+0000 mgr.y (mgr.14152) 98 : cluster 0 pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:36.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:35 smithi117 bash[28843]: audit 2024-03-04T15:42:35.680876+0000 mon.a (mon.0) 270 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-04T15:42:36.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:35 smithi117 bash[28843]: audit 2024-03-04T15:42:35.682171+0000 mon.a (mon.0) 271 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:42:36.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:35 smithi117 bash[28843]: cephadm 2024-03-04T15:42:35.683390+0000 mgr.y (mgr.14152) 99 : cephadm 1 Deploying daemon osd.0 on smithi117 2024-03-04T15:42:36.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:35 smithi151 bash[23756]: cluster 2024-03-04T15:42:34.727563+0000 mgr.y (mgr.14152) 98 : cluster 0 pgmap v61: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:36.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:35 smithi151 bash[23756]: audit 2024-03-04T15:42:35.680876+0000 mon.a (mon.0) 270 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-03-04T15:42:36.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:35 smithi151 bash[23756]: audit 2024-03-04T15:42:35.682171+0000 mon.a (mon.0) 271 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:42:36.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:35 smithi151 bash[23756]: cephadm 2024-03-04T15:42:35.683390+0000 mgr.y (mgr.14152) 99 : cephadm 1 Deploying daemon osd.0 on smithi117 2024-03-04T15:42:38.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:37 smithi117 bash[28843]: cluster 2024-03-04T15:42:36.728015+0000 mgr.y (mgr.14152) 100 : cluster 0 pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:38.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:37 smithi117 bash[21480]: cluster 2024-03-04T15:42:36.728015+0000 mgr.y (mgr.14152) 100 : cluster 0 pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:38.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:37 smithi151 bash[23756]: cluster 2024-03-04T15:42:36.728015+0000 mgr.y (mgr.14152) 100 : cluster 0 pgmap v62: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:39.759 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:39 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:42:39.759 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:42:39 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:42:39.760 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:39 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:42:40.071 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:42:39 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:42:40.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:39 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:42:40.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:39 smithi117 bash[28843]: cluster 2024-03-04T15:42:38.728452+0000 mgr.y (mgr.14152) 101 : cluster 0 pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:40.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:39 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:42:40.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:39 smithi117 bash[21480]: cluster 2024-03-04T15:42:38.728452+0000 mgr.y (mgr.14152) 101 : cluster 0 pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:40.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:39 smithi151 bash[23756]: cluster 2024-03-04T15:42:38.728452+0000 mgr.y (mgr.14152) 101 : cluster 0 pgmap v63: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:41.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:40 smithi117 bash[28843]: audit 2024-03-04T15:42:39.899798+0000 mon.a (mon.0) 272 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:41.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:40 smithi117 bash[28843]: audit 2024-03-04T15:42:39.907144+0000 mon.a (mon.0) 273 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:41.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:40 smithi117 bash[28843]: audit 2024-03-04T15:42:39.926675+0000 mon.a (mon.0) 274 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:42:41.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:40 smithi117 bash[21480]: audit 2024-03-04T15:42:39.899798+0000 mon.a (mon.0) 272 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:41.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:40 smithi117 bash[21480]: audit 2024-03-04T15:42:39.907144+0000 mon.a (mon.0) 273 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:41.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:40 smithi117 bash[21480]: audit 2024-03-04T15:42:39.926675+0000 mon.a (mon.0) 274 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:42:41.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:40 smithi151 bash[23756]: audit 2024-03-04T15:42:39.899798+0000 mon.a (mon.0) 272 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:41.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:40 smithi151 bash[23756]: audit 2024-03-04T15:42:39.907144+0000 mon.a (mon.0) 273 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:41.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:40 smithi151 bash[23756]: audit 2024-03-04T15:42:39.926675+0000 mon.a (mon.0) 274 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:42:42.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:41 smithi151 bash[23756]: cluster 2024-03-04T15:42:40.728873+0000 mgr.y (mgr.14152) 102 : cluster 0 pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:42.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:41 smithi117 bash[21480]: cluster 2024-03-04T15:42:40.728873+0000 mgr.y (mgr.14152) 102 : cluster 0 pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:42.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:41 smithi117 bash[28843]: cluster 2024-03-04T15:42:40.728873+0000 mgr.y (mgr.14152) 102 : cluster 0 pgmap v64: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:44.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:43 smithi151 bash[23756]: cluster 2024-03-04T15:42:42.729218+0000 mgr.y (mgr.14152) 103 : cluster 0 pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:44.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:43 smithi117 bash[21480]: cluster 2024-03-04T15:42:42.729218+0000 mgr.y (mgr.14152) 103 : cluster 0 pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:44.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:43 smithi117 bash[28843]: cluster 2024-03-04T15:42:42.729218+0000 mgr.y (mgr.14152) 103 : cluster 0 pgmap v65: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:46.249 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:46 smithi117 bash[21480]: cluster 2024-03-04T15:42:44.729597+0000 mgr.y (mgr.14152) 104 : cluster 0 pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:46.249 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:46 smithi117 bash[21480]: audit 2024-03-04T15:42:45.158684+0000 mon.a (mon.0) 275 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:46.249 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:46 smithi117 bash[21480]: audit 2024-03-04T15:42:45.162627+0000 mon.a (mon.0) 276 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:46.249 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:46 smithi117 bash[21480]: audit 2024-03-04T15:42:45.733833+0000 mon.a (mon.0) 277 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:42:46.249 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:46 smithi117 bash[21480]: audit 2024-03-04T15:42:45.735527+0000 mon.a (mon.0) 278 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:42:46.249 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:46 smithi117 bash[21480]: audit 2024-03-04T15:42:45.740212+0000 mon.a (mon.0) 279 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:46.249 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:46 smithi117 bash[28843]: cluster 2024-03-04T15:42:44.729597+0000 mgr.y (mgr.14152) 104 : cluster 0 pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:46.249 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:46 smithi117 bash[28843]: audit 2024-03-04T15:42:45.158684+0000 mon.a (mon.0) 275 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:46.250 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:46 smithi117 bash[28843]: audit 2024-03-04T15:42:45.162627+0000 mon.a (mon.0) 276 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:46.250 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:46 smithi117 bash[28843]: audit 2024-03-04T15:42:45.733833+0000 mon.a (mon.0) 277 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:42:46.250 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:46 smithi117 bash[28843]: audit 2024-03-04T15:42:45.735527+0000 mon.a (mon.0) 278 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:42:46.250 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:46 smithi117 bash[28843]: audit 2024-03-04T15:42:45.740212+0000 mon.a (mon.0) 279 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:46.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:46 smithi151 bash[23756]: cluster 2024-03-04T15:42:44.729597+0000 mgr.y (mgr.14152) 104 : cluster 0 pgmap v66: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:46.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:46 smithi151 bash[23756]: audit 2024-03-04T15:42:45.158684+0000 mon.a (mon.0) 275 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:46.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:46 smithi151 bash[23756]: audit 2024-03-04T15:42:45.162627+0000 mon.a (mon.0) 276 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:46.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:46 smithi151 bash[23756]: audit 2024-03-04T15:42:45.733833+0000 mon.a (mon.0) 277 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:42:46.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:46 smithi151 bash[23756]: audit 2024-03-04T15:42:45.735527+0000 mon.a (mon.0) 278 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:42:46.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:46 smithi151 bash[23756]: audit 2024-03-04T15:42:45.740212+0000 mon.a (mon.0) 279 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:47.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:47 smithi151 bash[23756]: audit 2024-03-04T15:42:46.252070+0000 mon.a (mon.0) 280 : audit 1 from='osd.0 [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-04T15:42:47.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:47 smithi117 bash[21480]: audit 2024-03-04T15:42:46.252070+0000 mon.a (mon.0) 280 : audit 1 from='osd.0 [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-04T15:42:47.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:47 smithi117 bash[28843]: audit 2024-03-04T15:42:46.252070+0000 mon.a (mon.0) 280 : audit 1 from='osd.0 [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-03-04T15:42:48.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:48 smithi151 bash[23756]: cluster 2024-03-04T15:42:46.730056+0000 mgr.y (mgr.14152) 105 : cluster 0 pgmap v67: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:48.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:48 smithi151 bash[23756]: audit 2024-03-04T15:42:47.165967+0000 mon.a (mon.0) 281 : audit 1 from='osd.0 [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-04T15:42:48.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:48 smithi151 bash[23756]: cluster 2024-03-04T15:42:47.168275+0000 mon.a (mon.0) 282 : cluster 0 osdmap e6: 1 total, 0 up, 1 in 2024-03-04T15:42:48.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:48 smithi151 bash[23756]: audit 2024-03-04T15:42:47.168617+0000 mon.a (mon.0) 283 : audit 1 from='osd.0 [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-04T15:42:48.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:48 smithi151 bash[23756]: audit 2024-03-04T15:42:47.168909+0000 mon.a (mon.0) 284 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:42:48.502 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:48.499+0000 7ff123fff700 1 -- 172.21.15.117:0/3321444526 <== mgr.14152 v2:172.21.15.117:6800/2865911351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff13c02db10 con 0x7ff12406d600 2024-03-04T15:42:48.503 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 0 on host 'smithi117' 2024-03-04T15:42:48.505 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:48.503+0000 7ff141042700 1 -- 172.21.15.117:0/3321444526 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7ff12406d600 msgr2=0x7ff12406fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:42:48.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:48.503+0000 7ff141042700 1 --2- 172.21.15.117:0/3321444526 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7ff12406d600 0x7ff12406fac0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff130006010 tx=0x7ff13001a040 comp rx=0 tx=0).stop 2024-03-04T15:42:48.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:48.503+0000 7ff141042700 1 -- 172.21.15.117:0/3321444526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff13c101e70 msgr2=0x7ff13c1975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:42:48.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:48.503+0000 7ff141042700 1 --2- 172.21.15.117:0/3321444526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff13c101e70 0x7ff13c1975b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7ff12c00e9d0 tx=0x7ff12c00ece0 comp rx=0 tx=0).stop 2024-03-04T15:42:48.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:48.503+0000 7ff141042700 1 -- 172.21.15.117:0/3321444526 shutdown_connections 2024-03-04T15:42:48.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:48.503+0000 7ff141042700 1 --2- 172.21.15.117:0/3321444526 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff13c103990 0x7ff13c19eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:42:48.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:48.503+0000 7ff141042700 1 --2- 172.21.15.117:0/3321444526 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7ff12406d600 0x7ff12406fac0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:42:48.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:48.503+0000 7ff141042700 1 --2- 172.21.15.117:0/3321444526 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff13c101e70 0x7ff13c1975b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:42:48.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:48.503+0000 7ff141042700 1 --2- 172.21.15.117:0/3321444526 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff13c102fd0 0x7ff13c197af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:42:48.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:48.503+0000 7ff141042700 1 -- 172.21.15.117:0/3321444526 >> 172.21.15.117:0/3321444526 conn(0x7ff13c0fd410 msgr2=0x7ff13c0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:42:48.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:48.503+0000 7ff141042700 1 -- 172.21.15.117:0/3321444526 shutdown_connections 2024-03-04T15:42:48.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:42:48.503+0000 7ff141042700 1 -- 172.21.15.117:0/3321444526 wait complete. 2024-03-04T15:42:48.520 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:48 smithi117 bash[28843]: cluster 2024-03-04T15:42:46.730056+0000 mgr.y (mgr.14152) 105 : cluster 0 pgmap v67: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:48.520 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:48 smithi117 bash[28843]: audit 2024-03-04T15:42:47.165967+0000 mon.a (mon.0) 281 : audit 1 from='osd.0 [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-04T15:42:48.520 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:48 smithi117 bash[28843]: cluster 2024-03-04T15:42:47.168275+0000 mon.a (mon.0) 282 : cluster 0 osdmap e6: 1 total, 0 up, 1 in 2024-03-04T15:42:48.520 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:48 smithi117 bash[28843]: audit 2024-03-04T15:42:47.168617+0000 mon.a (mon.0) 283 : audit 1 from='osd.0 [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-04T15:42:48.520 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:48 smithi117 bash[28843]: audit 2024-03-04T15:42:47.168909+0000 mon.a (mon.0) 284 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:42:48.520 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:48 smithi117 bash[21480]: cluster 2024-03-04T15:42:46.730056+0000 mgr.y (mgr.14152) 105 : cluster 0 pgmap v67: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:48.521 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:48 smithi117 bash[21480]: audit 2024-03-04T15:42:47.165967+0000 mon.a (mon.0) 281 : audit 1 from='osd.0 [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-03-04T15:42:48.521 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:48 smithi117 bash[21480]: cluster 2024-03-04T15:42:47.168275+0000 mon.a (mon.0) 282 : cluster 0 osdmap e6: 1 total, 0 up, 1 in 2024-03-04T15:42:48.521 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:48 smithi117 bash[21480]: audit 2024-03-04T15:42:47.168617+0000 mon.a (mon.0) 283 : audit 1 from='osd.0 [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-04T15:42:48.521 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:48 smithi117 bash[21480]: audit 2024-03-04T15:42:47.168909+0000 mon.a (mon.0) 284 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:42:49.196 DEBUG:teuthology.orchestra.run.smithi117:osd.0> sudo journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.0.service 2024-03-04T15:42:49.199 INFO:tasks.cephadm:Deploying osd.1 on smithi117 with /dev/vg_nvme/lv_3... 2024-03-04T15:42:49.199 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-04T15:42:49.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:49 smithi151 bash[23756]: audit 2024-03-04T15:42:48.168266+0000 mon.a (mon.0) 285 : audit 1 from='osd.0 [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-03-04T15:42:49.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:49 smithi151 bash[23756]: cluster 2024-03-04T15:42:48.175644+0000 mon.a (mon.0) 286 : cluster 0 osdmap e7: 1 total, 0 up, 1 in 2024-03-04T15:42:49.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:49 smithi151 bash[23756]: audit 2024-03-04T15:42:48.177617+0000 mon.a (mon.0) 287 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:42:49.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:49 smithi151 bash[23756]: audit 2024-03-04T15:42:48.187117+0000 mon.a (mon.0) 288 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:42:49.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:49 smithi151 bash[23756]: audit 2024-03-04T15:42:48.404275+0000 mon.a (mon.0) 289 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:49.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:49 smithi151 bash[23756]: audit 2024-03-04T15:42:48.409911+0000 mon.a (mon.0) 290 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:42:49.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:49 smithi151 bash[23756]: audit 2024-03-04T15:42:48.447758+0000 mon.a (mon.0) 291 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:49.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:49 smithi151 bash[23756]: audit 2024-03-04T15:42:48.667037+0000 mon.a (mon.0) 292 : audit 1 from='osd.0 [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120]' entity='osd.0' 2024-03-04T15:42:49.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:49 smithi117 bash[21480]: audit 2024-03-04T15:42:48.168266+0000 mon.a (mon.0) 285 : audit 1 from='osd.0 [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-03-04T15:42:49.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:49 smithi117 bash[21480]: cluster 2024-03-04T15:42:48.175644+0000 mon.a (mon.0) 286 : cluster 0 osdmap e7: 1 total, 0 up, 1 in 2024-03-04T15:42:49.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:49 smithi117 bash[21480]: audit 2024-03-04T15:42:48.177617+0000 mon.a (mon.0) 287 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:42:49.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:49 smithi117 bash[21480]: audit 2024-03-04T15:42:48.187117+0000 mon.a (mon.0) 288 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:42:49.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:49 smithi117 bash[21480]: audit 2024-03-04T15:42:48.404275+0000 mon.a (mon.0) 289 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:49.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:49 smithi117 bash[21480]: audit 2024-03-04T15:42:48.409911+0000 mon.a (mon.0) 290 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:42:49.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:49 smithi117 bash[21480]: audit 2024-03-04T15:42:48.447758+0000 mon.a (mon.0) 291 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:49.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:49 smithi117 bash[21480]: audit 2024-03-04T15:42:48.667037+0000 mon.a (mon.0) 292 : audit 1 from='osd.0 [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120]' entity='osd.0' 2024-03-04T15:42:49.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:49 smithi117 bash[28843]: audit 2024-03-04T15:42:48.168266+0000 mon.a (mon.0) 285 : audit 1 from='osd.0 [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-03-04T15:42:49.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:49 smithi117 bash[28843]: cluster 2024-03-04T15:42:48.175644+0000 mon.a (mon.0) 286 : cluster 0 osdmap e7: 1 total, 0 up, 1 in 2024-03-04T15:42:49.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:49 smithi117 bash[28843]: audit 2024-03-04T15:42:48.177617+0000 mon.a (mon.0) 287 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:42:49.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:49 smithi117 bash[28843]: audit 2024-03-04T15:42:48.187117+0000 mon.a (mon.0) 288 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:42:49.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:49 smithi117 bash[28843]: audit 2024-03-04T15:42:48.404275+0000 mon.a (mon.0) 289 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:49.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:49 smithi117 bash[28843]: audit 2024-03-04T15:42:48.409911+0000 mon.a (mon.0) 290 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:42:49.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:49 smithi117 bash[28843]: audit 2024-03-04T15:42:48.447758+0000 mon.a (mon.0) 291 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:42:49.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:49 smithi117 bash[28843]: audit 2024-03-04T15:42:48.667037+0000 mon.a (mon.0) 292 : audit 1 from='osd.0 [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120]' entity='osd.0' 2024-03-04T15:42:50.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:50 smithi117 bash[28843]: cluster 2024-03-04T15:42:47.268915+0000 osd.0 (osd.0) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:42:50.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:50 smithi117 bash[28843]: cluster 2024-03-04T15:42:47.269058+0000 osd.0 (osd.0) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:42:50.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:50 smithi117 bash[28843]: cluster 2024-03-04T15:42:48.730361+0000 mgr.y (mgr.14152) 106 : cluster 0 pgmap v70: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:50.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:50 smithi117 bash[28843]: audit 2024-03-04T15:42:49.185836+0000 mon.a (mon.0) 293 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:42:50.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:50 smithi117 bash[28843]: cluster 2024-03-04T15:42:49.680648+0000 mon.a (mon.0) 294 : cluster 1 osd.0 [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120] boot 2024-03-04T15:42:50.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:50 smithi117 bash[28843]: cluster 2024-03-04T15:42:49.680732+0000 mon.a (mon.0) 295 : cluster 0 osdmap e8: 1 total, 1 up, 1 in 2024-03-04T15:42:50.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:50 smithi117 bash[28843]: audit 2024-03-04T15:42:49.681034+0000 mon.a (mon.0) 296 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:42:50.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:50 smithi117 bash[21480]: cluster 2024-03-04T15:42:47.268915+0000 osd.0 (osd.0) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:42:50.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:50 smithi117 bash[21480]: cluster 2024-03-04T15:42:47.269058+0000 osd.0 (osd.0) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:42:50.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:50 smithi117 bash[21480]: cluster 2024-03-04T15:42:48.730361+0000 mgr.y (mgr.14152) 106 : cluster 0 pgmap v70: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:50.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:50 smithi117 bash[21480]: audit 2024-03-04T15:42:49.185836+0000 mon.a (mon.0) 293 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:42:50.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:50 smithi117 bash[21480]: cluster 2024-03-04T15:42:49.680648+0000 mon.a (mon.0) 294 : cluster 1 osd.0 [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120] boot 2024-03-04T15:42:50.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:50 smithi117 bash[21480]: cluster 2024-03-04T15:42:49.680732+0000 mon.a (mon.0) 295 : cluster 0 osdmap e8: 1 total, 1 up, 1 in 2024-03-04T15:42:50.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:50 smithi117 bash[21480]: audit 2024-03-04T15:42:49.681034+0000 mon.a (mon.0) 296 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:42:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:50 smithi151 bash[23756]: cluster 2024-03-04T15:42:47.268915+0000 osd.0 (osd.0) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:42:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:50 smithi151 bash[23756]: cluster 2024-03-04T15:42:47.269058+0000 osd.0 (osd.0) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:42:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:50 smithi151 bash[23756]: cluster 2024-03-04T15:42:48.730361+0000 mgr.y (mgr.14152) 106 : cluster 0 pgmap v70: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-03-04T15:42:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:50 smithi151 bash[23756]: audit 2024-03-04T15:42:49.185836+0000 mon.a (mon.0) 293 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:42:50.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:50 smithi151 bash[23756]: cluster 2024-03-04T15:42:49.680648+0000 mon.a (mon.0) 294 : cluster 1 osd.0 [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120] boot 2024-03-04T15:42:50.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:50 smithi151 bash[23756]: cluster 2024-03-04T15:42:49.680732+0000 mon.a (mon.0) 295 : cluster 0 osdmap e8: 1 total, 1 up, 1 in 2024-03-04T15:42:50.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:50 smithi151 bash[23756]: audit 2024-03-04T15:42:49.681034+0000 mon.a (mon.0) 296 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:42:52.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:51 smithi117 bash[21480]: cluster 2024-03-04T15:42:50.685416+0000 mon.a (mon.0) 297 : cluster 0 osdmap e9: 1 total, 1 up, 1 in 2024-03-04T15:42:52.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:51 smithi117 bash[28843]: cluster 2024-03-04T15:42:50.685416+0000 mon.a (mon.0) 297 : cluster 0 osdmap e9: 1 total, 1 up, 1 in 2024-03-04T15:42:52.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:51 smithi151 bash[23756]: cluster 2024-03-04T15:42:50.685416+0000 mon.a (mon.0) 297 : cluster 0 osdmap e9: 1 total, 1 up, 1 in 2024-03-04T15:42:53.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:52 smithi117 bash[21480]: cluster 2024-03-04T15:42:50.730838+0000 mgr.y (mgr.14152) 107 : cluster 0 pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:42:53.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:52 smithi117 bash[28843]: cluster 2024-03-04T15:42:50.730838+0000 mgr.y (mgr.14152) 107 : cluster 0 pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:42:53.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:52 smithi151 bash[23756]: cluster 2024-03-04T15:42:50.730838+0000 mgr.y (mgr.14152) 107 : cluster 0 pgmap v73: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:42:53.445 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:42:54.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:54 smithi117 bash[21480]: cluster 2024-03-04T15:42:52.731335+0000 mgr.y (mgr.14152) 108 : cluster 0 pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:42:54.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:54 smithi117 bash[28843]: cluster 2024-03-04T15:42:52.731335+0000 mgr.y (mgr.14152) 108 : cluster 0 pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:42:55.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:54 smithi151 bash[23756]: cluster 2024-03-04T15:42:52.731335+0000 mgr.y (mgr.14152) 108 : cluster 0 pgmap v74: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:42:57.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:56 smithi117 bash[21480]: cluster 2024-03-04T15:42:54.731712+0000 mgr.y (mgr.14152) 109 : cluster 0 pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:42:57.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:56 smithi117 bash[28843]: cluster 2024-03-04T15:42:54.731712+0000 mgr.y (mgr.14152) 109 : cluster 0 pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:42:57.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:56 smithi151 bash[23756]: cluster 2024-03-04T15:42:54.731712+0000 mgr.y (mgr.14152) 109 : cluster 0 pgmap v75: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:42:57.596 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-04T15:42:57.597 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-04T15:42:57.597 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-03-04T15:42:57.597 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-03-04T15:42:57.597 INFO:teuthology.orchestra.run.smithi117.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0150082 s, 699 MB/s 2024-03-04T15:42:57.597 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-03-04T15:42:58.417 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi117:vg_nvme/lv_3 2024-03-04T15:42:59.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:58 smithi117 bash[21480]: cluster 2024-03-04T15:42:56.732131+0000 mgr.y (mgr.14152) 110 : cluster 0 pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:42:59.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:58 smithi117 bash[28843]: cluster 2024-03-04T15:42:56.732131+0000 mgr.y (mgr.14152) 110 : cluster 0 pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:42:59.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:58 smithi151 bash[23756]: cluster 2024-03-04T15:42:56.732131+0000 mgr.y (mgr.14152) 110 : cluster 0 pgmap v76: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:00.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:59 smithi117 bash[21480]: cluster 2024-03-04T15:42:58.732616+0000 mgr.y (mgr.14152) 111 : cluster 0 pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:00.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:59 smithi117 bash[21480]: cephadm 2024-03-04T15:42:58.750658+0000 mgr.y (mgr.14152) 112 : cephadm 1 Detected new or changed devices on smithi117 2024-03-04T15:43:00.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:59 smithi117 bash[21480]: audit 2024-03-04T15:42:58.758808+0000 mon.a (mon.0) 298 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:00.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:59 smithi117 bash[21480]: audit 2024-03-04T15:42:58.767917+0000 mon.a (mon.0) 299 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:00.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:59 smithi117 bash[21480]: audit 2024-03-04T15:42:58.770132+0000 mon.a (mon.0) 300 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:43:00.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:59 smithi117 bash[21480]: cephadm 2024-03-04T15:42:58.771451+0000 mgr.y (mgr.14152) 113 : cephadm 1 Adjusting osd_memory_target on smithi117 to 16241M 2024-03-04T15:43:00.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:59 smithi117 bash[21480]: audit 2024-03-04T15:42:58.779510+0000 mon.a (mon.0) 301 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:00.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:59 smithi117 bash[21480]: audit 2024-03-04T15:42:58.784526+0000 mon.a (mon.0) 302 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:43:00.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:59 smithi117 bash[21480]: audit 2024-03-04T15:42:58.786091+0000 mon.a (mon.0) 303 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:43:00.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:42:59 smithi117 bash[21480]: audit 2024-03-04T15:42:58.797699+0000 mon.a (mon.0) 304 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:00.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:59 smithi117 bash[28843]: cluster 2024-03-04T15:42:58.732616+0000 mgr.y (mgr.14152) 111 : cluster 0 pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:00.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:59 smithi117 bash[28843]: cephadm 2024-03-04T15:42:58.750658+0000 mgr.y (mgr.14152) 112 : cephadm 1 Detected new or changed devices on smithi117 2024-03-04T15:43:00.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:59 smithi117 bash[28843]: audit 2024-03-04T15:42:58.758808+0000 mon.a (mon.0) 298 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:00.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:59 smithi117 bash[28843]: audit 2024-03-04T15:42:58.767917+0000 mon.a (mon.0) 299 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:00.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:59 smithi117 bash[28843]: audit 2024-03-04T15:42:58.770132+0000 mon.a (mon.0) 300 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:43:00.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:59 smithi117 bash[28843]: cephadm 2024-03-04T15:42:58.771451+0000 mgr.y (mgr.14152) 113 : cephadm 1 Adjusting osd_memory_target on smithi117 to 16241M 2024-03-04T15:43:00.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:59 smithi117 bash[28843]: audit 2024-03-04T15:42:58.779510+0000 mon.a (mon.0) 301 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:00.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:59 smithi117 bash[28843]: audit 2024-03-04T15:42:58.784526+0000 mon.a (mon.0) 302 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:43:00.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:59 smithi117 bash[28843]: audit 2024-03-04T15:42:58.786091+0000 mon.a (mon.0) 303 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:43:00.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:42:59 smithi117 bash[28843]: audit 2024-03-04T15:42:58.797699+0000 mon.a (mon.0) 304 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:00.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:59 smithi151 bash[23756]: cluster 2024-03-04T15:42:58.732616+0000 mgr.y (mgr.14152) 111 : cluster 0 pgmap v77: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:00.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:59 smithi151 bash[23756]: cephadm 2024-03-04T15:42:58.750658+0000 mgr.y (mgr.14152) 112 : cephadm 1 Detected new or changed devices on smithi117 2024-03-04T15:43:00.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:59 smithi151 bash[23756]: audit 2024-03-04T15:42:58.758808+0000 mon.a (mon.0) 298 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:00.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:59 smithi151 bash[23756]: audit 2024-03-04T15:42:58.767917+0000 mon.a (mon.0) 299 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:00.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:59 smithi151 bash[23756]: audit 2024-03-04T15:42:58.770132+0000 mon.a (mon.0) 300 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:43:00.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:59 smithi151 bash[23756]: cephadm 2024-03-04T15:42:58.771451+0000 mgr.y (mgr.14152) 113 : cephadm 1 Adjusting osd_memory_target on smithi117 to 16241M 2024-03-04T15:43:00.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:59 smithi151 bash[23756]: audit 2024-03-04T15:42:58.779510+0000 mon.a (mon.0) 301 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:00.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:59 smithi151 bash[23756]: audit 2024-03-04T15:42:58.784526+0000 mon.a (mon.0) 302 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:43:00.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:59 smithi151 bash[23756]: audit 2024-03-04T15:42:58.786091+0000 mon.a (mon.0) 303 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:43:00.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:42:59 smithi151 bash[23756]: audit 2024-03-04T15:42:58.797699+0000 mon.a (mon.0) 304 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:02.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:01 smithi117 bash[21480]: cluster 2024-03-04T15:43:00.733131+0000 mgr.y (mgr.14152) 114 : cluster 0 pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:02.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:01 smithi117 bash[28843]: cluster 2024-03-04T15:43:00.733131+0000 mgr.y (mgr.14152) 114 : cluster 0 pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:02.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:01 smithi151 bash[23756]: cluster 2024-03-04T15:43:00.733131+0000 mgr.y (mgr.14152) 114 : cluster 0 pgmap v78: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:03.146 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:43:04.057 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:03 smithi117 bash[28843]: cluster 2024-03-04T15:43:02.733724+0000 mgr.y (mgr.14152) 115 : cluster 0 pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:04.058 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:03 smithi117 bash[21480]: cluster 2024-03-04T15:43:02.733724+0000 mgr.y (mgr.14152) 115 : cluster 0 pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:04.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:03 smithi151 bash[23756]: cluster 2024-03-04T15:43:02.733724+0000 mgr.y (mgr.14152) 115 : cluster 0 pgmap v79: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:05.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.451+0000 7fd427b10700 1 -- 172.21.15.117:0/1015438501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420107b60 msgr2=0x7fd420109fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:43:05.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.451+0000 7fd427b10700 1 --2- 172.21.15.117:0/1015438501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420107b60 0x7fd420109fd0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd418009f90 tx=0x7fd41800b330 comp rx=0 tx=0).stop 2024-03-04T15:43:05.458 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd427b10700 1 -- 172.21.15.117:0/1015438501 shutdown_connections 2024-03-04T15:43:05.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd427b10700 1 --2- 172.21.15.117:0/1015438501 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420107b60 0x7fd420109fd0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:43:05.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd427b10700 1 --2- 172.21.15.117:0/1015438501 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd420102900 0x7fd420104cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:43:05.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd427b10700 1 --2- 172.21.15.117:0/1015438501 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd420105230 0x7fd420107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:43:05.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd427b10700 1 -- 172.21.15.117:0/1015438501 >> 172.21.15.117:0/1015438501 conn(0x7fd4200fc310 msgr2=0x7fd4200fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:43:05.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd427b10700 1 -- 172.21.15.117:0/1015438501 shutdown_connections 2024-03-04T15:43:05.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd427b10700 1 -- 172.21.15.117:0/1015438501 wait complete. 2024-03-04T15:43:05.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd427b10700 1 Processor -- start 2024-03-04T15:43:05.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd427b10700 1 -- start start 2024-03-04T15:43:05.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd427b10700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd420102900 0x7fd420197570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:43:05.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd427b10700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420105230 0x7fd420197ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:43:05.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd427b10700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd420107b60 0x7fd42019eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:43:05.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd427b10700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42010ce90 con 0x7fd420105230 2024-03-04T15:43:05.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd427b10700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd42010cd10 con 0x7fd420102900 2024-03-04T15:43:05.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd427b10700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd42010d010 con 0x7fd420107b60 2024-03-04T15:43:05.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd4250ab700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420105230 0x7fd420197ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:43:05.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd4250ab700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420105230 0x7fd420197ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53052/0 (socket says 172.21.15.117:53052) 2024-03-04T15:43:05.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd4250ab700 1 -- 172.21.15.117:0/871208415 learned_addr learned my addr 172.21.15.117:0/871208415 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:43:05.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd4260ad700 1 --2- 172.21.15.117:0/871208415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd420107b60 0x7fd42019eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:43:05.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd4258ac700 1 --2- 172.21.15.117:0/871208415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd420102900 0x7fd420197570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:43:05.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd4260ad700 1 -- 172.21.15.117:0/871208415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd420102900 msgr2=0x7fd420197570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:43:05.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd4260ad700 1 --2- 172.21.15.117:0/871208415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd420102900 0x7fd420197570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:43:05.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd4260ad700 1 -- 172.21.15.117:0/871208415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420105230 msgr2=0x7fd420197ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:43:05.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd4260ad700 1 --2- 172.21.15.117:0/871208415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420105230 0x7fd420197ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:43:05.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd4260ad700 1 -- 172.21.15.117:0/871208415 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd42019f180 con 0x7fd420107b60 2024-03-04T15:43:05.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd4250ab700 1 --2- 172.21.15.117:0/871208415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420105230 0x7fd420197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:43:05.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd4260ad700 1 --2- 172.21.15.117:0/871208415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd420107b60 0x7fd42019eae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd418009f90 tx=0x7fd418009b90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:43:05.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd416ffd700 1 -- 172.21.15.117:0/871208415 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd418007ac0 con 0x7fd420107b60 2024-03-04T15:43:05.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd416ffd700 1 -- 172.21.15.117:0/871208415 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd41801cb40 con 0x7fd420107b60 2024-03-04T15:43:05.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd427b10700 1 -- 172.21.15.117:0/871208415 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4201006f0 con 0x7fd420107b60 2024-03-04T15:43:05.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd416ffd700 1 -- 172.21.15.117:0/871208415 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd418027920 con 0x7fd420107b60 2024-03-04T15:43:05.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd427b10700 1 -- 172.21.15.117:0/871208415 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd420100c30 con 0x7fd420107b60 2024-03-04T15:43:05.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd427b10700 1 -- 172.21.15.117:0/871208415 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd42004faa0 con 0x7fd420107b60 2024-03-04T15:43:05.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.455+0000 7fd416ffd700 1 -- 172.21.15.117:0/871208415 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 14) v1 ==== 91341+0+0 (secure 0 0 0) 0x7fd41801c6c0 con 0x7fd420107b60 2024-03-04T15:43:05.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.459+0000 7fd416ffd700 1 --2- 172.21.15.117:0/871208415 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fd40c06d600 0x7fd40c06fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:43:05.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.459+0000 7fd416ffd700 1 -- 172.21.15.117:0/871208415 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1767+0+0 (secure 0 0 0) 0x7fd41800f030 con 0x7fd420107b60 2024-03-04T15:43:05.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.459+0000 7fd4258ac700 1 --2- 172.21.15.117:0/871208415 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fd40c06d600 0x7fd40c06fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:43:05.463 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.459+0000 7fd4258ac700 1 --2- 172.21.15.117:0/871208415 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fd40c06d600 0x7fd40c06fac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd41c005950 tx=0x7fd41c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:43:05.466 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.463+0000 7fd416ffd700 1 -- 172.21.15.117:0/871208415 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd41802c060 con 0x7fd420107b60 2024-03-04T15:43:05.671 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:05.667+0000 7fd427b10700 1 -- 172.21.15.117:0/871208415 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd42002da60 con 0x7fd40c06d600 2024-03-04T15:43:06.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:05 smithi117 bash[21480]: cluster 2024-03-04T15:43:04.734222+0000 mgr.y (mgr.14152) 116 : cluster 0 pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:06.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:05 smithi117 bash[21480]: audit 2024-03-04T15:43:05.673687+0000 mgr.y (mgr.14152) 117 : audit 0 from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:43:06.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:05 smithi117 bash[21480]: audit 2024-03-04T15:43:05.680443+0000 mon.a (mon.0) 305 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:43:06.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:05 smithi117 bash[21480]: audit 2024-03-04T15:43:05.683331+0000 mon.a (mon.0) 306 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:43:06.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:05 smithi117 bash[21480]: audit 2024-03-04T15:43:05.684508+0000 mon.a (mon.0) 307 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:43:06.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:05 smithi117 bash[28843]: cluster 2024-03-04T15:43:04.734222+0000 mgr.y (mgr.14152) 116 : cluster 0 pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:06.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:05 smithi117 bash[28843]: audit 2024-03-04T15:43:05.673687+0000 mgr.y (mgr.14152) 117 : audit 0 from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:43:06.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:05 smithi117 bash[28843]: audit 2024-03-04T15:43:05.680443+0000 mon.a (mon.0) 305 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:43:06.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:05 smithi117 bash[28843]: audit 2024-03-04T15:43:05.683331+0000 mon.a (mon.0) 306 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:43:06.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:05 smithi117 bash[28843]: audit 2024-03-04T15:43:05.684508+0000 mon.a (mon.0) 307 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:43:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:05 smithi151 bash[23756]: cluster 2024-03-04T15:43:04.734222+0000 mgr.y (mgr.14152) 116 : cluster 0 pgmap v80: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:05 smithi151 bash[23756]: audit 2024-03-04T15:43:05.673687+0000 mgr.y (mgr.14152) 117 : audit 0 from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:43:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:05 smithi151 bash[23756]: audit 2024-03-04T15:43:05.680443+0000 mon.a (mon.0) 305 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:43:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:05 smithi151 bash[23756]: audit 2024-03-04T15:43:05.683331+0000 mon.a (mon.0) 306 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:43:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:05 smithi151 bash[23756]: audit 2024-03-04T15:43:05.684508+0000 mon.a (mon.0) 307 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:43:08.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:07 smithi117 bash[21480]: cluster 2024-03-04T15:43:06.734699+0000 mgr.y (mgr.14152) 118 : cluster 0 pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:08.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:07 smithi117 bash[28843]: cluster 2024-03-04T15:43:06.734699+0000 mgr.y (mgr.14152) 118 : cluster 0 pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:08.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:07 smithi151 bash[23756]: cluster 2024-03-04T15:43:06.734699+0000 mgr.y (mgr.14152) 118 : cluster 0 pgmap v81: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:10.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:09 smithi117 bash[21480]: cluster 2024-03-04T15:43:08.735148+0000 mgr.y (mgr.14152) 119 : cluster 0 pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:10.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:09 smithi117 bash[28843]: cluster 2024-03-04T15:43:08.735148+0000 mgr.y (mgr.14152) 119 : cluster 0 pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:09 smithi151 bash[23756]: cluster 2024-03-04T15:43:08.735148+0000 mgr.y (mgr.14152) 119 : cluster 0 pgmap v82: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:12.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:11 smithi117 bash[21480]: cluster 2024-03-04T15:43:10.735654+0000 mgr.y (mgr.14152) 120 : cluster 0 pgmap v83: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:12.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:11 smithi117 bash[28843]: cluster 2024-03-04T15:43:10.735654+0000 mgr.y (mgr.14152) 120 : cluster 0 pgmap v83: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:11 smithi151 bash[23756]: cluster 2024-03-04T15:43:10.735654+0000 mgr.y (mgr.14152) 120 : cluster 0 pgmap v83: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:14.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:13 smithi117 bash[21480]: cluster 2024-03-04T15:43:12.736219+0000 mgr.y (mgr.14152) 121 : cluster 0 pgmap v84: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:14.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:13 smithi117 bash[28843]: cluster 2024-03-04T15:43:12.736219+0000 mgr.y (mgr.14152) 121 : cluster 0 pgmap v84: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:14.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:13 smithi151 bash[23756]: cluster 2024-03-04T15:43:12.736219+0000 mgr.y (mgr.14152) 121 : cluster 0 pgmap v84: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:15.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:14 smithi117 bash[21480]: audit 2024-03-04T15:43:13.808615+0000 mon.a (mon.0) 308 : audit 1 from='client.? 172.21.15.117:0/3379084685' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e7ba793-aa0e-4435-bfd6-1bf6d28da6ff"}]: dispatch 2024-03-04T15:43:15.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:14 smithi117 bash[21480]: audit 2024-03-04T15:43:13.810862+0000 mon.a (mon.0) 309 : audit 1 from='client.? 172.21.15.117:0/3379084685' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e7ba793-aa0e-4435-bfd6-1bf6d28da6ff"}]': finished 2024-03-04T15:43:15.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:14 smithi117 bash[21480]: cluster 2024-03-04T15:43:13.815581+0000 mon.a (mon.0) 310 : cluster 0 osdmap e10: 2 total, 1 up, 2 in 2024-03-04T15:43:15.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:14 smithi117 bash[21480]: audit 2024-03-04T15:43:13.815936+0000 mon.a (mon.0) 311 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T15:43:15.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:14 smithi117 bash[21480]: audit 2024-03-04T15:43:14.566098+0000 mon.a (mon.0) 312 : audit 0 from='client.? 172.21.15.117:0/1159651805' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:43:15.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:14 smithi117 bash[28843]: audit 2024-03-04T15:43:13.808615+0000 mon.a (mon.0) 308 : audit 1 from='client.? 172.21.15.117:0/3379084685' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e7ba793-aa0e-4435-bfd6-1bf6d28da6ff"}]: dispatch 2024-03-04T15:43:15.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:14 smithi117 bash[28843]: audit 2024-03-04T15:43:13.810862+0000 mon.a (mon.0) 309 : audit 1 from='client.? 172.21.15.117:0/3379084685' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e7ba793-aa0e-4435-bfd6-1bf6d28da6ff"}]': finished 2024-03-04T15:43:15.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:14 smithi117 bash[28843]: cluster 2024-03-04T15:43:13.815581+0000 mon.a (mon.0) 310 : cluster 0 osdmap e10: 2 total, 1 up, 2 in 2024-03-04T15:43:15.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:14 smithi117 bash[28843]: audit 2024-03-04T15:43:13.815936+0000 mon.a (mon.0) 311 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T15:43:15.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:14 smithi117 bash[28843]: audit 2024-03-04T15:43:14.566098+0000 mon.a (mon.0) 312 : audit 0 from='client.? 172.21.15.117:0/1159651805' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:43:15.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:14 smithi151 bash[23756]: audit 2024-03-04T15:43:13.808615+0000 mon.a (mon.0) 308 : audit 1 from='client.? 172.21.15.117:0/3379084685' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9e7ba793-aa0e-4435-bfd6-1bf6d28da6ff"}]: dispatch 2024-03-04T15:43:15.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:14 smithi151 bash[23756]: audit 2024-03-04T15:43:13.810862+0000 mon.a (mon.0) 309 : audit 1 from='client.? 172.21.15.117:0/3379084685' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9e7ba793-aa0e-4435-bfd6-1bf6d28da6ff"}]': finished 2024-03-04T15:43:15.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:14 smithi151 bash[23756]: cluster 2024-03-04T15:43:13.815581+0000 mon.a (mon.0) 310 : cluster 0 osdmap e10: 2 total, 1 up, 2 in 2024-03-04T15:43:15.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:14 smithi151 bash[23756]: audit 2024-03-04T15:43:13.815936+0000 mon.a (mon.0) 311 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T15:43:15.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:14 smithi151 bash[23756]: audit 2024-03-04T15:43:14.566098+0000 mon.a (mon.0) 312 : audit 0 from='client.? 172.21.15.117:0/1159651805' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:43:16.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:15 smithi117 bash[21480]: cluster 2024-03-04T15:43:14.736715+0000 mgr.y (mgr.14152) 122 : cluster 0 pgmap v86: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:16.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:15 smithi117 bash[28843]: cluster 2024-03-04T15:43:14.736715+0000 mgr.y (mgr.14152) 122 : cluster 0 pgmap v86: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:16.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:15 smithi151 bash[23756]: cluster 2024-03-04T15:43:14.736715+0000 mgr.y (mgr.14152) 122 : cluster 0 pgmap v86: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:18.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:17 smithi117 bash[21480]: cluster 2024-03-04T15:43:16.737304+0000 mgr.y (mgr.14152) 123 : cluster 0 pgmap v87: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:18.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:17 smithi117 bash[28843]: cluster 2024-03-04T15:43:16.737304+0000 mgr.y (mgr.14152) 123 : cluster 0 pgmap v87: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:18.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:17 smithi151 bash[23756]: cluster 2024-03-04T15:43:16.737304+0000 mgr.y (mgr.14152) 123 : cluster 0 pgmap v87: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:20.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:19 smithi151 bash[23756]: cluster 2024-03-04T15:43:18.737755+0000 mgr.y (mgr.14152) 124 : cluster 0 pgmap v88: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:20.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:19 smithi117 bash[21480]: cluster 2024-03-04T15:43:18.737755+0000 mgr.y (mgr.14152) 124 : cluster 0 pgmap v88: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:20.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:19 smithi117 bash[28843]: cluster 2024-03-04T15:43:18.737755+0000 mgr.y (mgr.14152) 124 : cluster 0 pgmap v88: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:22.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:21 smithi151 bash[23756]: cluster 2024-03-04T15:43:20.738139+0000 mgr.y (mgr.14152) 125 : cluster 0 pgmap v89: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:22.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:21 smithi117 bash[21480]: cluster 2024-03-04T15:43:20.738139+0000 mgr.y (mgr.14152) 125 : cluster 0 pgmap v89: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:22.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:21 smithi117 bash[28843]: cluster 2024-03-04T15:43:20.738139+0000 mgr.y (mgr.14152) 125 : cluster 0 pgmap v89: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:24.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:23 smithi117 bash[28843]: cluster 2024-03-04T15:43:22.738587+0000 mgr.y (mgr.14152) 126 : cluster 0 pgmap v90: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:24.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:23 smithi117 bash[21480]: cluster 2024-03-04T15:43:22.738587+0000 mgr.y (mgr.14152) 126 : cluster 0 pgmap v90: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:23 smithi151 bash[23756]: cluster 2024-03-04T15:43:22.738587+0000 mgr.y (mgr.14152) 126 : cluster 0 pgmap v90: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:25 smithi151 bash[23756]: cluster 2024-03-04T15:43:24.739040+0000 mgr.y (mgr.14152) 127 : cluster 0 pgmap v91: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:26.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:25 smithi117 bash[21480]: cluster 2024-03-04T15:43:24.739040+0000 mgr.y (mgr.14152) 127 : cluster 0 pgmap v91: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:26.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:25 smithi117 bash[28843]: cluster 2024-03-04T15:43:24.739040+0000 mgr.y (mgr.14152) 127 : cluster 0 pgmap v91: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:26 smithi151 bash[23756]: audit 2024-03-04T15:43:26.063421+0000 mon.a (mon.0) 313 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-04T15:43:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:26 smithi151 bash[23756]: audit 2024-03-04T15:43:26.065113+0000 mon.a (mon.0) 314 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:43:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:26 smithi151 bash[23756]: cephadm 2024-03-04T15:43:26.066524+0000 mgr.y (mgr.14152) 128 : cephadm 1 Deploying daemon osd.1 on smithi117 2024-03-04T15:43:27.211 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:26 smithi117 bash[28843]: audit 2024-03-04T15:43:26.063421+0000 mon.a (mon.0) 313 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-04T15:43:27.211 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:26 smithi117 bash[28843]: audit 2024-03-04T15:43:26.065113+0000 mon.a (mon.0) 314 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:43:27.211 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:26 smithi117 bash[28843]: cephadm 2024-03-04T15:43:26.066524+0000 mgr.y (mgr.14152) 128 : cephadm 1 Deploying daemon osd.1 on smithi117 2024-03-04T15:43:27.211 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:26 smithi117 bash[21480]: audit 2024-03-04T15:43:26.063421+0000 mon.a (mon.0) 313 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-03-04T15:43:27.211 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:26 smithi117 bash[21480]: audit 2024-03-04T15:43:26.065113+0000 mon.a (mon.0) 314 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:43:27.211 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:26 smithi117 bash[21480]: cephadm 2024-03-04T15:43:26.066524+0000 mgr.y (mgr.14152) 128 : cephadm 1 Deploying daemon osd.1 on smithi117 2024-03-04T15:43:28.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:27 smithi151 bash[23756]: cluster 2024-03-04T15:43:26.739502+0000 mgr.y (mgr.14152) 129 : cluster 0 pgmap v92: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:28.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:27 smithi117 bash[21480]: cluster 2024-03-04T15:43:26.739502+0000 mgr.y (mgr.14152) 129 : cluster 0 pgmap v92: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:28.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:27 smithi117 bash[28843]: cluster 2024-03-04T15:43:26.739502+0000 mgr.y (mgr.14152) 129 : cluster 0 pgmap v92: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:30.063 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:43:29 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:43:30.064 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:29 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:43:30.064 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:29 smithi117 bash[21480]: cluster 2024-03-04T15:43:28.739785+0000 mgr.y (mgr.14152) 130 : cluster 0 pgmap v93: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:30.064 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:29 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:43:30.064 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:29 smithi117 bash[28843]: cluster 2024-03-04T15:43:28.739785+0000 mgr.y (mgr.14152) 130 : cluster 0 pgmap v93: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:30.064 INFO:journalctl@ceph.osd.0.smithi117.stdout:Mar 04 15:43:29 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:43:30.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:29 smithi151 bash[23756]: cluster 2024-03-04T15:43:28.739785+0000 mgr.y (mgr.14152) 130 : cluster 0 pgmap v93: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:30.321 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:43:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:43:30.321 INFO:journalctl@ceph.osd.0.smithi117.stdout:Mar 04 15:43:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:43:30.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:43:30.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:43:31.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:30 smithi151 bash[23756]: audit 2024-03-04T15:43:30.211218+0000 mon.a (mon.0) 315 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:43:31.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:30 smithi151 bash[23756]: audit 2024-03-04T15:43:30.214579+0000 mon.a (mon.0) 316 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:31.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:30 smithi151 bash[23756]: audit 2024-03-04T15:43:30.221047+0000 mon.a (mon.0) 317 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:31.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:30 smithi117 bash[21480]: audit 2024-03-04T15:43:30.211218+0000 mon.a (mon.0) 315 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:43:31.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:30 smithi117 bash[21480]: audit 2024-03-04T15:43:30.214579+0000 mon.a (mon.0) 316 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:31.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:30 smithi117 bash[21480]: audit 2024-03-04T15:43:30.221047+0000 mon.a (mon.0) 317 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:31.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:30 smithi117 bash[28843]: audit 2024-03-04T15:43:30.211218+0000 mon.a (mon.0) 315 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:43:31.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:30 smithi117 bash[28843]: audit 2024-03-04T15:43:30.214579+0000 mon.a (mon.0) 316 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:31.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:30 smithi117 bash[28843]: audit 2024-03-04T15:43:30.221047+0000 mon.a (mon.0) 317 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:32.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:32 smithi117 bash[21480]: cluster 2024-03-04T15:43:30.740054+0000 mgr.y (mgr.14152) 131 : cluster 0 pgmap v94: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:32.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:32 smithi117 bash[28843]: cluster 2024-03-04T15:43:30.740054+0000 mgr.y (mgr.14152) 131 : cluster 0 pgmap v94: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:32 smithi151 bash[23756]: cluster 2024-03-04T15:43:30.740054+0000 mgr.y (mgr.14152) 131 : cluster 0 pgmap v94: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:34.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:34 smithi117 bash[21480]: cluster 2024-03-04T15:43:32.740521+0000 mgr.y (mgr.14152) 132 : cluster 0 pgmap v95: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:34.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:34 smithi117 bash[28843]: cluster 2024-03-04T15:43:32.740521+0000 mgr.y (mgr.14152) 132 : cluster 0 pgmap v95: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:34.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:34 smithi151 bash[23756]: cluster 2024-03-04T15:43:32.740521+0000 mgr.y (mgr.14152) 132 : cluster 0 pgmap v95: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:36.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:36 smithi117 bash[28843]: cluster 2024-03-04T15:43:34.740984+0000 mgr.y (mgr.14152) 133 : cluster 0 pgmap v96: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:36.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:36 smithi117 bash[28843]: audit 2024-03-04T15:43:35.498715+0000 mon.a (mon.0) 318 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:36.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:36 smithi117 bash[28843]: audit 2024-03-04T15:43:35.503294+0000 mon.a (mon.0) 319 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:36.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:36 smithi117 bash[28843]: audit 2024-03-04T15:43:35.504540+0000 mon.a (mon.0) 320 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:43:36.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:36 smithi117 bash[28843]: audit 2024-03-04T15:43:35.505477+0000 mon.a (mon.0) 321 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:43:36.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:36 smithi117 bash[28843]: audit 2024-03-04T15:43:35.510116+0000 mon.a (mon.0) 322 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:36 smithi117 bash[21480]: cluster 2024-03-04T15:43:34.740984+0000 mgr.y (mgr.14152) 133 : cluster 0 pgmap v96: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:36 smithi117 bash[21480]: audit 2024-03-04T15:43:35.498715+0000 mon.a (mon.0) 318 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:36 smithi117 bash[21480]: audit 2024-03-04T15:43:35.503294+0000 mon.a (mon.0) 319 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:36 smithi117 bash[21480]: audit 2024-03-04T15:43:35.504540+0000 mon.a (mon.0) 320 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:43:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:36 smithi117 bash[21480]: audit 2024-03-04T15:43:35.505477+0000 mon.a (mon.0) 321 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:43:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:36 smithi117 bash[21480]: audit 2024-03-04T15:43:35.510116+0000 mon.a (mon.0) 322 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:36.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:36 smithi151 bash[23756]: cluster 2024-03-04T15:43:34.740984+0000 mgr.y (mgr.14152) 133 : cluster 0 pgmap v96: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:36.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:36 smithi151 bash[23756]: audit 2024-03-04T15:43:35.498715+0000 mon.a (mon.0) 318 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:36.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:36 smithi151 bash[23756]: audit 2024-03-04T15:43:35.503294+0000 mon.a (mon.0) 319 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:36.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:36 smithi151 bash[23756]: audit 2024-03-04T15:43:35.504540+0000 mon.a (mon.0) 320 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:43:36.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:36 smithi151 bash[23756]: audit 2024-03-04T15:43:35.505477+0000 mon.a (mon.0) 321 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:43:36.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:36 smithi151 bash[23756]: audit 2024-03-04T15:43:35.510116+0000 mon.a (mon.0) 322 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:37.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:37 smithi117 bash[21480]: audit 2024-03-04T15:43:36.657898+0000 mon.c (mon.2) 4 : audit 1 from='osd.1 [v2:172.21.15.117:6810/444026972,v1:172.21.15.117:6811/444026972]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T15:43:37.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:37 smithi117 bash[21480]: audit 2024-03-04T15:43:36.658305+0000 mon.a (mon.0) 323 : audit 1 from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T15:43:37.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:37 smithi117 bash[28843]: audit 2024-03-04T15:43:36.657898+0000 mon.c (mon.2) 4 : audit 1 from='osd.1 [v2:172.21.15.117:6810/444026972,v1:172.21.15.117:6811/444026972]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T15:43:37.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:37 smithi117 bash[28843]: audit 2024-03-04T15:43:36.658305+0000 mon.a (mon.0) 323 : audit 1 from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T15:43:37.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:37 smithi151 bash[23756]: audit 2024-03-04T15:43:36.657898+0000 mon.c (mon.2) 4 : audit 1 from='osd.1 [v2:172.21.15.117:6810/444026972,v1:172.21.15.117:6811/444026972]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T15:43:37.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:37 smithi151 bash[23756]: audit 2024-03-04T15:43:36.658305+0000 mon.a (mon.0) 323 : audit 1 from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-03-04T15:43:38.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:38.726+0000 7fd416ffd700 1 -- 172.21.15.117:0/871208415 <== mgr.14152 v2:172.21.15.117:6800/2865911351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd42002da60 con 0x7fd40c06d600 2024-03-04T15:43:38.731 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 1 on host 'smithi117' 2024-03-04T15:43:38.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:38.730+0000 7fd427b10700 1 -- 172.21.15.117:0/871208415 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fd40c06d600 msgr2=0x7fd40c06fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:43:38.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:38.730+0000 7fd427b10700 1 --2- 172.21.15.117:0/871208415 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fd40c06d600 0x7fd40c06fac0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd41c005950 tx=0x7fd41c0058e0 comp rx=0 tx=0).stop 2024-03-04T15:43:38.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:38.730+0000 7fd427b10700 1 -- 172.21.15.117:0/871208415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd420107b60 msgr2=0x7fd42019eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:43:38.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:38.730+0000 7fd427b10700 1 --2- 172.21.15.117:0/871208415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd420107b60 0x7fd42019eae0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fd418009f90 tx=0x7fd418009b90 comp rx=0 tx=0).stop 2024-03-04T15:43:38.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:38.730+0000 7fd427b10700 1 -- 172.21.15.117:0/871208415 shutdown_connections 2024-03-04T15:43:38.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:38.730+0000 7fd427b10700 1 --2- 172.21.15.117:0/871208415 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fd40c06d600 0x7fd40c06fac0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:43:38.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:38.730+0000 7fd427b10700 1 --2- 172.21.15.117:0/871208415 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd420107b60 0x7fd42019eae0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:43:38.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:38.730+0000 7fd427b10700 1 --2- 172.21.15.117:0/871208415 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd420102900 0x7fd420197570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:43:38.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:38.730+0000 7fd427b10700 1 --2- 172.21.15.117:0/871208415 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd420105230 0x7fd420197ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:43:38.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:38.730+0000 7fd427b10700 1 -- 172.21.15.117:0/871208415 >> 172.21.15.117:0/871208415 conn(0x7fd4200fc310 msgr2=0x7fd420103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:43:38.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:38.730+0000 7fd427b10700 1 -- 172.21.15.117:0/871208415 shutdown_connections 2024-03-04T15:43:38.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:38.730+0000 7fd427b10700 1 -- 172.21.15.117:0/871208415 wait complete. 2024-03-04T15:43:38.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:38 smithi117 bash[21480]: cluster 2024-03-04T15:43:36.741321+0000 mgr.y (mgr.14152) 134 : cluster 0 pgmap v97: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:38.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:38 smithi117 bash[21480]: audit 2024-03-04T15:43:37.505382+0000 mon.a (mon.0) 324 : audit 1 from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-04T15:43:38.768 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:38 smithi117 bash[21480]: cluster 2024-03-04T15:43:37.508610+0000 mon.a (mon.0) 325 : cluster 0 osdmap e11: 2 total, 1 up, 2 in 2024-03-04T15:43:38.769 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:38 smithi117 bash[21480]: audit 2024-03-04T15:43:37.508617+0000 mon.c (mon.2) 5 : audit 1 from='osd.1 [v2:172.21.15.117:6810/444026972,v1:172.21.15.117:6811/444026972]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-04T15:43:38.769 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:38 smithi117 bash[21480]: audit 2024-03-04T15:43:37.508848+0000 mon.a (mon.0) 326 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T15:43:38.769 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:38 smithi117 bash[21480]: audit 2024-03-04T15:43:37.509280+0000 mon.a (mon.0) 327 : audit 1 from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-04T15:43:38.769 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:38 smithi117 bash[28843]: cluster 2024-03-04T15:43:36.741321+0000 mgr.y (mgr.14152) 134 : cluster 0 pgmap v97: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:38.769 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:38 smithi117 bash[28843]: audit 2024-03-04T15:43:37.505382+0000 mon.a (mon.0) 324 : audit 1 from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-04T15:43:38.769 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:38 smithi117 bash[28843]: cluster 2024-03-04T15:43:37.508610+0000 mon.a (mon.0) 325 : cluster 0 osdmap e11: 2 total, 1 up, 2 in 2024-03-04T15:43:38.769 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:38 smithi117 bash[28843]: audit 2024-03-04T15:43:37.508617+0000 mon.c (mon.2) 5 : audit 1 from='osd.1 [v2:172.21.15.117:6810/444026972,v1:172.21.15.117:6811/444026972]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-04T15:43:38.769 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:38 smithi117 bash[28843]: audit 2024-03-04T15:43:37.508848+0000 mon.a (mon.0) 326 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T15:43:38.769 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:38 smithi117 bash[28843]: audit 2024-03-04T15:43:37.509280+0000 mon.a (mon.0) 327 : audit 1 from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-04T15:43:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:38 smithi151 bash[23756]: cluster 2024-03-04T15:43:36.741321+0000 mgr.y (mgr.14152) 134 : cluster 0 pgmap v97: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:38 smithi151 bash[23756]: audit 2024-03-04T15:43:37.505382+0000 mon.a (mon.0) 324 : audit 1 from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-03-04T15:43:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:38 smithi151 bash[23756]: cluster 2024-03-04T15:43:37.508610+0000 mon.a (mon.0) 325 : cluster 0 osdmap e11: 2 total, 1 up, 2 in 2024-03-04T15:43:38.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:38 smithi151 bash[23756]: audit 2024-03-04T15:43:37.508617+0000 mon.c (mon.2) 5 : audit 1 from='osd.1 [v2:172.21.15.117:6810/444026972,v1:172.21.15.117:6811/444026972]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-04T15:43:38.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:38 smithi151 bash[23756]: audit 2024-03-04T15:43:37.508848+0000 mon.a (mon.0) 326 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T15:43:38.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:38 smithi151 bash[23756]: audit 2024-03-04T15:43:37.509280+0000 mon.a (mon.0) 327 : audit 1 from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-04T15:43:39.343 DEBUG:teuthology.orchestra.run.smithi117:osd.1> sudo journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.1.service 2024-03-04T15:43:39.346 INFO:tasks.cephadm:Deploying osd.2 on smithi117 with /dev/vg_nvme/lv_2... 2024-03-04T15:43:39.346 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-04T15:43:39.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:39 smithi117 bash[21480]: audit 2024-03-04T15:43:38.507632+0000 mon.a (mon.0) 328 : audit 1 from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-03-04T15:43:39.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:39 smithi117 bash[21480]: cluster 2024-03-04T15:43:38.513283+0000 mon.a (mon.0) 329 : cluster 0 osdmap e12: 2 total, 1 up, 2 in 2024-03-04T15:43:39.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:39 smithi117 bash[21480]: audit 2024-03-04T15:43:38.514572+0000 mon.a (mon.0) 330 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T15:43:39.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:39 smithi117 bash[21480]: audit 2024-03-04T15:43:38.524649+0000 mon.a (mon.0) 331 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T15:43:39.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:39 smithi117 bash[21480]: audit 2024-03-04T15:43:38.671994+0000 mon.a (mon.0) 332 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:39.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:39 smithi117 bash[21480]: audit 2024-03-04T15:43:38.680228+0000 mon.a (mon.0) 333 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:39.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:39 smithi117 bash[21480]: audit 2024-03-04T15:43:38.701367+0000 mon.a (mon.0) 334 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:43:39.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:39 smithi117 bash[28843]: audit 2024-03-04T15:43:38.507632+0000 mon.a (mon.0) 328 : audit 1 from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-03-04T15:43:39.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:39 smithi117 bash[28843]: cluster 2024-03-04T15:43:38.513283+0000 mon.a (mon.0) 329 : cluster 0 osdmap e12: 2 total, 1 up, 2 in 2024-03-04T15:43:39.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:39 smithi117 bash[28843]: audit 2024-03-04T15:43:38.514572+0000 mon.a (mon.0) 330 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T15:43:39.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:39 smithi117 bash[28843]: audit 2024-03-04T15:43:38.524649+0000 mon.a (mon.0) 331 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T15:43:39.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:39 smithi117 bash[28843]: audit 2024-03-04T15:43:38.671994+0000 mon.a (mon.0) 332 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:39.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:39 smithi117 bash[28843]: audit 2024-03-04T15:43:38.680228+0000 mon.a (mon.0) 333 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:39.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:39 smithi117 bash[28843]: audit 2024-03-04T15:43:38.701367+0000 mon.a (mon.0) 334 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:43:39.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:39 smithi151 bash[23756]: audit 2024-03-04T15:43:38.507632+0000 mon.a (mon.0) 328 : audit 1 from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-03-04T15:43:39.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:39 smithi151 bash[23756]: cluster 2024-03-04T15:43:38.513283+0000 mon.a (mon.0) 329 : cluster 0 osdmap e12: 2 total, 1 up, 2 in 2024-03-04T15:43:39.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:39 smithi151 bash[23756]: audit 2024-03-04T15:43:38.514572+0000 mon.a (mon.0) 330 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T15:43:39.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:39 smithi151 bash[23756]: audit 2024-03-04T15:43:38.524649+0000 mon.a (mon.0) 331 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T15:43:39.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:39 smithi151 bash[23756]: audit 2024-03-04T15:43:38.671994+0000 mon.a (mon.0) 332 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:39.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:39 smithi151 bash[23756]: audit 2024-03-04T15:43:38.680228+0000 mon.a (mon.0) 333 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:39.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:39 smithi151 bash[23756]: audit 2024-03-04T15:43:38.701367+0000 mon.a (mon.0) 334 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:43:40.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:40 smithi117 bash[21480]: cluster 2024-03-04T15:43:37.699460+0000 osd.1 (osd.1) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:43:40.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:40 smithi117 bash[21480]: cluster 2024-03-04T15:43:37.699552+0000 osd.1 (osd.1) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:43:40.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:40 smithi117 bash[21480]: cluster 2024-03-04T15:43:38.741611+0000 mgr.y (mgr.14152) 135 : cluster 0 pgmap v100: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:40.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:40 smithi117 bash[21480]: cluster 2024-03-04T15:43:39.526312+0000 mon.a (mon.0) 335 : cluster 1 osd.1 [v2:172.21.15.117:6810/444026972,v1:172.21.15.117:6811/444026972] boot 2024-03-04T15:43:40.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:40 smithi117 bash[21480]: cluster 2024-03-04T15:43:39.526432+0000 mon.a (mon.0) 336 : cluster 0 osdmap e13: 2 total, 2 up, 2 in 2024-03-04T15:43:40.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:40 smithi117 bash[21480]: audit 2024-03-04T15:43:39.527112+0000 mon.a (mon.0) 337 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T15:43:40.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:40 smithi117 bash[28843]: cluster 2024-03-04T15:43:37.699460+0000 osd.1 (osd.1) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:43:40.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:40 smithi117 bash[28843]: cluster 2024-03-04T15:43:37.699552+0000 osd.1 (osd.1) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:43:40.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:40 smithi117 bash[28843]: cluster 2024-03-04T15:43:38.741611+0000 mgr.y (mgr.14152) 135 : cluster 0 pgmap v100: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:40.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:40 smithi117 bash[28843]: cluster 2024-03-04T15:43:39.526312+0000 mon.a (mon.0) 335 : cluster 1 osd.1 [v2:172.21.15.117:6810/444026972,v1:172.21.15.117:6811/444026972] boot 2024-03-04T15:43:40.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:40 smithi117 bash[28843]: cluster 2024-03-04T15:43:39.526432+0000 mon.a (mon.0) 336 : cluster 0 osdmap e13: 2 total, 2 up, 2 in 2024-03-04T15:43:40.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:40 smithi117 bash[28843]: audit 2024-03-04T15:43:39.527112+0000 mon.a (mon.0) 337 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T15:43:40.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:40 smithi151 bash[23756]: cluster 2024-03-04T15:43:37.699460+0000 osd.1 (osd.1) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:43:40.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:40 smithi151 bash[23756]: cluster 2024-03-04T15:43:37.699552+0000 osd.1 (osd.1) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:43:40.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:40 smithi151 bash[23756]: cluster 2024-03-04T15:43:38.741611+0000 mgr.y (mgr.14152) 135 : cluster 0 pgmap v100: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-03-04T15:43:40.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:40 smithi151 bash[23756]: cluster 2024-03-04T15:43:39.526312+0000 mon.a (mon.0) 335 : cluster 1 osd.1 [v2:172.21.15.117:6810/444026972,v1:172.21.15.117:6811/444026972] boot 2024-03-04T15:43:40.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:40 smithi151 bash[23756]: cluster 2024-03-04T15:43:39.526432+0000 mon.a (mon.0) 336 : cluster 0 osdmap e13: 2 total, 2 up, 2 in 2024-03-04T15:43:40.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:40 smithi151 bash[23756]: audit 2024-03-04T15:43:39.527112+0000 mon.a (mon.0) 337 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T15:43:41.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:41 smithi117 bash[21480]: cluster 2024-03-04T15:43:40.538490+0000 mon.a (mon.0) 338 : cluster 0 osdmap e14: 2 total, 2 up, 2 in 2024-03-04T15:43:41.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:41 smithi117 bash[28843]: cluster 2024-03-04T15:43:40.538490+0000 mon.a (mon.0) 338 : cluster 0 osdmap e14: 2 total, 2 up, 2 in 2024-03-04T15:43:41.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:41 smithi151 bash[23756]: cluster 2024-03-04T15:43:40.538490+0000 mon.a (mon.0) 338 : cluster 0 osdmap e14: 2 total, 2 up, 2 in 2024-03-04T15:43:42.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:42 smithi117 bash[28843]: cluster 2024-03-04T15:43:40.742010+0000 mgr.y (mgr.14152) 136 : cluster 0 pgmap v103: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:42.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:42 smithi117 bash[21480]: cluster 2024-03-04T15:43:40.742010+0000 mgr.y (mgr.14152) 136 : cluster 0 pgmap v103: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:42.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:42 smithi151 bash[23756]: cluster 2024-03-04T15:43:40.742010+0000 mgr.y (mgr.14152) 136 : cluster 0 pgmap v103: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:43.678 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:43:44.813 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:44 smithi117 bash[28843]: cluster 2024-03-04T15:43:42.742489+0000 mgr.y (mgr.14152) 137 : cluster 0 pgmap v104: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:44.814 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:44 smithi117 bash[21480]: cluster 2024-03-04T15:43:42.742489+0000 mgr.y (mgr.14152) 137 : cluster 0 pgmap v104: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:44.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:44 smithi151 bash[23756]: cluster 2024-03-04T15:43:42.742489+0000 mgr.y (mgr.14152) 137 : cluster 0 pgmap v104: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:46.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:46 smithi117 bash[21480]: cluster 2024-03-04T15:43:44.742948+0000 mgr.y (mgr.14152) 138 : cluster 0 pgmap v105: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:46.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:46 smithi117 bash[28843]: cluster 2024-03-04T15:43:44.742948+0000 mgr.y (mgr.14152) 138 : cluster 0 pgmap v105: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:46.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:46 smithi151 bash[23756]: cluster 2024-03-04T15:43:44.742948+0000 mgr.y (mgr.14152) 138 : cluster 0 pgmap v105: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:47.655 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-04T15:43:47.655 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-04T15:43:47.656 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-03-04T15:43:47.656 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-03-04T15:43:47.656 INFO:teuthology.orchestra.run.smithi117.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0150885 s, 695 MB/s 2024-03-04T15:43:47.656 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-03-04T15:43:48.690 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi117:vg_nvme/lv_2 2024-03-04T15:43:48.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:48 smithi117 bash[21480]: cluster 2024-03-04T15:43:46.743448+0000 mgr.y (mgr.14152) 139 : cluster 0 pgmap v106: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:48.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:48 smithi117 bash[28843]: cluster 2024-03-04T15:43:46.743448+0000 mgr.y (mgr.14152) 139 : cluster 0 pgmap v106: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:48.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:48 smithi151 bash[23756]: cluster 2024-03-04T15:43:46.743448+0000 mgr.y (mgr.14152) 139 : cluster 0 pgmap v106: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:50.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[21480]: cluster 2024-03-04T15:43:48.743908+0000 mgr.y (mgr.14152) 140 : cluster 0 pgmap v107: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:50.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[21480]: cephadm 2024-03-04T15:43:48.792890+0000 mgr.y (mgr.14152) 141 : cephadm 1 Detected new or changed devices on smithi117 2024-03-04T15:43:50.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[21480]: audit 2024-03-04T15:43:48.800356+0000 mon.a (mon.0) 339 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:50.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[21480]: audit 2024-03-04T15:43:48.808201+0000 mon.a (mon.0) 340 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:50.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[21480]: audit 2024-03-04T15:43:48.810428+0000 mon.a (mon.0) 341 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:43:50.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[21480]: audit 2024-03-04T15:43:48.812044+0000 mon.a (mon.0) 342 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:43:50.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[21480]: cephadm 2024-03-04T15:43:48.813175+0000 mgr.y (mgr.14152) 142 : cephadm 1 Adjusting osd_memory_target on smithi117 to 8120M 2024-03-04T15:43:50.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[21480]: audit 2024-03-04T15:43:48.818064+0000 mon.a (mon.0) 343 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:50.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[21480]: audit 2024-03-04T15:43:48.821300+0000 mon.a (mon.0) 344 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:43:50.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[21480]: audit 2024-03-04T15:43:48.823184+0000 mon.a (mon.0) 345 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:43:50.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[21480]: audit 2024-03-04T15:43:48.831648+0000 mon.a (mon.0) 346 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:50.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[28843]: cluster 2024-03-04T15:43:48.743908+0000 mgr.y (mgr.14152) 140 : cluster 0 pgmap v107: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:50.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[28843]: cephadm 2024-03-04T15:43:48.792890+0000 mgr.y (mgr.14152) 141 : cephadm 1 Detected new or changed devices on smithi117 2024-03-04T15:43:50.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[28843]: audit 2024-03-04T15:43:48.800356+0000 mon.a (mon.0) 339 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:50.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[28843]: audit 2024-03-04T15:43:48.808201+0000 mon.a (mon.0) 340 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:50.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[28843]: audit 2024-03-04T15:43:48.810428+0000 mon.a (mon.0) 341 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:43:50.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[28843]: audit 2024-03-04T15:43:48.812044+0000 mon.a (mon.0) 342 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:43:50.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[28843]: cephadm 2024-03-04T15:43:48.813175+0000 mgr.y (mgr.14152) 142 : cephadm 1 Adjusting osd_memory_target on smithi117 to 8120M 2024-03-04T15:43:50.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[28843]: audit 2024-03-04T15:43:48.818064+0000 mon.a (mon.0) 343 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:50.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[28843]: audit 2024-03-04T15:43:48.821300+0000 mon.a (mon.0) 344 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:43:50.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[28843]: audit 2024-03-04T15:43:48.823184+0000 mon.a (mon.0) 345 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:43:50.074 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:49 smithi117 bash[28843]: audit 2024-03-04T15:43:48.831648+0000 mon.a (mon.0) 346 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:50.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:49 smithi151 bash[23756]: cluster 2024-03-04T15:43:48.743908+0000 mgr.y (mgr.14152) 140 : cluster 0 pgmap v107: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:50.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:49 smithi151 bash[23756]: cephadm 2024-03-04T15:43:48.792890+0000 mgr.y (mgr.14152) 141 : cephadm 1 Detected new or changed devices on smithi117 2024-03-04T15:43:50.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:49 smithi151 bash[23756]: audit 2024-03-04T15:43:48.800356+0000 mon.a (mon.0) 339 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:50.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:49 smithi151 bash[23756]: audit 2024-03-04T15:43:48.808201+0000 mon.a (mon.0) 340 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:50.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:49 smithi151 bash[23756]: audit 2024-03-04T15:43:48.810428+0000 mon.a (mon.0) 341 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:43:50.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:49 smithi151 bash[23756]: audit 2024-03-04T15:43:48.812044+0000 mon.a (mon.0) 342 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:43:50.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:49 smithi151 bash[23756]: cephadm 2024-03-04T15:43:48.813175+0000 mgr.y (mgr.14152) 142 : cephadm 1 Adjusting osd_memory_target on smithi117 to 8120M 2024-03-04T15:43:50.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:49 smithi151 bash[23756]: audit 2024-03-04T15:43:48.818064+0000 mon.a (mon.0) 343 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:50.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:49 smithi151 bash[23756]: audit 2024-03-04T15:43:48.821300+0000 mon.a (mon.0) 344 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:43:50.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:49 smithi151 bash[23756]: audit 2024-03-04T15:43:48.823184+0000 mon.a (mon.0) 345 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:43:50.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:49 smithi151 bash[23756]: audit 2024-03-04T15:43:48.831648+0000 mon.a (mon.0) 346 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:43:52.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:51 smithi117 bash[21480]: cluster 2024-03-04T15:43:50.744388+0000 mgr.y (mgr.14152) 143 : cluster 0 pgmap v108: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:52.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:51 smithi117 bash[28843]: cluster 2024-03-04T15:43:50.744388+0000 mgr.y (mgr.14152) 143 : cluster 0 pgmap v108: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:52.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:51 smithi151 bash[23756]: cluster 2024-03-04T15:43:50.744388+0000 mgr.y (mgr.14152) 143 : cluster 0 pgmap v108: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:53.432 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:43:54.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:53 smithi117 bash[28843]: cluster 2024-03-04T15:43:52.744824+0000 mgr.y (mgr.14152) 144 : cluster 0 pgmap v109: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:54.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:53 smithi117 bash[21480]: cluster 2024-03-04T15:43:52.744824+0000 mgr.y (mgr.14152) 144 : cluster 0 pgmap v109: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:53 smithi151 bash[23756]: cluster 2024-03-04T15:43:52.744824+0000 mgr.y (mgr.14152) 144 : cluster 0 pgmap v109: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:55.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.934+0000 7ff69e2a1700 1 -- 172.21.15.117:0/3782953975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff698105290 msgr2=0x7ff6981056f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:43:55.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.934+0000 7ff69e2a1700 1 --2- 172.21.15.117:0/3782953975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff698105290 0x7ff6981056f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7ff680009990 tx=0x7ff680009ca0 comp rx=0 tx=0).stop 2024-03-04T15:43:55.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.934+0000 7ff69e2a1700 1 -- 172.21.15.117:0/3782953975 shutdown_connections 2024-03-04T15:43:55.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.934+0000 7ff69e2a1700 1 --2- 172.21.15.117:0/3782953975 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff698105c30 0x7ff69810c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:43:55.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.934+0000 7ff69e2a1700 1 --2- 172.21.15.117:0/3782953975 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff698104090 0x7ff6981044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:43:55.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.934+0000 7ff69e2a1700 1 --2- 172.21.15.117:0/3782953975 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff698105290 0x7ff6981056f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:43:55.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.934+0000 7ff69e2a1700 1 -- 172.21.15.117:0/3782953975 >> 172.21.15.117:0/3782953975 conn(0x7ff6980ff670 msgr2=0x7ff698101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:43:55.943 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.934+0000 7ff69e2a1700 1 -- 172.21.15.117:0/3782953975 shutdown_connections 2024-03-04T15:43:55.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.934+0000 7ff69e2a1700 1 -- 172.21.15.117:0/3782953975 wait complete. 2024-03-04T15:43:55.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.934+0000 7ff69e2a1700 1 Processor -- start 2024-03-04T15:43:55.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.934+0000 7ff69e2a1700 1 -- start start 2024-03-04T15:43:55.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.934+0000 7ff69e2a1700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff698104090 0x7ff6981997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:43:55.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.934+0000 7ff69e2a1700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff698105290 0x7ff698199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:43:55.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.934+0000 7ff69e2a1700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff698105c30 0x7ff6981a0d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:43:55.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.934+0000 7ff69e2a1700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69810f1c0 con 0x7ff698105290 2024-03-04T15:43:55.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.934+0000 7ff69e2a1700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff69810f040 con 0x7ff698104090 2024-03-04T15:43:55.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.934+0000 7ff69e2a1700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff69810f340 con 0x7ff698105c30 2024-03-04T15:43:55.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff68ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff698105290 0x7ff698199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:43:55.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff69c83e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff698105c30 0x7ff6981a0d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:43:55.944 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff68ffff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff698105290 0x7ff698199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50498/0 (socket says 172.21.15.117:50498) 2024-03-04T15:43:55.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff68ffff700 1 -- 172.21.15.117:0/1803068970 learned_addr learned my addr 172.21.15.117:0/1803068970 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:43:55.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff69c83e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff698105c30 0x7ff6981a0d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:60472/0 (socket says 172.21.15.117:60472) 2024-03-04T15:43:55.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff697fff700 1 --2- 172.21.15.117:0/1803068970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff698104090 0x7ff6981997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:43:55.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff68ffff700 1 -- 172.21.15.117:0/1803068970 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff698105c30 msgr2=0x7ff6981a0d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:43:55.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff68ffff700 1 --2- 172.21.15.117:0/1803068970 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff698105c30 0x7ff6981a0d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:43:55.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff68ffff700 1 -- 172.21.15.117:0/1803068970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff698104090 msgr2=0x7ff6981997d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:43:55.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff68ffff700 1 --2- 172.21.15.117:0/1803068970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff698104090 0x7ff6981997d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:43:55.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff68ffff700 1 -- 172.21.15.117:0/1803068970 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6981a1430 con 0x7ff698105290 2024-03-04T15:43:55.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff69c83e700 1 --2- 172.21.15.117:0/1803068970 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff698105c30 0x7ff6981a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:43:55.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff68ffff700 1 --2- 172.21.15.117:0/1803068970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff698105290 0x7ff698199d10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff680000c00 tx=0x7ff68000bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:43:55.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff695ffb700 1 -- 172.21.15.117:0/1803068970 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff680027070 con 0x7ff698105290 2024-03-04T15:43:55.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff695ffb700 1 -- 172.21.15.117:0/1803068970 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff680022470 con 0x7ff698105290 2024-03-04T15:43:55.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff69e2a1700 1 -- 172.21.15.117:0/1803068970 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6981a16c0 con 0x7ff698105290 2024-03-04T15:43:55.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff695ffb700 1 -- 172.21.15.117:0/1803068970 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff68000f6c0 con 0x7ff698105290 2024-03-04T15:43:55.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff69e2a1700 1 -- 172.21.15.117:0/1803068970 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6981a1b10 con 0x7ff698105290 2024-03-04T15:43:55.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff69e2a1700 1 -- 172.21.15.117:0/1803068970 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff69804faa0 con 0x7ff698105290 2024-03-04T15:43:55.946 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff695ffb700 1 -- 172.21.15.117:0/1803068970 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 14) v1 ==== 91341+0+0 (secure 0 0 0) 0x7ff68002a070 con 0x7ff698105290 2024-03-04T15:43:55.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff695ffb700 1 --2- 172.21.15.117:0/1803068970 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7ff68406d600 0x7ff68406fac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:43:55.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff695ffb700 1 -- 172.21.15.117:0/1803068970 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2199+0+0 (secure 0 0 0) 0x7ff68008d490 con 0x7ff698105290 2024-03-04T15:43:55.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff697fff700 1 --2- 172.21.15.117:0/1803068970 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7ff68406d600 0x7ff68406fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:43:55.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.938+0000 7ff697fff700 1 --2- 172.21.15.117:0/1803068970 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7ff68406d600 0x7ff68406fac0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff688005950 tx=0x7ff6880058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:43:55.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:55.946+0000 7ff695ffb700 1 -- 172.21.15.117:0/1803068970 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff680013070 con 0x7ff698105290 2024-03-04T15:43:56.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:55 smithi117 bash[21480]: cluster 2024-03-04T15:43:54.745315+0000 mgr.y (mgr.14152) 145 : cluster 0 pgmap v110: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:56.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:55 smithi117 bash[28843]: cluster 2024-03-04T15:43:54.745315+0000 mgr.y (mgr.14152) 145 : cluster 0 pgmap v110: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:56.163 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:43:56.158+0000 7ff69e2a1700 1 -- 172.21.15.117:0/1803068970 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7ff698062220 con 0x7ff68406d600 2024-03-04T15:43:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:55 smithi151 bash[23756]: cluster 2024-03-04T15:43:54.745315+0000 mgr.y (mgr.14152) 145 : cluster 0 pgmap v110: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:57.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:56 smithi117 bash[21480]: audit 2024-03-04T15:43:56.164523+0000 mgr.y (mgr.14152) 146 : audit 0 from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:43:57.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:56 smithi117 bash[21480]: audit 2024-03-04T15:43:56.171338+0000 mon.a (mon.0) 347 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:43:57.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:56 smithi117 bash[21480]: audit 2024-03-04T15:43:56.176989+0000 mon.a (mon.0) 348 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:43:57.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:56 smithi117 bash[21480]: audit 2024-03-04T15:43:56.178304+0000 mon.a (mon.0) 349 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:43:57.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:56 smithi117 bash[28843]: audit 2024-03-04T15:43:56.164523+0000 mgr.y (mgr.14152) 146 : audit 0 from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:43:57.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:56 smithi117 bash[28843]: audit 2024-03-04T15:43:56.171338+0000 mon.a (mon.0) 347 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:43:57.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:56 smithi117 bash[28843]: audit 2024-03-04T15:43:56.176989+0000 mon.a (mon.0) 348 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:43:57.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:56 smithi117 bash[28843]: audit 2024-03-04T15:43:56.178304+0000 mon.a (mon.0) 349 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:43:57.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:56 smithi151 bash[23756]: audit 2024-03-04T15:43:56.164523+0000 mgr.y (mgr.14152) 146 : audit 0 from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:43:57.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:56 smithi151 bash[23756]: audit 2024-03-04T15:43:56.171338+0000 mon.a (mon.0) 347 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:43:57.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:56 smithi151 bash[23756]: audit 2024-03-04T15:43:56.176989+0000 mon.a (mon.0) 348 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:43:57.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:56 smithi151 bash[23756]: audit 2024-03-04T15:43:56.178304+0000 mon.a (mon.0) 349 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:43:58.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:57 smithi117 bash[21480]: cluster 2024-03-04T15:43:56.745751+0000 mgr.y (mgr.14152) 147 : cluster 0 pgmap v111: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:58.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:57 smithi117 bash[28843]: cluster 2024-03-04T15:43:56.745751+0000 mgr.y (mgr.14152) 147 : cluster 0 pgmap v111: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:43:58.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:57 smithi151 bash[23756]: cluster 2024-03-04T15:43:56.745751+0000 mgr.y (mgr.14152) 147 : cluster 0 pgmap v111: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:00.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:43:59 smithi151 bash[23756]: cluster 2024-03-04T15:43:58.746235+0000 mgr.y (mgr.14152) 148 : cluster 0 pgmap v112: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:00.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:43:59 smithi117 bash[28843]: cluster 2024-03-04T15:43:58.746235+0000 mgr.y (mgr.14152) 148 : cluster 0 pgmap v112: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:00.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:43:59 smithi117 bash[21480]: cluster 2024-03-04T15:43:58.746235+0000 mgr.y (mgr.14152) 148 : cluster 0 pgmap v112: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:02.082 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:01 smithi117 bash[21480]: cluster 2024-03-04T15:44:00.746744+0000 mgr.y (mgr.14152) 149 : cluster 0 pgmap v113: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:02.082 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:01 smithi117 bash[28843]: cluster 2024-03-04T15:44:00.746744+0000 mgr.y (mgr.14152) 149 : cluster 0 pgmap v113: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:02.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:01 smithi151 bash[23756]: cluster 2024-03-04T15:44:00.746744+0000 mgr.y (mgr.14152) 149 : cluster 0 pgmap v113: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:04.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:03 smithi151 bash[23756]: cluster 2024-03-04T15:44:02.747124+0000 mgr.y (mgr.14152) 150 : cluster 0 pgmap v114: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:04.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:03 smithi117 bash[21480]: cluster 2024-03-04T15:44:02.747124+0000 mgr.y (mgr.14152) 150 : cluster 0 pgmap v114: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:04.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:03 smithi117 bash[28843]: cluster 2024-03-04T15:44:02.747124+0000 mgr.y (mgr.14152) 150 : cluster 0 pgmap v114: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:05.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:04 smithi151 bash[23756]: audit 2024-03-04T15:44:04.357327+0000 mon.c (mon.2) 6 : audit 1 from='client.? 172.21.15.117:0/849919814' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49710dbf-78b9-4983-bc61-2a5d0527e417"}]: dispatch 2024-03-04T15:44:05.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:04 smithi151 bash[23756]: audit 2024-03-04T15:44:04.357980+0000 mon.a (mon.0) 350 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49710dbf-78b9-4983-bc61-2a5d0527e417"}]: dispatch 2024-03-04T15:44:05.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:04 smithi151 bash[23756]: audit 2024-03-04T15:44:04.362709+0000 mon.a (mon.0) 351 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49710dbf-78b9-4983-bc61-2a5d0527e417"}]': finished 2024-03-04T15:44:05.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:04 smithi151 bash[23756]: cluster 2024-03-04T15:44:04.367425+0000 mon.a (mon.0) 352 : cluster 0 osdmap e15: 3 total, 2 up, 3 in 2024-03-04T15:44:05.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:04 smithi151 bash[23756]: audit 2024-03-04T15:44:04.367870+0000 mon.a (mon.0) 353 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:44:05.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:04 smithi117 bash[21480]: audit 2024-03-04T15:44:04.357327+0000 mon.c (mon.2) 6 : audit 1 from='client.? 172.21.15.117:0/849919814' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49710dbf-78b9-4983-bc61-2a5d0527e417"}]: dispatch 2024-03-04T15:44:05.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:04 smithi117 bash[21480]: audit 2024-03-04T15:44:04.357980+0000 mon.a (mon.0) 350 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49710dbf-78b9-4983-bc61-2a5d0527e417"}]: dispatch 2024-03-04T15:44:05.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:04 smithi117 bash[21480]: audit 2024-03-04T15:44:04.362709+0000 mon.a (mon.0) 351 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49710dbf-78b9-4983-bc61-2a5d0527e417"}]': finished 2024-03-04T15:44:05.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:04 smithi117 bash[21480]: cluster 2024-03-04T15:44:04.367425+0000 mon.a (mon.0) 352 : cluster 0 osdmap e15: 3 total, 2 up, 3 in 2024-03-04T15:44:05.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:04 smithi117 bash[21480]: audit 2024-03-04T15:44:04.367870+0000 mon.a (mon.0) 353 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:44:05.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:04 smithi117 bash[28843]: audit 2024-03-04T15:44:04.357327+0000 mon.c (mon.2) 6 : audit 1 from='client.? 172.21.15.117:0/849919814' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49710dbf-78b9-4983-bc61-2a5d0527e417"}]: dispatch 2024-03-04T15:44:05.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:04 smithi117 bash[28843]: audit 2024-03-04T15:44:04.357980+0000 mon.a (mon.0) 350 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "49710dbf-78b9-4983-bc61-2a5d0527e417"}]: dispatch 2024-03-04T15:44:05.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:04 smithi117 bash[28843]: audit 2024-03-04T15:44:04.362709+0000 mon.a (mon.0) 351 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "49710dbf-78b9-4983-bc61-2a5d0527e417"}]': finished 2024-03-04T15:44:05.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:04 smithi117 bash[28843]: cluster 2024-03-04T15:44:04.367425+0000 mon.a (mon.0) 352 : cluster 0 osdmap e15: 3 total, 2 up, 3 in 2024-03-04T15:44:05.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:04 smithi117 bash[28843]: audit 2024-03-04T15:44:04.367870+0000 mon.a (mon.0) 353 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:44:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:05 smithi151 bash[23756]: cluster 2024-03-04T15:44:04.747544+0000 mgr.y (mgr.14152) 151 : cluster 0 pgmap v116: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:05 smithi151 bash[23756]: audit 2024-03-04T15:44:05.088641+0000 mon.a (mon.0) 354 : audit 0 from='client.? 172.21.15.117:0/3584581631' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:44:06.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:05 smithi117 bash[21480]: cluster 2024-03-04T15:44:04.747544+0000 mgr.y (mgr.14152) 151 : cluster 0 pgmap v116: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:06.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:05 smithi117 bash[21480]: audit 2024-03-04T15:44:05.088641+0000 mon.a (mon.0) 354 : audit 0 from='client.? 172.21.15.117:0/3584581631' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:44:06.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:05 smithi117 bash[28843]: cluster 2024-03-04T15:44:04.747544+0000 mgr.y (mgr.14152) 151 : cluster 0 pgmap v116: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:06.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:05 smithi117 bash[28843]: audit 2024-03-04T15:44:05.088641+0000 mon.a (mon.0) 354 : audit 0 from='client.? 172.21.15.117:0/3584581631' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:44:08.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:07 smithi151 bash[23756]: cluster 2024-03-04T15:44:06.748086+0000 mgr.y (mgr.14152) 152 : cluster 0 pgmap v117: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:08.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:07 smithi117 bash[28843]: cluster 2024-03-04T15:44:06.748086+0000 mgr.y (mgr.14152) 152 : cluster 0 pgmap v117: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:08.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:07 smithi117 bash[21480]: cluster 2024-03-04T15:44:06.748086+0000 mgr.y (mgr.14152) 152 : cluster 0 pgmap v117: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:09 smithi151 bash[23756]: cluster 2024-03-04T15:44:08.748556+0000 mgr.y (mgr.14152) 153 : cluster 0 pgmap v118: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:10.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:09 smithi117 bash[21480]: cluster 2024-03-04T15:44:08.748556+0000 mgr.y (mgr.14152) 153 : cluster 0 pgmap v118: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:10.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:09 smithi117 bash[28843]: cluster 2024-03-04T15:44:08.748556+0000 mgr.y (mgr.14152) 153 : cluster 0 pgmap v118: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:11 smithi151 bash[23756]: cluster 2024-03-04T15:44:10.749015+0000 mgr.y (mgr.14152) 154 : cluster 0 pgmap v119: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:12.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:11 smithi117 bash[21480]: cluster 2024-03-04T15:44:10.749015+0000 mgr.y (mgr.14152) 154 : cluster 0 pgmap v119: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:12.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:11 smithi117 bash[28843]: cluster 2024-03-04T15:44:10.749015+0000 mgr.y (mgr.14152) 154 : cluster 0 pgmap v119: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:14.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:13 smithi151 bash[23756]: cluster 2024-03-04T15:44:12.749476+0000 mgr.y (mgr.14152) 155 : cluster 0 pgmap v120: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:14.315 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:13 smithi117 bash[21480]: cluster 2024-03-04T15:44:12.749476+0000 mgr.y (mgr.14152) 155 : cluster 0 pgmap v120: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:14.315 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:13 smithi117 bash[28843]: cluster 2024-03-04T15:44:12.749476+0000 mgr.y (mgr.14152) 155 : cluster 0 pgmap v120: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:16.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:15 smithi117 bash[21480]: cluster 2024-03-04T15:44:14.749893+0000 mgr.y (mgr.14152) 156 : cluster 0 pgmap v121: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:16.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:15 smithi117 bash[28843]: cluster 2024-03-04T15:44:14.749893+0000 mgr.y (mgr.14152) 156 : cluster 0 pgmap v121: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:16.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:15 smithi151 bash[23756]: cluster 2024-03-04T15:44:14.749893+0000 mgr.y (mgr.14152) 156 : cluster 0 pgmap v121: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:17.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:16 smithi151 bash[23756]: audit 2024-03-04T15:44:16.426862+0000 mon.a (mon.0) 355 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-04T15:44:17.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:16 smithi151 bash[23756]: audit 2024-03-04T15:44:16.428518+0000 mon.a (mon.0) 356 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:44:17.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:16 smithi151 bash[23756]: cephadm 2024-03-04T15:44:16.429980+0000 mgr.y (mgr.14152) 157 : cephadm 1 Deploying daemon osd.2 on smithi117 2024-03-04T15:44:17.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:16 smithi117 bash[21480]: audit 2024-03-04T15:44:16.426862+0000 mon.a (mon.0) 355 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-04T15:44:17.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:16 smithi117 bash[21480]: audit 2024-03-04T15:44:16.428518+0000 mon.a (mon.0) 356 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:44:17.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:16 smithi117 bash[21480]: cephadm 2024-03-04T15:44:16.429980+0000 mgr.y (mgr.14152) 157 : cephadm 1 Deploying daemon osd.2 on smithi117 2024-03-04T15:44:17.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:16 smithi117 bash[28843]: audit 2024-03-04T15:44:16.426862+0000 mon.a (mon.0) 355 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-03-04T15:44:17.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:16 smithi117 bash[28843]: audit 2024-03-04T15:44:16.428518+0000 mon.a (mon.0) 356 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:44:17.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:16 smithi117 bash[28843]: cephadm 2024-03-04T15:44:16.429980+0000 mgr.y (mgr.14152) 157 : cephadm 1 Deploying daemon osd.2 on smithi117 2024-03-04T15:44:17.925 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:17 smithi117 bash[21480]: cluster 2024-03-04T15:44:16.750351+0000 mgr.y (mgr.14152) 158 : cluster 0 pgmap v122: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:17.925 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:17 smithi117 bash[28843]: cluster 2024-03-04T15:44:16.750351+0000 mgr.y (mgr.14152) 158 : cluster 0 pgmap v122: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:18.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:17 smithi151 bash[23756]: cluster 2024-03-04T15:44:16.750351+0000 mgr.y (mgr.14152) 158 : cluster 0 pgmap v122: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:20.132 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:19 smithi117 bash[28843]: cluster 2024-03-04T15:44:18.750762+0000 mgr.y (mgr.14152) 159 : cluster 0 pgmap v123: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:20.133 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:19 smithi117 bash[21480]: cluster 2024-03-04T15:44:18.750762+0000 mgr.y (mgr.14152) 159 : cluster 0 pgmap v123: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:20.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:19 smithi151 bash[23756]: cluster 2024-03-04T15:44:18.750762+0000 mgr.y (mgr.14152) 159 : cluster 0 pgmap v123: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:20.399 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:20 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:44:20.399 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:44:20 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:44:20.399 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:20 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:44:20.400 INFO:journalctl@ceph.osd.1.smithi117.stdout:Mar 04 15:44:20 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:44:20.400 INFO:journalctl@ceph.osd.0.smithi117.stdout:Mar 04 15:44:20 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:44:20.685 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:44:20 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:44:20.685 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:20 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:44:20.685 INFO:journalctl@ceph.osd.0.smithi117.stdout:Mar 04 15:44:20 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:44:20.686 INFO:journalctl@ceph.osd.1.smithi117.stdout:Mar 04 15:44:20 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:44:20.686 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:20 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:44:21.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:21 smithi117 bash[21480]: audit 2024-03-04T15:44:20.680521+0000 mon.a (mon.0) 357 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:21.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:21 smithi117 bash[21480]: audit 2024-03-04T15:44:20.687645+0000 mon.a (mon.0) 358 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:21.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:21 smithi117 bash[21480]: audit 2024-03-04T15:44:20.696551+0000 mon.a (mon.0) 359 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:44:21.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:21 smithi117 bash[28843]: audit 2024-03-04T15:44:20.680521+0000 mon.a (mon.0) 357 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:21.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:21 smithi117 bash[28843]: audit 2024-03-04T15:44:20.687645+0000 mon.a (mon.0) 358 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:21.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:21 smithi117 bash[28843]: audit 2024-03-04T15:44:20.696551+0000 mon.a (mon.0) 359 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:44:21.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:21 smithi151 bash[23756]: audit 2024-03-04T15:44:20.680521+0000 mon.a (mon.0) 357 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:21.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:21 smithi151 bash[23756]: audit 2024-03-04T15:44:20.687645+0000 mon.a (mon.0) 358 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:21.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:21 smithi151 bash[23756]: audit 2024-03-04T15:44:20.696551+0000 mon.a (mon.0) 359 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:44:23.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:22 smithi117 bash[21480]: cluster 2024-03-04T15:44:20.751151+0000 mgr.y (mgr.14152) 160 : cluster 0 pgmap v124: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:23.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:22 smithi117 bash[28843]: cluster 2024-03-04T15:44:20.751151+0000 mgr.y (mgr.14152) 160 : cluster 0 pgmap v124: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:23.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:22 smithi151 bash[23756]: cluster 2024-03-04T15:44:20.751151+0000 mgr.y (mgr.14152) 160 : cluster 0 pgmap v124: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:25.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:24 smithi117 bash[21480]: cluster 2024-03-04T15:44:22.751426+0000 mgr.y (mgr.14152) 161 : cluster 0 pgmap v125: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:25.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:24 smithi117 bash[28843]: cluster 2024-03-04T15:44:22.751426+0000 mgr.y (mgr.14152) 161 : cluster 0 pgmap v125: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:25.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:24 smithi151 bash[23756]: cluster 2024-03-04T15:44:22.751426+0000 mgr.y (mgr.14152) 161 : cluster 0 pgmap v125: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:27.053 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:26 smithi117 bash[21480]: cluster 2024-03-04T15:44:24.751838+0000 mgr.y (mgr.14152) 162 : cluster 0 pgmap v126: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:27.053 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:26 smithi117 bash[28843]: cluster 2024-03-04T15:44:24.751838+0000 mgr.y (mgr.14152) 162 : cluster 0 pgmap v126: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:26 smithi151 bash[23756]: cluster 2024-03-04T15:44:24.751838+0000 mgr.y (mgr.14152) 162 : cluster 0 pgmap v126: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:28.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:27 smithi117 bash[21480]: audit 2024-03-04T15:44:27.057263+0000 mon.a (mon.0) 360 : audit 1 from='osd.2 [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-04T15:44:28.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:27 smithi117 bash[21480]: audit 2024-03-04T15:44:27.584650+0000 mon.a (mon.0) 361 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:28.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:27 smithi117 bash[21480]: audit 2024-03-04T15:44:27.591996+0000 mon.a (mon.0) 362 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:28.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:27 smithi117 bash[28843]: audit 2024-03-04T15:44:27.057263+0000 mon.a (mon.0) 360 : audit 1 from='osd.2 [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-04T15:44:28.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:27 smithi117 bash[28843]: audit 2024-03-04T15:44:27.584650+0000 mon.a (mon.0) 361 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:28.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:27 smithi117 bash[28843]: audit 2024-03-04T15:44:27.591996+0000 mon.a (mon.0) 362 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:28.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:27 smithi151 bash[23756]: audit 2024-03-04T15:44:27.057263+0000 mon.a (mon.0) 360 : audit 1 from='osd.2 [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-03-04T15:44:28.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:27 smithi151 bash[23756]: audit 2024-03-04T15:44:27.584650+0000 mon.a (mon.0) 361 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:28.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:27 smithi151 bash[23756]: audit 2024-03-04T15:44:27.591996+0000 mon.a (mon.0) 362 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:29.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:28 smithi117 bash[28843]: cluster 2024-03-04T15:44:26.752295+0000 mgr.y (mgr.14152) 163 : cluster 0 pgmap v127: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:29.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:28 smithi117 bash[28843]: audit 2024-03-04T15:44:27.710741+0000 mon.a (mon.0) 363 : audit 1 from='osd.2 [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-04T15:44:29.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:28 smithi117 bash[28843]: cluster 2024-03-04T15:44:27.716824+0000 mon.a (mon.0) 364 : cluster 0 osdmap e16: 3 total, 2 up, 3 in 2024-03-04T15:44:29.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:28 smithi117 bash[28843]: audit 2024-03-04T15:44:27.717195+0000 mon.a (mon.0) 365 : audit 1 from='osd.2 [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-04T15:44:29.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:28 smithi117 bash[28843]: audit 2024-03-04T15:44:27.717583+0000 mon.a (mon.0) 366 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:44:29.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:28 smithi117 bash[28843]: audit 2024-03-04T15:44:28.165181+0000 mon.a (mon.0) 367 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:44:29.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:28 smithi117 bash[28843]: audit 2024-03-04T15:44:28.165902+0000 mon.a (mon.0) 368 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:44:29.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:28 smithi117 bash[28843]: audit 2024-03-04T15:44:28.172248+0000 mon.a (mon.0) 369 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:29.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:28 smithi117 bash[21480]: cluster 2024-03-04T15:44:26.752295+0000 mgr.y (mgr.14152) 163 : cluster 0 pgmap v127: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:29.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:28 smithi117 bash[21480]: audit 2024-03-04T15:44:27.710741+0000 mon.a (mon.0) 363 : audit 1 from='osd.2 [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-04T15:44:29.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:28 smithi117 bash[21480]: cluster 2024-03-04T15:44:27.716824+0000 mon.a (mon.0) 364 : cluster 0 osdmap e16: 3 total, 2 up, 3 in 2024-03-04T15:44:29.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:28 smithi117 bash[21480]: audit 2024-03-04T15:44:27.717195+0000 mon.a (mon.0) 365 : audit 1 from='osd.2 [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-04T15:44:29.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:28 smithi117 bash[21480]: audit 2024-03-04T15:44:27.717583+0000 mon.a (mon.0) 366 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:44:29.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:28 smithi117 bash[21480]: audit 2024-03-04T15:44:28.165181+0000 mon.a (mon.0) 367 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:44:29.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:28 smithi117 bash[21480]: audit 2024-03-04T15:44:28.165902+0000 mon.a (mon.0) 368 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:44:29.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:28 smithi117 bash[21480]: audit 2024-03-04T15:44:28.172248+0000 mon.a (mon.0) 369 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:28 smithi151 bash[23756]: cluster 2024-03-04T15:44:26.752295+0000 mgr.y (mgr.14152) 163 : cluster 0 pgmap v127: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:28 smithi151 bash[23756]: audit 2024-03-04T15:44:27.710741+0000 mon.a (mon.0) 363 : audit 1 from='osd.2 [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427]' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-03-04T15:44:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:28 smithi151 bash[23756]: cluster 2024-03-04T15:44:27.716824+0000 mon.a (mon.0) 364 : cluster 0 osdmap e16: 3 total, 2 up, 3 in 2024-03-04T15:44:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:28 smithi151 bash[23756]: audit 2024-03-04T15:44:27.717195+0000 mon.a (mon.0) 365 : audit 1 from='osd.2 [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-04T15:44:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:28 smithi151 bash[23756]: audit 2024-03-04T15:44:27.717583+0000 mon.a (mon.0) 366 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:44:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:28 smithi151 bash[23756]: audit 2024-03-04T15:44:28.165181+0000 mon.a (mon.0) 367 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:44:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:28 smithi151 bash[23756]: audit 2024-03-04T15:44:28.165902+0000 mon.a (mon.0) 368 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:44:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:28 smithi151 bash[23756]: audit 2024-03-04T15:44:28.172248+0000 mon.a (mon.0) 369 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:30.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:29 smithi117 bash[28843]: audit 2024-03-04T15:44:28.713038+0000 mon.a (mon.0) 370 : audit 1 from='osd.2 [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-03-04T15:44:30.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:29 smithi117 bash[28843]: cluster 2024-03-04T15:44:28.720297+0000 mon.a (mon.0) 371 : cluster 0 osdmap e17: 3 total, 2 up, 3 in 2024-03-04T15:44:30.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:29 smithi117 bash[28843]: audit 2024-03-04T15:44:28.722553+0000 mon.a (mon.0) 372 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:44:30.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:29 smithi117 bash[28843]: audit 2024-03-04T15:44:28.751088+0000 mon.a (mon.0) 373 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:44:30.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:29 smithi117 bash[28843]: cluster 2024-03-04T15:44:28.752683+0000 mgr.y (mgr.14152) 164 : cluster 0 pgmap v130: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:30.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:29 smithi117 bash[28843]: audit 2024-03-04T15:44:29.200240+0000 mon.a (mon.0) 374 : audit 1 from='osd.2 [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427]' entity='osd.2' 2024-03-04T15:44:30.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:29 smithi117 bash[21480]: audit 2024-03-04T15:44:28.713038+0000 mon.a (mon.0) 370 : audit 1 from='osd.2 [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-03-04T15:44:30.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:29 smithi117 bash[21480]: cluster 2024-03-04T15:44:28.720297+0000 mon.a (mon.0) 371 : cluster 0 osdmap e17: 3 total, 2 up, 3 in 2024-03-04T15:44:30.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:29 smithi117 bash[21480]: audit 2024-03-04T15:44:28.722553+0000 mon.a (mon.0) 372 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:44:30.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:29 smithi117 bash[21480]: audit 2024-03-04T15:44:28.751088+0000 mon.a (mon.0) 373 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:44:30.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:29 smithi117 bash[21480]: cluster 2024-03-04T15:44:28.752683+0000 mgr.y (mgr.14152) 164 : cluster 0 pgmap v130: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:30.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:29 smithi117 bash[21480]: audit 2024-03-04T15:44:29.200240+0000 mon.a (mon.0) 374 : audit 1 from='osd.2 [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427]' entity='osd.2' 2024-03-04T15:44:30.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:29 smithi151 bash[23756]: audit 2024-03-04T15:44:28.713038+0000 mon.a (mon.0) 370 : audit 1 from='osd.2 [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427]' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-03-04T15:44:30.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:29 smithi151 bash[23756]: cluster 2024-03-04T15:44:28.720297+0000 mon.a (mon.0) 371 : cluster 0 osdmap e17: 3 total, 2 up, 3 in 2024-03-04T15:44:30.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:29 smithi151 bash[23756]: audit 2024-03-04T15:44:28.722553+0000 mon.a (mon.0) 372 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:44:30.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:29 smithi151 bash[23756]: audit 2024-03-04T15:44:28.751088+0000 mon.a (mon.0) 373 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:44:30.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:29 smithi151 bash[23756]: cluster 2024-03-04T15:44:28.752683+0000 mgr.y (mgr.14152) 164 : cluster 0 pgmap v130: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-03-04T15:44:30.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:29 smithi151 bash[23756]: audit 2024-03-04T15:44:29.200240+0000 mon.a (mon.0) 374 : audit 1 from='osd.2 [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427]' entity='osd.2' 2024-03-04T15:44:31.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:30 smithi117 bash[21480]: cluster 2024-03-04T15:44:28.024937+0000 osd.2 (osd.2) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:44:31.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:30 smithi117 bash[21480]: cluster 2024-03-04T15:44:28.025010+0000 osd.2 (osd.2) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:44:31.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:30 smithi117 bash[21480]: audit 2024-03-04T15:44:29.728591+0000 mon.a (mon.0) 375 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:44:31.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:30 smithi117 bash[21480]: cluster 2024-03-04T15:44:30.208254+0000 mon.a (mon.0) 376 : cluster 1 osd.2 [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427] boot 2024-03-04T15:44:31.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:30 smithi117 bash[21480]: cluster 2024-03-04T15:44:30.208366+0000 mon.a (mon.0) 377 : cluster 0 osdmap e18: 3 total, 3 up, 3 in 2024-03-04T15:44:31.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:30 smithi117 bash[21480]: audit 2024-03-04T15:44:30.209125+0000 mon.a (mon.0) 378 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:44:31.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:30 smithi117 bash[28843]: cluster 2024-03-04T15:44:28.024937+0000 osd.2 (osd.2) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:44:31.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:30 smithi117 bash[28843]: cluster 2024-03-04T15:44:28.025010+0000 osd.2 (osd.2) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:44:31.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:30 smithi117 bash[28843]: audit 2024-03-04T15:44:29.728591+0000 mon.a (mon.0) 375 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:44:31.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:30 smithi117 bash[28843]: cluster 2024-03-04T15:44:30.208254+0000 mon.a (mon.0) 376 : cluster 1 osd.2 [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427] boot 2024-03-04T15:44:31.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:30 smithi117 bash[28843]: cluster 2024-03-04T15:44:30.208366+0000 mon.a (mon.0) 377 : cluster 0 osdmap e18: 3 total, 3 up, 3 in 2024-03-04T15:44:31.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:30 smithi117 bash[28843]: audit 2024-03-04T15:44:30.209125+0000 mon.a (mon.0) 378 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:44:31.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:30 smithi151 bash[23756]: cluster 2024-03-04T15:44:28.024937+0000 osd.2 (osd.2) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:44:31.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:30 smithi151 bash[23756]: cluster 2024-03-04T15:44:28.025010+0000 osd.2 (osd.2) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:44:31.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:30 smithi151 bash[23756]: audit 2024-03-04T15:44:29.728591+0000 mon.a (mon.0) 375 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:44:31.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:30 smithi151 bash[23756]: cluster 2024-03-04T15:44:30.208254+0000 mon.a (mon.0) 376 : cluster 1 osd.2 [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427] boot 2024-03-04T15:44:31.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:30 smithi151 bash[23756]: cluster 2024-03-04T15:44:30.208366+0000 mon.a (mon.0) 377 : cluster 0 osdmap e18: 3 total, 3 up, 3 in 2024-03-04T15:44:31.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:30 smithi151 bash[23756]: audit 2024-03-04T15:44:30.209125+0000 mon.a (mon.0) 378 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:44:31.380 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:31.373+0000 7ff695ffb700 1 -- 172.21.15.117:0/1803068970 <== mgr.14152 v2:172.21.15.117:6800/2865911351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff698062220 con 0x7ff68406d600 2024-03-04T15:44:31.381 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 2 on host 'smithi117' 2024-03-04T15:44:31.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:31.377+0000 7ff69e2a1700 1 -- 172.21.15.117:0/1803068970 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7ff68406d600 msgr2=0x7ff68406fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:44:31.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:31.377+0000 7ff69e2a1700 1 --2- 172.21.15.117:0/1803068970 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7ff68406d600 0x7ff68406fac0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff688005950 tx=0x7ff6880058e0 comp rx=0 tx=0).stop 2024-03-04T15:44:31.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:31.377+0000 7ff69e2a1700 1 -- 172.21.15.117:0/1803068970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff698105290 msgr2=0x7ff698199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:44:31.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:31.377+0000 7ff69e2a1700 1 --2- 172.21.15.117:0/1803068970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff698105290 0x7ff698199d10 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7ff680000c00 tx=0x7ff68000bbe0 comp rx=0 tx=0).stop 2024-03-04T15:44:31.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:31.377+0000 7ff69e2a1700 1 -- 172.21.15.117:0/1803068970 shutdown_connections 2024-03-04T15:44:31.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:31.377+0000 7ff69e2a1700 1 --2- 172.21.15.117:0/1803068970 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7ff68406d600 0x7ff68406fac0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:44:31.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:31.377+0000 7ff69e2a1700 1 --2- 172.21.15.117:0/1803068970 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff698105c30 0x7ff6981a0d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:44:31.383 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:31.377+0000 7ff69e2a1700 1 --2- 172.21.15.117:0/1803068970 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff698104090 0x7ff6981997d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:44:31.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:31.377+0000 7ff69e2a1700 1 --2- 172.21.15.117:0/1803068970 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff698105290 0x7ff698199d10 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:44:31.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:31.377+0000 7ff69e2a1700 1 -- 172.21.15.117:0/1803068970 >> 172.21.15.117:0/1803068970 conn(0x7ff6980ff670 msgr2=0x7ff698108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:44:31.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:31.377+0000 7ff69e2a1700 1 -- 172.21.15.117:0/1803068970 shutdown_connections 2024-03-04T15:44:31.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:31.377+0000 7ff69e2a1700 1 -- 172.21.15.117:0/1803068970 wait complete. 2024-03-04T15:44:32.021 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:31 smithi117 bash[28843]: cluster 2024-03-04T15:44:30.753225+0000 mgr.y (mgr.14152) 165 : cluster 0 pgmap v132: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:32.021 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:31 smithi117 bash[28843]: audit 2024-03-04T15:44:30.883565+0000 mon.a (mon.0) 379 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:44:32.021 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:31 smithi117 bash[28843]: audit 2024-03-04T15:44:31.327349+0000 mon.a (mon.0) 380 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:44:32.021 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:31 smithi117 bash[28843]: audit 2024-03-04T15:44:31.334204+0000 mon.a (mon.0) 381 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:32.022 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:31 smithi117 bash[28843]: audit 2024-03-04T15:44:31.344025+0000 mon.a (mon.0) 382 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:32.022 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:31 smithi117 bash[21480]: cluster 2024-03-04T15:44:30.753225+0000 mgr.y (mgr.14152) 165 : cluster 0 pgmap v132: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:32.022 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:31 smithi117 bash[21480]: audit 2024-03-04T15:44:30.883565+0000 mon.a (mon.0) 379 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:44:32.022 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:31 smithi117 bash[21480]: audit 2024-03-04T15:44:31.327349+0000 mon.a (mon.0) 380 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:44:32.022 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:31 smithi117 bash[21480]: audit 2024-03-04T15:44:31.334204+0000 mon.a (mon.0) 381 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:32.022 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:31 smithi117 bash[21480]: audit 2024-03-04T15:44:31.344025+0000 mon.a (mon.0) 382 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:32.024 DEBUG:teuthology.orchestra.run.smithi117:osd.2> sudo journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.2.service 2024-03-04T15:44:32.026 INFO:tasks.cephadm:Deploying osd.3 on smithi117 with /dev/vg_nvme/lv_1... 2024-03-04T15:44:32.027 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-04T15:44:32.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:31 smithi151 bash[23756]: cluster 2024-03-04T15:44:30.753225+0000 mgr.y (mgr.14152) 165 : cluster 0 pgmap v132: 0 pgs: ; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:32.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:31 smithi151 bash[23756]: audit 2024-03-04T15:44:30.883565+0000 mon.a (mon.0) 379 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:44:32.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:31 smithi151 bash[23756]: audit 2024-03-04T15:44:31.327349+0000 mon.a (mon.0) 380 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:44:32.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:31 smithi151 bash[23756]: audit 2024-03-04T15:44:31.334204+0000 mon.a (mon.0) 381 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:32.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:31 smithi151 bash[23756]: audit 2024-03-04T15:44:31.344025+0000 mon.a (mon.0) 382 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:33.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:32 smithi117 bash[21480]: audit 2024-03-04T15:44:31.738739+0000 mon.a (mon.0) 383 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-04T15:44:33.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:32 smithi117 bash[21480]: cluster 2024-03-04T15:44:31.743675+0000 mon.a (mon.0) 384 : cluster 0 osdmap e19: 3 total, 3 up, 3 in 2024-03-04T15:44:33.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:32 smithi117 bash[21480]: audit 2024-03-04T15:44:31.745775+0000 mon.a (mon.0) 385 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:44:33.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:32 smithi117 bash[28843]: audit 2024-03-04T15:44:31.738739+0000 mon.a (mon.0) 383 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-04T15:44:33.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:32 smithi117 bash[28843]: cluster 2024-03-04T15:44:31.743675+0000 mon.a (mon.0) 384 : cluster 0 osdmap e19: 3 total, 3 up, 3 in 2024-03-04T15:44:33.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:32 smithi117 bash[28843]: audit 2024-03-04T15:44:31.745775+0000 mon.a (mon.0) 385 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:44:33.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:32 smithi151 bash[23756]: audit 2024-03-04T15:44:31.738739+0000 mon.a (mon.0) 383 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-03-04T15:44:33.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:32 smithi151 bash[23756]: cluster 2024-03-04T15:44:31.743675+0000 mon.a (mon.0) 384 : cluster 0 osdmap e19: 3 total, 3 up, 3 in 2024-03-04T15:44:33.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:32 smithi151 bash[23756]: audit 2024-03-04T15:44:31.745775+0000 mon.a (mon.0) 385 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:44:34.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:33 smithi117 bash[21480]: audit 2024-03-04T15:44:32.748276+0000 mon.a (mon.0) 386 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-04T15:44:34.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:33 smithi117 bash[21480]: cluster 2024-03-04T15:44:32.753800+0000 mgr.y (mgr.14152) 166 : cluster 0 pgmap v135: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:34.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:33 smithi117 bash[21480]: cluster 2024-03-04T15:44:32.755609+0000 mon.a (mon.0) 387 : cluster 0 osdmap e20: 3 total, 3 up, 3 in 2024-03-04T15:44:34.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:33 smithi117 bash[28843]: audit 2024-03-04T15:44:32.748276+0000 mon.a (mon.0) 386 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-04T15:44:34.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:33 smithi117 bash[28843]: cluster 2024-03-04T15:44:32.753800+0000 mgr.y (mgr.14152) 166 : cluster 0 pgmap v135: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:34.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:33 smithi117 bash[28843]: cluster 2024-03-04T15:44:32.755609+0000 mon.a (mon.0) 387 : cluster 0 osdmap e20: 3 total, 3 up, 3 in 2024-03-04T15:44:34.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:33 smithi151 bash[23756]: audit 2024-03-04T15:44:32.748276+0000 mon.a (mon.0) 386 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-03-04T15:44:34.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:33 smithi151 bash[23756]: cluster 2024-03-04T15:44:32.753800+0000 mgr.y (mgr.14152) 166 : cluster 0 pgmap v135: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:34.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:33 smithi151 bash[23756]: cluster 2024-03-04T15:44:32.755609+0000 mon.a (mon.0) 387 : cluster 0 osdmap e20: 3 total, 3 up, 3 in 2024-03-04T15:44:35.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:34 smithi117 bash[21480]: cluster 2024-03-04T15:44:33.766356+0000 mon.a (mon.0) 388 : cluster 0 osdmap e21: 3 total, 3 up, 3 in 2024-03-04T15:44:35.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:34 smithi117 bash[21480]: audit 2024-03-04T15:44:34.056604+0000 mon.a (mon.0) 389 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:35.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:34 smithi117 bash[28843]: cluster 2024-03-04T15:44:33.766356+0000 mon.a (mon.0) 388 : cluster 0 osdmap e21: 3 total, 3 up, 3 in 2024-03-04T15:44:35.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:34 smithi117 bash[28843]: audit 2024-03-04T15:44:34.056604+0000 mon.a (mon.0) 389 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:35.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:34 smithi151 bash[23756]: cluster 2024-03-04T15:44:33.766356+0000 mon.a (mon.0) 388 : cluster 0 osdmap e21: 3 total, 3 up, 3 in 2024-03-04T15:44:35.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:34 smithi151 bash[23756]: audit 2024-03-04T15:44:34.056604+0000 mon.a (mon.0) 389 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:36.336 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:44:37.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:36 smithi117 bash[21480]: cluster 2024-03-04T15:44:34.754336+0000 mgr.y (mgr.14152) 167 : cluster 0 pgmap v137: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:37.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:36 smithi117 bash[21480]: cluster 2024-03-04T15:44:35.781945+0000 mon.a (mon.0) 390 : cluster 0 mgrmap e15: y(active, since 4m), standbys: x 2024-03-04T15:44:37.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:36 smithi117 bash[21480]: audit 2024-03-04T15:44:35.831147+0000 mon.a (mon.0) 391 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T15:44:37.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:36 smithi117 bash[28843]: cluster 2024-03-04T15:44:34.754336+0000 mgr.y (mgr.14152) 167 : cluster 0 pgmap v137: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:37.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:36 smithi117 bash[28843]: cluster 2024-03-04T15:44:35.781945+0000 mon.a (mon.0) 390 : cluster 0 mgrmap e15: y(active, since 4m), standbys: x 2024-03-04T15:44:37.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:36 smithi117 bash[28843]: audit 2024-03-04T15:44:35.831147+0000 mon.a (mon.0) 391 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T15:44:37.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:36 smithi151 bash[23756]: cluster 2024-03-04T15:44:34.754336+0000 mgr.y (mgr.14152) 167 : cluster 0 pgmap v137: 1 pgs: 1 unknown; 0 B data, 80 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:37.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:36 smithi151 bash[23756]: cluster 2024-03-04T15:44:35.781945+0000 mon.a (mon.0) 390 : cluster 0 mgrmap e15: y(active, since 4m), standbys: x 2024-03-04T15:44:37.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:36 smithi151 bash[23756]: audit 2024-03-04T15:44:35.831147+0000 mon.a (mon.0) 391 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T15:44:38.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[21480]: audit 2024-03-04T15:44:36.714523+0000 mon.a (mon.0) 392 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T15:44:38.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[21480]: audit 2024-03-04T15:44:36.716551+0000 mon.a (mon.0) 393 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:44:38.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[21480]: audit 2024-03-04T15:44:36.720398+0000 mon.a (mon.0) 394 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:44:38.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[21480]: audit 2024-03-04T15:44:36.720543+0000 mon.a (mon.0) 395 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:44:38.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[21480]: audit 2024-03-04T15:44:36.723057+0000 mon.b (mon.1) 8 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T15:44:38.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[21480]: audit 2024-03-04T15:44:36.723379+0000 mon.a (mon.0) 396 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:44:38.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[21480]: audit 2024-03-04T15:44:36.723578+0000 mon.a (mon.0) 397 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:44:38.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[21480]: audit 2024-03-04T15:44:36.723757+0000 mon.a (mon.0) 398 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:44:38.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[21480]: audit 2024-03-04T15:44:37.628519+0000 mon.b (mon.1) 9 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T15:44:38.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[21480]: audit 2024-03-04T15:44:37.656179+0000 mon.a (mon.0) 399 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:38.823 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[21480]: audit 2024-03-04T15:44:37.658142+0000 mon.a (mon.0) 400 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:44:38.823 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[21480]: audit 2024-03-04T15:44:37.658718+0000 mon.a (mon.0) 401 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:44:38.823 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[21480]: audit 2024-03-04T15:44:37.658950+0000 mon.a (mon.0) 402 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:44:38.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[28843]: audit 2024-03-04T15:44:36.714523+0000 mon.a (mon.0) 392 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T15:44:38.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[28843]: audit 2024-03-04T15:44:36.716551+0000 mon.a (mon.0) 393 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:44:38.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[28843]: audit 2024-03-04T15:44:36.720398+0000 mon.a (mon.0) 394 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:44:38.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[28843]: audit 2024-03-04T15:44:36.720543+0000 mon.a (mon.0) 395 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:44:38.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[28843]: audit 2024-03-04T15:44:36.723057+0000 mon.b (mon.1) 8 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T15:44:38.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[28843]: audit 2024-03-04T15:44:36.723379+0000 mon.a (mon.0) 396 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:44:38.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[28843]: audit 2024-03-04T15:44:36.723578+0000 mon.a (mon.0) 397 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:44:38.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[28843]: audit 2024-03-04T15:44:36.723757+0000 mon.a (mon.0) 398 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:44:38.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[28843]: audit 2024-03-04T15:44:37.628519+0000 mon.b (mon.1) 9 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T15:44:38.824 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[28843]: audit 2024-03-04T15:44:37.656179+0000 mon.a (mon.0) 399 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:38.824 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[28843]: audit 2024-03-04T15:44:37.658142+0000 mon.a (mon.0) 400 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:44:38.824 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[28843]: audit 2024-03-04T15:44:37.658718+0000 mon.a (mon.0) 401 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:44:38.824 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:38 smithi117 bash[28843]: audit 2024-03-04T15:44:37.658950+0000 mon.a (mon.0) 402 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:44:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:38 smithi151 bash[23756]: audit 2024-03-04T15:44:36.714523+0000 mon.a (mon.0) 392 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T15:44:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:38 smithi151 bash[23756]: audit 2024-03-04T15:44:36.716551+0000 mon.a (mon.0) 393 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:44:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:38 smithi151 bash[23756]: audit 2024-03-04T15:44:36.720398+0000 mon.a (mon.0) 394 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:44:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:38 smithi151 bash[23756]: audit 2024-03-04T15:44:36.720543+0000 mon.a (mon.0) 395 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:44:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:38 smithi151 bash[23756]: audit 2024-03-04T15:44:36.723057+0000 mon.b (mon.1) 8 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T15:44:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:38 smithi151 bash[23756]: audit 2024-03-04T15:44:36.723379+0000 mon.a (mon.0) 396 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:44:38.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:38 smithi151 bash[23756]: audit 2024-03-04T15:44:36.723578+0000 mon.a (mon.0) 397 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:44:38.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:38 smithi151 bash[23756]: audit 2024-03-04T15:44:36.723757+0000 mon.a (mon.0) 398 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:44:38.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:38 smithi151 bash[23756]: audit 2024-03-04T15:44:37.628519+0000 mon.b (mon.1) 9 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T15:44:38.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:38 smithi151 bash[23756]: audit 2024-03-04T15:44:37.656179+0000 mon.a (mon.0) 399 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:38.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:38 smithi151 bash[23756]: audit 2024-03-04T15:44:37.658142+0000 mon.a (mon.0) 400 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:44:38.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:38 smithi151 bash[23756]: audit 2024-03-04T15:44:37.658718+0000 mon.a (mon.0) 401 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:44:38.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:38 smithi151 bash[23756]: audit 2024-03-04T15:44:37.658950+0000 mon.a (mon.0) 402 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:44:39.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:39 smithi117 bash[21480]: cluster 2024-03-04T15:44:36.754842+0000 mgr.y (mgr.14152) 168 : cluster 0 pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:39.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:39 smithi117 bash[21480]: audit 2024-03-04T15:44:37.657684+0000 mon.c (mon.2) 7 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T15:44:39.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:39 smithi117 bash[21480]: audit 2024-03-04T15:44:38.547560+0000 mon.c (mon.2) 8 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T15:44:39.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:39 smithi117 bash[28843]: cluster 2024-03-04T15:44:36.754842+0000 mgr.y (mgr.14152) 168 : cluster 0 pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:39.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:39 smithi117 bash[28843]: audit 2024-03-04T15:44:37.657684+0000 mon.c (mon.2) 7 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T15:44:39.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:39 smithi117 bash[28843]: audit 2024-03-04T15:44:38.547560+0000 mon.c (mon.2) 8 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T15:44:39.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:39 smithi151 bash[23756]: cluster 2024-03-04T15:44:36.754842+0000 mgr.y (mgr.14152) 168 : cluster 0 pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:39.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:39 smithi151 bash[23756]: audit 2024-03-04T15:44:37.657684+0000 mon.c (mon.2) 7 : audit 1 from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-03-04T15:44:39.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:39 smithi151 bash[23756]: audit 2024-03-04T15:44:38.547560+0000 mon.c (mon.2) 8 : audit 1 from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-03-04T15:44:40.163 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-04T15:44:40.163 INFO:teuthology.orchestra.run.smithi117.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-04T15:44:40.163 INFO:teuthology.orchestra.run.smithi117.stderr: stderr: 10+0 records in 2024-03-04T15:44:40.164 INFO:teuthology.orchestra.run.smithi117.stderr:10+0 records out 2024-03-04T15:44:40.164 INFO:teuthology.orchestra.run.smithi117.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0148916 s, 704 MB/s 2024-03-04T15:44:40.164 INFO:teuthology.orchestra.run.smithi117.stderr:--> Zapping successful for: 2024-03-04T15:44:40.833 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:40 smithi117 bash[21480]: cluster 2024-03-04T15:44:38.755306+0000 mgr.y (mgr.14152) 169 : cluster 0 pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:40.833 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:40 smithi117 bash[28843]: cluster 2024-03-04T15:44:38.755306+0000 mgr.y (mgr.14152) 169 : cluster 0 pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:40.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:40 smithi151 bash[23756]: cluster 2024-03-04T15:44:38.755306+0000 mgr.y (mgr.14152) 169 : cluster 0 pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 86 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:40.968 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi117:vg_nvme/lv_1 2024-03-04T15:44:42.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:42 smithi151 bash[23756]: cluster 2024-03-04T15:44:40.755872+0000 mgr.y (mgr.14152) 170 : cluster 0 pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-03-04T15:44:42.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:42 smithi151 bash[23756]: cephadm 2024-03-04T15:44:41.377052+0000 mgr.y (mgr.14152) 171 : cephadm 1 Detected new or changed devices on smithi117 2024-03-04T15:44:42.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:42 smithi151 bash[23756]: audit 2024-03-04T15:44:41.386187+0000 mon.a (mon.0) 403 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:42.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:42 smithi151 bash[23756]: audit 2024-03-04T15:44:41.396708+0000 mon.a (mon.0) 404 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:42.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:42 smithi151 bash[23756]: audit 2024-03-04T15:44:41.399455+0000 mon.a (mon.0) 405 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:44:42.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:42 smithi151 bash[23756]: audit 2024-03-04T15:44:41.401111+0000 mon.a (mon.0) 406 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:44:42.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:42 smithi151 bash[23756]: audit 2024-03-04T15:44:41.402699+0000 mon.a (mon.0) 407 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:44:42.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:42 smithi151 bash[23756]: cephadm 2024-03-04T15:44:41.403845+0000 mgr.y (mgr.14152) 172 : cephadm 1 Adjusting osd_memory_target on smithi117 to 5413M 2024-03-04T15:44:42.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:42 smithi151 bash[23756]: audit 2024-03-04T15:44:41.410076+0000 mon.a (mon.0) 408 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:42.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:42 smithi151 bash[23756]: audit 2024-03-04T15:44:41.414842+0000 mon.a (mon.0) 409 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:44:42.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:42 smithi151 bash[23756]: audit 2024-03-04T15:44:41.416472+0000 mon.a (mon.0) 410 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:44:42.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:42 smithi151 bash[23756]: audit 2024-03-04T15:44:41.426694+0000 mon.a (mon.0) 411 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:42.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[21480]: cluster 2024-03-04T15:44:40.755872+0000 mgr.y (mgr.14152) 170 : cluster 0 pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-03-04T15:44:42.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[21480]: cephadm 2024-03-04T15:44:41.377052+0000 mgr.y (mgr.14152) 171 : cephadm 1 Detected new or changed devices on smithi117 2024-03-04T15:44:42.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[21480]: audit 2024-03-04T15:44:41.386187+0000 mon.a (mon.0) 403 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:42.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[21480]: audit 2024-03-04T15:44:41.396708+0000 mon.a (mon.0) 404 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:42.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[21480]: audit 2024-03-04T15:44:41.399455+0000 mon.a (mon.0) 405 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:44:42.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[21480]: audit 2024-03-04T15:44:41.401111+0000 mon.a (mon.0) 406 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:44:42.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[21480]: audit 2024-03-04T15:44:41.402699+0000 mon.a (mon.0) 407 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:44:42.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[21480]: cephadm 2024-03-04T15:44:41.403845+0000 mgr.y (mgr.14152) 172 : cephadm 1 Adjusting osd_memory_target on smithi117 to 5413M 2024-03-04T15:44:42.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[21480]: audit 2024-03-04T15:44:41.410076+0000 mon.a (mon.0) 408 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:42.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[21480]: audit 2024-03-04T15:44:41.414842+0000 mon.a (mon.0) 409 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:44:42.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[21480]: audit 2024-03-04T15:44:41.416472+0000 mon.a (mon.0) 410 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:44:42.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[21480]: audit 2024-03-04T15:44:41.426694+0000 mon.a (mon.0) 411 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:42.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[28843]: cluster 2024-03-04T15:44:40.755872+0000 mgr.y (mgr.14152) 170 : cluster 0 pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 4.5 KiB/s rd, 196 KiB/s wr, 13 op/s 2024-03-04T15:44:42.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[28843]: cephadm 2024-03-04T15:44:41.377052+0000 mgr.y (mgr.14152) 171 : cephadm 1 Detected new or changed devices on smithi117 2024-03-04T15:44:42.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[28843]: audit 2024-03-04T15:44:41.386187+0000 mon.a (mon.0) 403 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:42.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[28843]: audit 2024-03-04T15:44:41.396708+0000 mon.a (mon.0) 404 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:42.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[28843]: audit 2024-03-04T15:44:41.399455+0000 mon.a (mon.0) 405 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:44:42.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[28843]: audit 2024-03-04T15:44:41.401111+0000 mon.a (mon.0) 406 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:44:42.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[28843]: audit 2024-03-04T15:44:41.402699+0000 mon.a (mon.0) 407 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:44:42.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[28843]: cephadm 2024-03-04T15:44:41.403845+0000 mgr.y (mgr.14152) 172 : cephadm 1 Adjusting osd_memory_target on smithi117 to 5413M 2024-03-04T15:44:42.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[28843]: audit 2024-03-04T15:44:41.410076+0000 mon.a (mon.0) 408 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:42.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[28843]: audit 2024-03-04T15:44:41.414842+0000 mon.a (mon.0) 409 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:44:42.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[28843]: audit 2024-03-04T15:44:41.416472+0000 mon.a (mon.0) 410 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:44:42.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:42 smithi117 bash[28843]: audit 2024-03-04T15:44:41.426694+0000 mon.a (mon.0) 411 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:44:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:44 smithi151 bash[23756]: cluster 2024-03-04T15:44:42.756472+0000 mgr.y (mgr.14152) 173 : cluster 0 pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-04T15:44:44.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:44 smithi117 bash[21480]: cluster 2024-03-04T15:44:42.756472+0000 mgr.y (mgr.14152) 173 : cluster 0 pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-04T15:44:44.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:44 smithi117 bash[28843]: cluster 2024-03-04T15:44:42.756472+0000 mgr.y (mgr.14152) 173 : cluster 0 pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 131 KiB/s wr, 9 op/s 2024-03-04T15:44:45.716 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:44:46.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:46 smithi151 bash[23756]: cluster 2024-03-04T15:44:44.757030+0000 mgr.y (mgr.14152) 174 : cluster 0 pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-04T15:44:46.685 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:46 smithi117 bash[21480]: cluster 2024-03-04T15:44:44.757030+0000 mgr.y (mgr.14152) 174 : cluster 0 pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-04T15:44:46.685 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:46 smithi117 bash[28843]: cluster 2024-03-04T15:44:44.757030+0000 mgr.y (mgr.14152) 174 : cluster 0 pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 2.2 KiB/s rd, 98 KiB/s wr, 6 op/s 2024-03-04T15:44:48.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.049+0000 7f91af9af700 1 -- 172.21.15.117:0/2646492806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a8107b50 msgr2=0x7f91a8109fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:44:48.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.049+0000 7f91af9af700 1 --2- 172.21.15.117:0/2646492806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a8107b50 0x7f91a8109fc0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f91a0009990 tx=0x7f91a0009ca0 comp rx=0 tx=0).stop 2024-03-04T15:44:48.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.049+0000 7f91af9af700 1 -- 172.21.15.117:0/2646492806 shutdown_connections 2024-03-04T15:44:48.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.049+0000 7f91af9af700 1 --2- 172.21.15.117:0/2646492806 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a8107b50 0x7f91a8109fc0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:44:48.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.049+0000 7f91af9af700 1 --2- 172.21.15.117:0/2646492806 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91a81028f0 0x7f91a8104ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:44:48.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.049+0000 7f91af9af700 1 --2- 172.21.15.117:0/2646492806 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a8105220 0x7f91a8107610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:44:48.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.049+0000 7f91af9af700 1 -- 172.21.15.117:0/2646492806 >> 172.21.15.117:0/2646492806 conn(0x7f91a80fc300 msgr2=0x7f91a80fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:44:48.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.049+0000 7f91af9af700 1 -- 172.21.15.117:0/2646492806 shutdown_connections 2024-03-04T15:44:48.056 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.049+0000 7f91af9af700 1 -- 172.21.15.117:0/2646492806 wait complete. 2024-03-04T15:44:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.049+0000 7f91af9af700 1 Processor -- start 2024-03-04T15:44:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.049+0000 7f91af9af700 1 -- start start 2024-03-04T15:44:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.049+0000 7f91af9af700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a81028f0 0x7f91a81996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:44:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.049+0000 7f91af9af700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a8105220 0x7f91a8199c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:44:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.049+0000 7f91af9af700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91a8107b50 0x7f91a81a0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:44:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91af9af700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a810ce90 con 0x7f91a81028f0 2024-03-04T15:44:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91af9af700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91a810cd10 con 0x7f91a8107b50 2024-03-04T15:44:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91af9af700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91a810d010 con 0x7f91a8105220 2024-03-04T15:44:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91acf4a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a8105220 0x7f91a8199c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:44:48.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91ad74b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a81028f0 0x7f91a81996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:44:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91acf4a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a8105220 0x7f91a8199c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:59806/0 (socket says 172.21.15.117:59806) 2024-03-04T15:44:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91acf4a700 1 -- 172.21.15.117:0/209705215 learned_addr learned my addr 172.21.15.117:0/209705215 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:44:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91adf4c700 1 --2- 172.21.15.117:0/209705215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91a8107b50 0x7f91a81a0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:44:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91acf4a700 1 -- 172.21.15.117:0/209705215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91a8107b50 msgr2=0x7f91a81a0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:44:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91acf4a700 1 --2- 172.21.15.117:0/209705215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91a8107b50 0x7f91a81a0c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:44:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91acf4a700 1 -- 172.21.15.117:0/209705215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a81028f0 msgr2=0x7f91a81996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:44:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91acf4a700 1 --2- 172.21.15.117:0/209705215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a81028f0 0x7f91a81996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:44:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91acf4a700 1 -- 172.21.15.117:0/209705215 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91a81a1330 con 0x7f91a8105220 2024-03-04T15:44:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91ad74b700 1 --2- 172.21.15.117:0/209705215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a81028f0 0x7f91a81996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:44:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91acf4a700 1 --2- 172.21.15.117:0/209705215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a8105220 0x7f91a8199c10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f919c00d980 tx=0x7f919c00dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:44:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91a67fc700 1 -- 172.21.15.117:0/209705215 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c016070 con 0x7f91a8105220 2024-03-04T15:44:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91af9af700 1 -- 172.21.15.117:0/209705215 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91a81a1620 con 0x7f91a8105220 2024-03-04T15:44:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91a67fc700 1 -- 172.21.15.117:0/209705215 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f919c00de70 con 0x7f91a8105220 2024-03-04T15:44:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91a67fc700 1 -- 172.21.15.117:0/209705215 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f919c010e00 con 0x7f91a8105220 2024-03-04T15:44:48.059 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91af9af700 1 -- 172.21.15.117:0/209705215 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91a81a1bb0 con 0x7f91a8105220 2024-03-04T15:44:48.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91af9af700 1 -- 172.21.15.117:0/209705215 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91a804faa0 con 0x7f91a8105220 2024-03-04T15:44:48.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91a67fc700 1 -- 172.21.15.117:0/209705215 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 15) v1 ==== 91397+0+0 (secure 0 0 0) 0x7f919c019070 con 0x7f91a8105220 2024-03-04T15:44:48.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91a67fc700 1 --2- 172.21.15.117:0/209705215 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f919006d690 0x7f919006fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:44:48.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91a67fc700 1 -- 172.21.15.117:0/209705215 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3017+0+0 (secure 0 0 0) 0x7f919c08c220 con 0x7f91a8105220 2024-03-04T15:44:48.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91ad74b700 1 --2- 172.21.15.117:0/209705215 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f919006d690 0x7f919006fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:44:48.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.053+0000 7f91ad74b700 1 --2- 172.21.15.117:0/209705215 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f919006d690 0x7f919006fb50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f919800c720 tx=0x7f9198009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:44:48.065 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.061+0000 7f91a67fc700 1 -- 172.21.15.117:0/209705215 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f919c05ac20 con 0x7f91a8105220 2024-03-04T15:44:48.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:44:48.277+0000 7f91af9af700 1 -- 172.21.15.117:0/209705215 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f91a802daf0 con 0x7f919006d690 2024-03-04T15:44:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:48 smithi151 bash[23756]: cluster 2024-03-04T15:44:46.757616+0000 mgr.y (mgr.14152) 175 : cluster 0 pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-04T15:44:48.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:48 smithi151 bash[23756]: audit 2024-03-04T15:44:48.289873+0000 mon.a (mon.0) 412 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:44:48.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:48 smithi151 bash[23756]: audit 2024-03-04T15:44:48.293098+0000 mon.a (mon.0) 413 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:44:48.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:48 smithi151 bash[23756]: audit 2024-03-04T15:44:48.294113+0000 mon.a (mon.0) 414 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:44:48.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:48 smithi117 bash[28843]: cluster 2024-03-04T15:44:46.757616+0000 mgr.y (mgr.14152) 175 : cluster 0 pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-04T15:44:48.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:48 smithi117 bash[28843]: audit 2024-03-04T15:44:48.289873+0000 mon.a (mon.0) 412 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:44:48.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:48 smithi117 bash[28843]: audit 2024-03-04T15:44:48.293098+0000 mon.a (mon.0) 413 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:44:48.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:48 smithi117 bash[28843]: audit 2024-03-04T15:44:48.294113+0000 mon.a (mon.0) 414 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:44:48.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:48 smithi117 bash[21480]: cluster 2024-03-04T15:44:46.757616+0000 mgr.y (mgr.14152) 175 : cluster 0 pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.8 KiB/s rd, 78 KiB/s wr, 5 op/s 2024-03-04T15:44:48.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:48 smithi117 bash[21480]: audit 2024-03-04T15:44:48.289873+0000 mon.a (mon.0) 412 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:44:48.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:48 smithi117 bash[21480]: audit 2024-03-04T15:44:48.293098+0000 mon.a (mon.0) 413 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:44:48.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:48 smithi117 bash[21480]: audit 2024-03-04T15:44:48.294113+0000 mon.a (mon.0) 414 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:44:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:49 smithi151 bash[23756]: audit 2024-03-04T15:44:48.283078+0000 mgr.y (mgr.14152) 176 : audit 0 from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:44:49.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:49 smithi117 bash[28843]: audit 2024-03-04T15:44:48.283078+0000 mgr.y (mgr.14152) 176 : audit 0 from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:44:49.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:49 smithi117 bash[21480]: audit 2024-03-04T15:44:48.283078+0000 mgr.y (mgr.14152) 176 : audit 0 from='client.24206 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi117:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:44:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:50 smithi151 bash[23756]: cluster 2024-03-04T15:44:48.758063+0000 mgr.y (mgr.14152) 177 : cluster 0 pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-04T15:44:50.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:50 smithi117 bash[21480]: cluster 2024-03-04T15:44:48.758063+0000 mgr.y (mgr.14152) 177 : cluster 0 pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-04T15:44:50.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:50 smithi117 bash[28843]: cluster 2024-03-04T15:44:48.758063+0000 mgr.y (mgr.14152) 177 : cluster 0 pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-04T15:44:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:52 smithi151 bash[23756]: cluster 2024-03-04T15:44:50.758621+0000 mgr.y (mgr.14152) 178 : cluster 0 pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-04T15:44:52.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:52 smithi117 bash[21480]: cluster 2024-03-04T15:44:50.758621+0000 mgr.y (mgr.14152) 178 : cluster 0 pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-04T15:44:52.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:52 smithi117 bash[28843]: cluster 2024-03-04T15:44:50.758621+0000 mgr.y (mgr.14152) 178 : cluster 0 pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 65 KiB/s wr, 4 op/s 2024-03-04T15:44:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:54 smithi151 bash[23756]: cluster 2024-03-04T15:44:52.759132+0000 mgr.y (mgr.14152) 179 : cluster 0 pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:54.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:54 smithi117 bash[28843]: cluster 2024-03-04T15:44:52.759132+0000 mgr.y (mgr.14152) 179 : cluster 0 pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:54.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:54 smithi117 bash[21480]: cluster 2024-03-04T15:44:52.759132+0000 mgr.y (mgr.14152) 179 : cluster 0 pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:56.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:56 smithi151 bash[23756]: cluster 2024-03-04T15:44:54.759626+0000 mgr.y (mgr.14152) 180 : cluster 0 pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:56.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:56 smithi117 bash[28843]: cluster 2024-03-04T15:44:54.759626+0000 mgr.y (mgr.14152) 180 : cluster 0 pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:56.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:56 smithi117 bash[21480]: cluster 2024-03-04T15:44:54.759626+0000 mgr.y (mgr.14152) 180 : cluster 0 pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:57 smithi151 bash[23756]: audit 2024-03-04T15:44:56.530150+0000 mon.c (mon.2) 9 : audit 1 from='client.? 172.21.15.117:0/702102446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e1fa658-d1a9-4129-ac7e-18f317b3b318"}]: dispatch 2024-03-04T15:44:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:57 smithi151 bash[23756]: audit 2024-03-04T15:44:56.530876+0000 mon.a (mon.0) 415 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e1fa658-d1a9-4129-ac7e-18f317b3b318"}]: dispatch 2024-03-04T15:44:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:57 smithi151 bash[23756]: audit 2024-03-04T15:44:56.535669+0000 mon.a (mon.0) 416 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e1fa658-d1a9-4129-ac7e-18f317b3b318"}]': finished 2024-03-04T15:44:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:57 smithi151 bash[23756]: cluster 2024-03-04T15:44:56.542509+0000 mon.a (mon.0) 417 : cluster 0 osdmap e22: 4 total, 3 up, 4 in 2024-03-04T15:44:57.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:57 smithi151 bash[23756]: audit 2024-03-04T15:44:56.542765+0000 mon.a (mon.0) 418 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:44:57.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:57 smithi151 bash[23756]: audit 2024-03-04T15:44:57.307242+0000 mon.a (mon.0) 419 : audit 0 from='client.? 172.21.15.117:0/2316617819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:44:57.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:57 smithi117 bash[28843]: audit 2024-03-04T15:44:56.530150+0000 mon.c (mon.2) 9 : audit 1 from='client.? 172.21.15.117:0/702102446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e1fa658-d1a9-4129-ac7e-18f317b3b318"}]: dispatch 2024-03-04T15:44:57.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:57 smithi117 bash[28843]: audit 2024-03-04T15:44:56.530876+0000 mon.a (mon.0) 415 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e1fa658-d1a9-4129-ac7e-18f317b3b318"}]: dispatch 2024-03-04T15:44:57.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:57 smithi117 bash[28843]: audit 2024-03-04T15:44:56.535669+0000 mon.a (mon.0) 416 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e1fa658-d1a9-4129-ac7e-18f317b3b318"}]': finished 2024-03-04T15:44:57.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:57 smithi117 bash[28843]: cluster 2024-03-04T15:44:56.542509+0000 mon.a (mon.0) 417 : cluster 0 osdmap e22: 4 total, 3 up, 4 in 2024-03-04T15:44:57.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:57 smithi117 bash[28843]: audit 2024-03-04T15:44:56.542765+0000 mon.a (mon.0) 418 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:44:57.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:57 smithi117 bash[28843]: audit 2024-03-04T15:44:57.307242+0000 mon.a (mon.0) 419 : audit 0 from='client.? 172.21.15.117:0/2316617819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:44:57.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:57 smithi117 bash[21480]: audit 2024-03-04T15:44:56.530150+0000 mon.c (mon.2) 9 : audit 1 from='client.? 172.21.15.117:0/702102446' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e1fa658-d1a9-4129-ac7e-18f317b3b318"}]: dispatch 2024-03-04T15:44:57.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:57 smithi117 bash[21480]: audit 2024-03-04T15:44:56.530876+0000 mon.a (mon.0) 415 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1e1fa658-d1a9-4129-ac7e-18f317b3b318"}]: dispatch 2024-03-04T15:44:57.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:57 smithi117 bash[21480]: audit 2024-03-04T15:44:56.535669+0000 mon.a (mon.0) 416 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1e1fa658-d1a9-4129-ac7e-18f317b3b318"}]': finished 2024-03-04T15:44:57.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:57 smithi117 bash[21480]: cluster 2024-03-04T15:44:56.542509+0000 mon.a (mon.0) 417 : cluster 0 osdmap e22: 4 total, 3 up, 4 in 2024-03-04T15:44:57.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:57 smithi117 bash[21480]: audit 2024-03-04T15:44:56.542765+0000 mon.a (mon.0) 418 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:44:57.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:57 smithi117 bash[21480]: audit 2024-03-04T15:44:57.307242+0000 mon.a (mon.0) 419 : audit 0 from='client.? 172.21.15.117:0/2316617819' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:44:58.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:44:58 smithi117 bash[21480]: cluster 2024-03-04T15:44:56.759998+0000 mgr.y (mgr.14152) 181 : cluster 0 pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:58.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:44:58 smithi117 bash[28843]: cluster 2024-03-04T15:44:56.759998+0000 mgr.y (mgr.14152) 181 : cluster 0 pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:44:58.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:44:58 smithi151 bash[23756]: cluster 2024-03-04T15:44:56.759998+0000 mgr.y (mgr.14152) 181 : cluster 0 pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:00.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:00 smithi117 bash[28843]: cluster 2024-03-04T15:44:58.760454+0000 mgr.y (mgr.14152) 182 : cluster 0 pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:00.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:00 smithi117 bash[21480]: cluster 2024-03-04T15:44:58.760454+0000 mgr.y (mgr.14152) 182 : cluster 0 pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:00 smithi151 bash[23756]: cluster 2024-03-04T15:44:58.760454+0000 mgr.y (mgr.14152) 182 : cluster 0 pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:02.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:02 smithi117 bash[28843]: cluster 2024-03-04T15:45:00.760972+0000 mgr.y (mgr.14152) 183 : cluster 0 pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:02.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:02 smithi117 bash[21480]: cluster 2024-03-04T15:45:00.760972+0000 mgr.y (mgr.14152) 183 : cluster 0 pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:02.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:02 smithi151 bash[23756]: cluster 2024-03-04T15:45:00.760972+0000 mgr.y (mgr.14152) 183 : cluster 0 pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:04.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:04 smithi117 bash[21480]: cluster 2024-03-04T15:45:02.761483+0000 mgr.y (mgr.14152) 184 : cluster 0 pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:04.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:04 smithi117 bash[28843]: cluster 2024-03-04T15:45:02.761483+0000 mgr.y (mgr.14152) 184 : cluster 0 pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:04.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:04 smithi151 bash[23756]: cluster 2024-03-04T15:45:02.761483+0000 mgr.y (mgr.14152) 184 : cluster 0 pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:06.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:06 smithi117 bash[21480]: cluster 2024-03-04T15:45:04.761966+0000 mgr.y (mgr.14152) 185 : cluster 0 pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:06.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:06 smithi117 bash[28843]: cluster 2024-03-04T15:45:04.761966+0000 mgr.y (mgr.14152) 185 : cluster 0 pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:06.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:06 smithi151 bash[23756]: cluster 2024-03-04T15:45:04.761966+0000 mgr.y (mgr.14152) 185 : cluster 0 pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:08.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:08 smithi117 bash[21480]: cluster 2024-03-04T15:45:06.762531+0000 mgr.y (mgr.14152) 186 : cluster 0 pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:08.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:08 smithi117 bash[28843]: cluster 2024-03-04T15:45:06.762531+0000 mgr.y (mgr.14152) 186 : cluster 0 pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:08.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:08 smithi151 bash[23756]: cluster 2024-03-04T15:45:06.762531+0000 mgr.y (mgr.14152) 186 : cluster 0 pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:09.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:09 smithi117 bash[21480]: audit 2024-03-04T15:45:08.894589+0000 mon.a (mon.0) 420 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-04T15:45:09.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:09 smithi117 bash[21480]: audit 2024-03-04T15:45:08.896391+0000 mon.a (mon.0) 421 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:45:09.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:09 smithi117 bash[28843]: audit 2024-03-04T15:45:08.894589+0000 mon.a (mon.0) 420 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-04T15:45:09.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:09 smithi117 bash[28843]: audit 2024-03-04T15:45:08.896391+0000 mon.a (mon.0) 421 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:45:09.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:09 smithi151 bash[23756]: audit 2024-03-04T15:45:08.894589+0000 mon.a (mon.0) 420 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-03-04T15:45:09.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:09 smithi151 bash[23756]: audit 2024-03-04T15:45:08.896391+0000 mon.a (mon.0) 421 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:45:10.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:10 smithi117 bash[28843]: cluster 2024-03-04T15:45:08.763057+0000 mgr.y (mgr.14152) 187 : cluster 0 pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:10.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:10 smithi117 bash[28843]: cephadm 2024-03-04T15:45:08.897935+0000 mgr.y (mgr.14152) 188 : cephadm 1 Deploying daemon osd.3 on smithi117 2024-03-04T15:45:10.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:10 smithi117 bash[21480]: cluster 2024-03-04T15:45:08.763057+0000 mgr.y (mgr.14152) 187 : cluster 0 pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:10.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:10 smithi117 bash[21480]: cephadm 2024-03-04T15:45:08.897935+0000 mgr.y (mgr.14152) 188 : cephadm 1 Deploying daemon osd.3 on smithi117 2024-03-04T15:45:10.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:10 smithi151 bash[23756]: cluster 2024-03-04T15:45:08.763057+0000 mgr.y (mgr.14152) 187 : cluster 0 pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:10.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:10 smithi151 bash[23756]: cephadm 2024-03-04T15:45:08.897935+0000 mgr.y (mgr.14152) 188 : cephadm 1 Deploying daemon osd.3 on smithi117 2024-03-04T15:45:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:12 smithi151 bash[23756]: cluster 2024-03-04T15:45:10.763600+0000 mgr.y (mgr.14152) 189 : cluster 0 pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:12.991 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:12 smithi117 bash[21480]: cluster 2024-03-04T15:45:10.763600+0000 mgr.y (mgr.14152) 189 : cluster 0 pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:12.991 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:12 smithi117 bash[28843]: cluster 2024-03-04T15:45:10.763600+0000 mgr.y (mgr.14152) 189 : cluster 0 pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:13.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:12 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:45:13.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:13 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:45:13.321 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:45:12 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:45:13.322 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:45:13 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:45:13.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:12 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:45:13.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:13 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:45:13.322 INFO:journalctl@ceph.osd.1.smithi117.stdout:Mar 04 15:45:12 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:45:13.322 INFO:journalctl@ceph.osd.1.smithi117.stdout:Mar 04 15:45:13 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:45:13.323 INFO:journalctl@ceph.osd.2.smithi117.stdout:Mar 04 15:45:12 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:45:13.323 INFO:journalctl@ceph.osd.2.smithi117.stdout:Mar 04 15:45:13 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:45:13.323 INFO:journalctl@ceph.osd.0.smithi117.stdout:Mar 04 15:45:12 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:45:13.323 INFO:journalctl@ceph.osd.0.smithi117.stdout:Mar 04 15:45:13 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:45:14.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:14 smithi151 bash[23756]: cluster 2024-03-04T15:45:12.764030+0000 mgr.y (mgr.14152) 190 : cluster 0 pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:14.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:14 smithi151 bash[23756]: audit 2024-03-04T15:45:13.377873+0000 mon.a (mon.0) 422 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:14.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:14 smithi151 bash[23756]: audit 2024-03-04T15:45:13.384887+0000 mon.a (mon.0) 423 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:14.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:14 smithi151 bash[23756]: audit 2024-03-04T15:45:13.390905+0000 mon.a (mon.0) 424 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:45:14.691 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:14 smithi117 bash[21480]: cluster 2024-03-04T15:45:12.764030+0000 mgr.y (mgr.14152) 190 : cluster 0 pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:14.691 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:14 smithi117 bash[21480]: audit 2024-03-04T15:45:13.377873+0000 mon.a (mon.0) 422 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:14.691 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:14 smithi117 bash[21480]: audit 2024-03-04T15:45:13.384887+0000 mon.a (mon.0) 423 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:14.691 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:14 smithi117 bash[21480]: audit 2024-03-04T15:45:13.390905+0000 mon.a (mon.0) 424 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:45:14.692 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:14 smithi117 bash[28843]: cluster 2024-03-04T15:45:12.764030+0000 mgr.y (mgr.14152) 190 : cluster 0 pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:14.692 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:14 smithi117 bash[28843]: audit 2024-03-04T15:45:13.377873+0000 mon.a (mon.0) 422 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:14.692 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:14 smithi117 bash[28843]: audit 2024-03-04T15:45:13.384887+0000 mon.a (mon.0) 423 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:14.692 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:14 smithi117 bash[28843]: audit 2024-03-04T15:45:13.390905+0000 mon.a (mon.0) 424 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:45:16.659 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:16 smithi117 bash[21480]: cluster 2024-03-04T15:45:14.764480+0000 mgr.y (mgr.14152) 191 : cluster 0 pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:16.659 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:16 smithi117 bash[28843]: cluster 2024-03-04T15:45:14.764480+0000 mgr.y (mgr.14152) 191 : cluster 0 pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:16.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:16 smithi151 bash[23756]: cluster 2024-03-04T15:45:14.764480+0000 mgr.y (mgr.14152) 191 : cluster 0 pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:18.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:18 smithi151 bash[23756]: cluster 2024-03-04T15:45:16.765065+0000 mgr.y (mgr.14152) 192 : cluster 0 pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:18.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:18 smithi117 bash[21480]: cluster 2024-03-04T15:45:16.765065+0000 mgr.y (mgr.14152) 192 : cluster 0 pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:18.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:18 smithi117 bash[28843]: cluster 2024-03-04T15:45:16.765065+0000 mgr.y (mgr.14152) 192 : cluster 0 pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:20.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:20 smithi151 bash[23756]: cluster 2024-03-04T15:45:18.765737+0000 mgr.y (mgr.14152) 193 : cluster 0 pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:20.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:20 smithi151 bash[23756]: audit 2024-03-04T15:45:19.561750+0000 mon.c (mon.2) 10 : audit 1 from='osd.3 [v2:172.21.15.117:6826/3320151079,v1:172.21.15.117:6827/3320151079]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-04T15:45:20.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:20 smithi151 bash[23756]: audit 2024-03-04T15:45:19.562139+0000 mon.a (mon.0) 425 : audit 1 from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-04T15:45:20.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:20 smithi151 bash[23756]: audit 2024-03-04T15:45:20.278903+0000 mon.a (mon.0) 426 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:20.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:20 smithi151 bash[23756]: audit 2024-03-04T15:45:20.286513+0000 mon.a (mon.0) 427 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:20.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:20 smithi151 bash[23756]: audit 2024-03-04T15:45:20.287537+0000 mon.a (mon.0) 428 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:45:20.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:20 smithi151 bash[23756]: audit 2024-03-04T15:45:20.288281+0000 mon.a (mon.0) 429 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:45:20.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:20 smithi151 bash[23756]: audit 2024-03-04T15:45:20.295685+0000 mon.a (mon.0) 430 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:20.719 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:20 smithi117 bash[21480]: cluster 2024-03-04T15:45:18.765737+0000 mgr.y (mgr.14152) 193 : cluster 0 pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:20.719 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:20 smithi117 bash[21480]: audit 2024-03-04T15:45:19.561750+0000 mon.c (mon.2) 10 : audit 1 from='osd.3 [v2:172.21.15.117:6826/3320151079,v1:172.21.15.117:6827/3320151079]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-04T15:45:20.719 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:20 smithi117 bash[21480]: audit 2024-03-04T15:45:19.562139+0000 mon.a (mon.0) 425 : audit 1 from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-04T15:45:20.719 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:20 smithi117 bash[21480]: audit 2024-03-04T15:45:20.278903+0000 mon.a (mon.0) 426 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:20.719 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:20 smithi117 bash[21480]: audit 2024-03-04T15:45:20.286513+0000 mon.a (mon.0) 427 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:20.719 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:20 smithi117 bash[21480]: audit 2024-03-04T15:45:20.287537+0000 mon.a (mon.0) 428 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:45:20.719 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:20 smithi117 bash[21480]: audit 2024-03-04T15:45:20.288281+0000 mon.a (mon.0) 429 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:45:20.719 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:20 smithi117 bash[21480]: audit 2024-03-04T15:45:20.295685+0000 mon.a (mon.0) 430 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:20.719 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:20 smithi117 bash[28843]: cluster 2024-03-04T15:45:18.765737+0000 mgr.y (mgr.14152) 193 : cluster 0 pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:20.719 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:20 smithi117 bash[28843]: audit 2024-03-04T15:45:19.561750+0000 mon.c (mon.2) 10 : audit 1 from='osd.3 [v2:172.21.15.117:6826/3320151079,v1:172.21.15.117:6827/3320151079]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-04T15:45:20.720 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:20 smithi117 bash[28843]: audit 2024-03-04T15:45:19.562139+0000 mon.a (mon.0) 425 : audit 1 from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-03-04T15:45:20.720 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:20 smithi117 bash[28843]: audit 2024-03-04T15:45:20.278903+0000 mon.a (mon.0) 426 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:20.720 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:20 smithi117 bash[28843]: audit 2024-03-04T15:45:20.286513+0000 mon.a (mon.0) 427 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:20.720 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:20 smithi117 bash[28843]: audit 2024-03-04T15:45:20.287537+0000 mon.a (mon.0) 428 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:45:20.720 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:20 smithi117 bash[28843]: audit 2024-03-04T15:45:20.288281+0000 mon.a (mon.0) 429 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:45:20.720 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:20 smithi117 bash[28843]: audit 2024-03-04T15:45:20.295685+0000 mon.a (mon.0) 430 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:21.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:21 smithi151 bash[23756]: audit 2024-03-04T15:45:20.400435+0000 mon.a (mon.0) 431 : audit 1 from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-04T15:45:21.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:21 smithi151 bash[23756]: cluster 2024-03-04T15:45:20.405714+0000 mon.a (mon.0) 432 : cluster 0 osdmap e23: 4 total, 3 up, 4 in 2024-03-04T15:45:21.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:21 smithi151 bash[23756]: audit 2024-03-04T15:45:20.406251+0000 mon.a (mon.0) 433 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:45:21.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:21 smithi151 bash[23756]: audit 2024-03-04T15:45:20.406821+0000 mon.c (mon.2) 11 : audit 1 from='osd.3 [v2:172.21.15.117:6826/3320151079,v1:172.21.15.117:6827/3320151079]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-04T15:45:21.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:21 smithi151 bash[23756]: audit 2024-03-04T15:45:20.407500+0000 mon.a (mon.0) 434 : audit 1 from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-04T15:45:21.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:21 smithi117 bash[28843]: audit 2024-03-04T15:45:20.400435+0000 mon.a (mon.0) 431 : audit 1 from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-04T15:45:21.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:21 smithi117 bash[28843]: cluster 2024-03-04T15:45:20.405714+0000 mon.a (mon.0) 432 : cluster 0 osdmap e23: 4 total, 3 up, 4 in 2024-03-04T15:45:21.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:21 smithi117 bash[28843]: audit 2024-03-04T15:45:20.406251+0000 mon.a (mon.0) 433 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:45:21.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:21 smithi117 bash[28843]: audit 2024-03-04T15:45:20.406821+0000 mon.c (mon.2) 11 : audit 1 from='osd.3 [v2:172.21.15.117:6826/3320151079,v1:172.21.15.117:6827/3320151079]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-04T15:45:21.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:21 smithi117 bash[28843]: audit 2024-03-04T15:45:20.407500+0000 mon.a (mon.0) 434 : audit 1 from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-04T15:45:21.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:21 smithi117 bash[21480]: audit 2024-03-04T15:45:20.400435+0000 mon.a (mon.0) 431 : audit 1 from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-03-04T15:45:21.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:21 smithi117 bash[21480]: cluster 2024-03-04T15:45:20.405714+0000 mon.a (mon.0) 432 : cluster 0 osdmap e23: 4 total, 3 up, 4 in 2024-03-04T15:45:21.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:21 smithi117 bash[21480]: audit 2024-03-04T15:45:20.406251+0000 mon.a (mon.0) 433 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:45:21.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:21 smithi117 bash[21480]: audit 2024-03-04T15:45:20.406821+0000 mon.c (mon.2) 11 : audit 1 from='osd.3 [v2:172.21.15.117:6826/3320151079,v1:172.21.15.117:6827/3320151079]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-04T15:45:21.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:21 smithi117 bash[21480]: audit 2024-03-04T15:45:20.407500+0000 mon.a (mon.0) 434 : audit 1 from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]: dispatch 2024-03-04T15:45:22.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:22 smithi151 bash[23756]: cluster 2024-03-04T15:45:20.766189+0000 mgr.y (mgr.14152) 194 : cluster 0 pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:22.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:22 smithi151 bash[23756]: audit 2024-03-04T15:45:21.410784+0000 mon.a (mon.0) 435 : audit 1 from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-03-04T15:45:22.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:22 smithi151 bash[23756]: cluster 2024-03-04T15:45:21.422613+0000 mon.a (mon.0) 436 : cluster 0 osdmap e24: 4 total, 3 up, 4 in 2024-03-04T15:45:22.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:22 smithi151 bash[23756]: audit 2024-03-04T15:45:21.423248+0000 mon.a (mon.0) 437 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:45:22.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:22 smithi151 bash[23756]: audit 2024-03-04T15:45:21.431151+0000 mon.a (mon.0) 438 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:45:22.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:22 smithi151 bash[23756]: audit 2024-03-04T15:45:21.948252+0000 mon.a (mon.0) 439 : audit 1 from='osd.3 ' entity='osd.3' 2024-03-04T15:45:22.726 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:22 smithi117 bash[28843]: cluster 2024-03-04T15:45:20.766189+0000 mgr.y (mgr.14152) 194 : cluster 0 pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:22.726 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:22 smithi117 bash[28843]: audit 2024-03-04T15:45:21.410784+0000 mon.a (mon.0) 435 : audit 1 from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-03-04T15:45:22.726 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:22 smithi117 bash[28843]: cluster 2024-03-04T15:45:21.422613+0000 mon.a (mon.0) 436 : cluster 0 osdmap e24: 4 total, 3 up, 4 in 2024-03-04T15:45:22.726 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:22 smithi117 bash[28843]: audit 2024-03-04T15:45:21.423248+0000 mon.a (mon.0) 437 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:45:22.726 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:22 smithi117 bash[28843]: audit 2024-03-04T15:45:21.431151+0000 mon.a (mon.0) 438 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:45:22.726 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:22 smithi117 bash[28843]: audit 2024-03-04T15:45:21.948252+0000 mon.a (mon.0) 439 : audit 1 from='osd.3 ' entity='osd.3' 2024-03-04T15:45:22.727 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:22 smithi117 bash[21480]: cluster 2024-03-04T15:45:20.766189+0000 mgr.y (mgr.14152) 194 : cluster 0 pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:22.727 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:22 smithi117 bash[21480]: audit 2024-03-04T15:45:21.410784+0000 mon.a (mon.0) 435 : audit 1 from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi117", "root=default"]}]': finished 2024-03-04T15:45:22.727 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:22 smithi117 bash[21480]: cluster 2024-03-04T15:45:21.422613+0000 mon.a (mon.0) 436 : cluster 0 osdmap e24: 4 total, 3 up, 4 in 2024-03-04T15:45:22.727 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:22 smithi117 bash[21480]: audit 2024-03-04T15:45:21.423248+0000 mon.a (mon.0) 437 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:45:22.727 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:22 smithi117 bash[21480]: audit 2024-03-04T15:45:21.431151+0000 mon.a (mon.0) 438 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:45:22.727 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:22 smithi117 bash[21480]: audit 2024-03-04T15:45:21.948252+0000 mon.a (mon.0) 439 : audit 1 from='osd.3 ' entity='osd.3' 2024-03-04T15:45:23.391 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:45:23.388+0000 7f91a67fc700 1 -- 172.21.15.117:0/209705215 <== mgr.14152 v2:172.21.15.117:6800/2865911351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f91a802daf0 con 0x7f919006d690 2024-03-04T15:45:23.391 INFO:teuthology.orchestra.run.smithi117.stdout:Created osd(s) 3 on host 'smithi117' 2024-03-04T15:45:23.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:45:23.392+0000 7f91af9af700 1 -- 172.21.15.117:0/209705215 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f919006d690 msgr2=0x7f919006fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:45:23.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:45:23.392+0000 7f91af9af700 1 --2- 172.21.15.117:0/209705215 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f919006d690 0x7f919006fb50 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f919800c720 tx=0x7f9198009040 comp rx=0 tx=0).stop 2024-03-04T15:45:23.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:45:23.392+0000 7f91af9af700 1 -- 172.21.15.117:0/209705215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a8105220 msgr2=0x7f91a8199c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:45:23.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:45:23.392+0000 7f91af9af700 1 --2- 172.21.15.117:0/209705215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a8105220 0x7f91a8199c10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f919c00d980 tx=0x7f919c00dc90 comp rx=0 tx=0).stop 2024-03-04T15:45:23.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:45:23.392+0000 7f91af9af700 1 -- 172.21.15.117:0/209705215 shutdown_connections 2024-03-04T15:45:23.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:45:23.392+0000 7f91af9af700 1 --2- 172.21.15.117:0/209705215 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f91a81028f0 0x7f91a81996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:45:23.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:45:23.392+0000 7f91af9af700 1 --2- 172.21.15.117:0/209705215 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f919006d690 0x7f919006fb50 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:45:23.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:45:23.392+0000 7f91af9af700 1 --2- 172.21.15.117:0/209705215 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f91a8107b50 0x7f91a81a0c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:45:23.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:45:23.392+0000 7f91af9af700 1 --2- 172.21.15.117:0/209705215 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f91a8105220 0x7f91a8199c10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:45:23.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:45:23.392+0000 7f91af9af700 1 -- 172.21.15.117:0/209705215 >> 172.21.15.117:0/209705215 conn(0x7f91a80fc300 msgr2=0x7f91a8103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:45:23.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:45:23.392+0000 7f91af9af700 1 -- 172.21.15.117:0/209705215 shutdown_connections 2024-03-04T15:45:23.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:45:23.392+0000 7f91af9af700 1 -- 172.21.15.117:0/209705215 wait complete. 2024-03-04T15:45:23.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:23 smithi151 bash[23756]: cluster 2024-03-04T15:45:20.549257+0000 osd.3 (osd.3) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:45:23.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:23 smithi151 bash[23756]: cluster 2024-03-04T15:45:20.549371+0000 osd.3 (osd.3) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:45:23.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:23 smithi151 bash[23756]: audit 2024-03-04T15:45:22.425024+0000 mon.a (mon.0) 440 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:45:23.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:23 smithi151 bash[23756]: cluster 2024-03-04T15:45:22.952940+0000 mon.a (mon.0) 441 : cluster 1 osd.3 [v2:172.21.15.117:6826/3320151079,v1:172.21.15.117:6827/3320151079] boot 2024-03-04T15:45:23.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:23 smithi151 bash[23756]: cluster 2024-03-04T15:45:22.953015+0000 mon.a (mon.0) 442 : cluster 0 osdmap e25: 4 total, 4 up, 4 in 2024-03-04T15:45:23.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:23 smithi151 bash[23756]: audit 2024-03-04T15:45:22.953386+0000 mon.a (mon.0) 443 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:45:23.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:23 smithi151 bash[23756]: audit 2024-03-04T15:45:23.367586+0000 mon.a (mon.0) 444 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:23.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:23 smithi151 bash[23756]: audit 2024-03-04T15:45:23.368575+0000 mon.a (mon.0) 445 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:45:23.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:23 smithi151 bash[23756]: audit 2024-03-04T15:45:23.378549+0000 mon.a (mon.0) 446 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:23.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:23 smithi117 bash[28843]: cluster 2024-03-04T15:45:20.549257+0000 osd.3 (osd.3) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:45:23.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:23 smithi117 bash[28843]: cluster 2024-03-04T15:45:20.549371+0000 osd.3 (osd.3) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:45:23.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:23 smithi117 bash[28843]: audit 2024-03-04T15:45:22.425024+0000 mon.a (mon.0) 440 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:45:23.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:23 smithi117 bash[28843]: cluster 2024-03-04T15:45:22.952940+0000 mon.a (mon.0) 441 : cluster 1 osd.3 [v2:172.21.15.117:6826/3320151079,v1:172.21.15.117:6827/3320151079] boot 2024-03-04T15:45:23.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:23 smithi117 bash[28843]: cluster 2024-03-04T15:45:22.953015+0000 mon.a (mon.0) 442 : cluster 0 osdmap e25: 4 total, 4 up, 4 in 2024-03-04T15:45:23.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:23 smithi117 bash[28843]: audit 2024-03-04T15:45:22.953386+0000 mon.a (mon.0) 443 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:45:23.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:23 smithi117 bash[28843]: audit 2024-03-04T15:45:23.367586+0000 mon.a (mon.0) 444 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:23.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:23 smithi117 bash[28843]: audit 2024-03-04T15:45:23.368575+0000 mon.a (mon.0) 445 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:45:23.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:23 smithi117 bash[28843]: audit 2024-03-04T15:45:23.378549+0000 mon.a (mon.0) 446 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:23.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:23 smithi117 bash[21480]: cluster 2024-03-04T15:45:20.549257+0000 osd.3 (osd.3) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:45:23.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:23 smithi117 bash[21480]: cluster 2024-03-04T15:45:20.549371+0000 osd.3 (osd.3) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:45:23.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:23 smithi117 bash[21480]: audit 2024-03-04T15:45:22.425024+0000 mon.a (mon.0) 440 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:45:23.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:23 smithi117 bash[21480]: cluster 2024-03-04T15:45:22.952940+0000 mon.a (mon.0) 441 : cluster 1 osd.3 [v2:172.21.15.117:6826/3320151079,v1:172.21.15.117:6827/3320151079] boot 2024-03-04T15:45:23.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:23 smithi117 bash[21480]: cluster 2024-03-04T15:45:22.953015+0000 mon.a (mon.0) 442 : cluster 0 osdmap e25: 4 total, 4 up, 4 in 2024-03-04T15:45:23.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:23 smithi117 bash[21480]: audit 2024-03-04T15:45:22.953386+0000 mon.a (mon.0) 443 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:45:23.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:23 smithi117 bash[21480]: audit 2024-03-04T15:45:23.367586+0000 mon.a (mon.0) 444 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:23.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:23 smithi117 bash[21480]: audit 2024-03-04T15:45:23.368575+0000 mon.a (mon.0) 445 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:45:23.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:23 smithi117 bash[21480]: audit 2024-03-04T15:45:23.378549+0000 mon.a (mon.0) 446 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:24.001 DEBUG:teuthology.orchestra.run.smithi117:osd.3> sudo journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.3.service 2024-03-04T15:45:24.004 INFO:tasks.cephadm:Deploying osd.4 on smithi151 with /dev/vg_nvme/lv_4... 2024-03-04T15:45:24.005 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-03-04T15:45:24.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:24 smithi117 bash[28843]: cluster 2024-03-04T15:45:22.766551+0000 mgr.y (mgr.14152) 195 : cluster 0 pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:24.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:24 smithi117 bash[21480]: cluster 2024-03-04T15:45:22.766551+0000 mgr.y (mgr.14152) 195 : cluster 0 pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:24.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:24 smithi151 bash[23756]: cluster 2024-03-04T15:45:22.766551+0000 mgr.y (mgr.14152) 195 : cluster 0 pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-03-04T15:45:25.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:25 smithi117 bash[21480]: cluster 2024-03-04T15:45:24.429819+0000 mon.a (mon.0) 447 : cluster 0 osdmap e26: 4 total, 4 up, 4 in 2024-03-04T15:45:25.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:25 smithi117 bash[28843]: cluster 2024-03-04T15:45:24.429819+0000 mon.a (mon.0) 447 : cluster 0 osdmap e26: 4 total, 4 up, 4 in 2024-03-04T15:45:25.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:25 smithi151 bash[23756]: cluster 2024-03-04T15:45:24.429819+0000 mon.a (mon.0) 447 : cluster 0 osdmap e26: 4 total, 4 up, 4 in 2024-03-04T15:45:26.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:26 smithi117 bash[21480]: cluster 2024-03-04T15:45:24.767050+0000 mgr.y (mgr.14152) 196 : cluster 0 pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:26.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:26 smithi117 bash[28843]: cluster 2024-03-04T15:45:24.767050+0000 mgr.y (mgr.14152) 196 : cluster 0 pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:26.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:26 smithi151 bash[23756]: cluster 2024-03-04T15:45:24.767050+0000 mgr.y (mgr.14152) 196 : cluster 0 pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:28.714 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T15:45:28.738 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:28 smithi151 bash[23756]: cluster 2024-03-04T15:45:26.767611+0000 mgr.y (mgr.14152) 197 : cluster 0 pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:28.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:28 smithi117 bash[21480]: cluster 2024-03-04T15:45:26.767611+0000 mgr.y (mgr.14152) 197 : cluster 0 pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:28.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:28 smithi117 bash[28843]: cluster 2024-03-04T15:45:26.767611+0000 mgr.y (mgr.14152) 197 : cluster 0 pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:30 smithi151 bash[23756]: cluster 2024-03-04T15:45:28.768096+0000 mgr.y (mgr.14152) 198 : cluster 0 pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:30.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:30 smithi117 bash[28843]: cluster 2024-03-04T15:45:28.768096+0000 mgr.y (mgr.14152) 198 : cluster 0 pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:30.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:30 smithi117 bash[21480]: cluster 2024-03-04T15:45:28.768096+0000 mgr.y (mgr.14152) 198 : cluster 0 pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:31.397 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-03-04T15:45:31.397 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-03-04T15:45:31.397 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-03-04T15:45:31.397 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-03-04T15:45:31.397 INFO:teuthology.orchestra.run.smithi151.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.01376 s, 762 MB/s 2024-03-04T15:45:31.397 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-03-04T15:45:31.982 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi151:vg_nvme/lv_4 2024-03-04T15:45:32.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:32 smithi117 bash[21480]: cluster 2024-03-04T15:45:30.768644+0000 mgr.y (mgr.14152) 199 : cluster 0 pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:32 smithi117 bash[28843]: cluster 2024-03-04T15:45:30.768644+0000 mgr.y (mgr.14152) 199 : cluster 0 pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:32.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:32 smithi151 bash[23756]: cluster 2024-03-04T15:45:30.768644+0000 mgr.y (mgr.14152) 199 : cluster 0 pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:33.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[21480]: cephadm 2024-03-04T15:45:32.510820+0000 mgr.y (mgr.14152) 200 : cephadm 1 Detected new or changed devices on smithi117 2024-03-04T15:45:33.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[21480]: audit 2024-03-04T15:45:32.520748+0000 mon.a (mon.0) 448 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:33.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[21480]: audit 2024-03-04T15:45:32.532048+0000 mon.a (mon.0) 449 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:33.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[21480]: audit 2024-03-04T15:45:32.534701+0000 mon.a (mon.0) 450 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:45:33.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[21480]: audit 2024-03-04T15:45:32.536661+0000 mon.a (mon.0) 451 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:45:33.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[21480]: audit 2024-03-04T15:45:32.538531+0000 mon.a (mon.0) 452 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:45:33.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[21480]: audit 2024-03-04T15:45:32.540221+0000 mon.a (mon.0) 453 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:45:33.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[21480]: cephadm 2024-03-04T15:45:32.541567+0000 mgr.y (mgr.14152) 201 : cephadm 1 Adjusting osd_memory_target on smithi117 to 4060M 2024-03-04T15:45:33.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[21480]: audit 2024-03-04T15:45:32.549691+0000 mon.a (mon.0) 454 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:33.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[21480]: audit 2024-03-04T15:45:32.554645+0000 mon.a (mon.0) 455 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:45:33.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[21480]: audit 2024-03-04T15:45:32.556202+0000 mon.a (mon.0) 456 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:45:33.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[21480]: audit 2024-03-04T15:45:32.566732+0000 mon.a (mon.0) 457 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:33.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[28843]: cephadm 2024-03-04T15:45:32.510820+0000 mgr.y (mgr.14152) 200 : cephadm 1 Detected new or changed devices on smithi117 2024-03-04T15:45:33.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[28843]: audit 2024-03-04T15:45:32.520748+0000 mon.a (mon.0) 448 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:33.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[28843]: audit 2024-03-04T15:45:32.532048+0000 mon.a (mon.0) 449 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:33.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[28843]: audit 2024-03-04T15:45:32.534701+0000 mon.a (mon.0) 450 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:45:33.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[28843]: audit 2024-03-04T15:45:32.536661+0000 mon.a (mon.0) 451 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:45:33.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[28843]: audit 2024-03-04T15:45:32.538531+0000 mon.a (mon.0) 452 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:45:33.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[28843]: audit 2024-03-04T15:45:32.540221+0000 mon.a (mon.0) 453 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:45:33.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[28843]: cephadm 2024-03-04T15:45:32.541567+0000 mgr.y (mgr.14152) 201 : cephadm 1 Adjusting osd_memory_target on smithi117 to 4060M 2024-03-04T15:45:33.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[28843]: audit 2024-03-04T15:45:32.549691+0000 mon.a (mon.0) 454 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:33.824 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[28843]: audit 2024-03-04T15:45:32.554645+0000 mon.a (mon.0) 455 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:45:33.824 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[28843]: audit 2024-03-04T15:45:32.556202+0000 mon.a (mon.0) 456 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:45:33.824 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:33 smithi117 bash[28843]: audit 2024-03-04T15:45:32.566732+0000 mon.a (mon.0) 457 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:33.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:33 smithi151 bash[23756]: cephadm 2024-03-04T15:45:32.510820+0000 mgr.y (mgr.14152) 200 : cephadm 1 Detected new or changed devices on smithi117 2024-03-04T15:45:33.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:33 smithi151 bash[23756]: audit 2024-03-04T15:45:32.520748+0000 mon.a (mon.0) 448 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:33.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:33 smithi151 bash[23756]: audit 2024-03-04T15:45:32.532048+0000 mon.a (mon.0) 449 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:33.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:33 smithi151 bash[23756]: audit 2024-03-04T15:45:32.534701+0000 mon.a (mon.0) 450 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:45:33.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:33 smithi151 bash[23756]: audit 2024-03-04T15:45:32.536661+0000 mon.a (mon.0) 451 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:45:33.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:33 smithi151 bash[23756]: audit 2024-03-04T15:45:32.538531+0000 mon.a (mon.0) 452 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:45:33.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:33 smithi151 bash[23756]: audit 2024-03-04T15:45:32.540221+0000 mon.a (mon.0) 453 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:45:33.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:33 smithi151 bash[23756]: cephadm 2024-03-04T15:45:32.541567+0000 mgr.y (mgr.14152) 201 : cephadm 1 Adjusting osd_memory_target on smithi117 to 4060M 2024-03-04T15:45:33.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:33 smithi151 bash[23756]: audit 2024-03-04T15:45:32.549691+0000 mon.a (mon.0) 454 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:33.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:33 smithi151 bash[23756]: audit 2024-03-04T15:45:32.554645+0000 mon.a (mon.0) 455 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:45:33.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:33 smithi151 bash[23756]: audit 2024-03-04T15:45:32.556202+0000 mon.a (mon.0) 456 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:45:33.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:33 smithi151 bash[23756]: audit 2024-03-04T15:45:32.566732+0000 mon.a (mon.0) 457 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:45:34.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:34 smithi117 bash[21480]: cluster 2024-03-04T15:45:32.769210+0000 mgr.y (mgr.14152) 202 : cluster 0 pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:34.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:34 smithi117 bash[28843]: cluster 2024-03-04T15:45:32.769210+0000 mgr.y (mgr.14152) 202 : cluster 0 pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:34.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:34 smithi151 bash[23756]: cluster 2024-03-04T15:45:32.769210+0000 mgr.y (mgr.14152) 202 : cluster 0 pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:36.688 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T15:45:36.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:36 smithi151 bash[23756]: cluster 2024-03-04T15:45:34.769730+0000 mgr.y (mgr.14152) 203 : cluster 0 pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:37.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:36 smithi117 bash[28843]: cluster 2024-03-04T15:45:34.769730+0000 mgr.y (mgr.14152) 203 : cluster 0 pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:37.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:36 smithi117 bash[21480]: cluster 2024-03-04T15:45:34.769730+0000 mgr.y (mgr.14152) 203 : cluster 0 pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:38.846 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:38 smithi151 bash[23756]: cluster 2024-03-04T15:45:36.770426+0000 mgr.y (mgr.14152) 204 : cluster 0 pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:38.996 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe421374700 1 -- 172.21.15.151:0/812750856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe41c074840 msgr2=0x7fe41c074cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:45:38.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe421374700 1 --2- 172.21.15.151:0/812750856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe41c074840 0x7fe41c074cc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe40c009990 tx=0x7fe40c009ca0 comp rx=0 tx=0).stop 2024-03-04T15:45:38.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe421374700 1 -- 172.21.15.151:0/812750856 shutdown_connections 2024-03-04T15:45:38.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe421374700 1 --2- 172.21.15.151:0/812750856 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe41c075e10 0x7fe41c074270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:45:38.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe421374700 1 --2- 172.21.15.151:0/812750856 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe41c109d40 0x7fe41c10c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:45:38.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe421374700 1 --2- 172.21.15.151:0/812750856 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe41c074840 0x7fe41c074cc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:45:38.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe421374700 1 -- 172.21.15.151:0/812750856 >> 172.21.15.151:0/812750856 conn(0x7fe41c0fd410 msgr2=0x7fe41c0ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:45:38.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe421374700 1 -- 172.21.15.151:0/812750856 shutdown_connections 2024-03-04T15:45:38.997 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe421374700 1 -- 172.21.15.151:0/812750856 wait complete. 2024-03-04T15:45:38.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe421374700 1 Processor -- start 2024-03-04T15:45:38.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe421374700 1 -- start start 2024-03-04T15:45:38.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe421374700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe41c074840 0x7fe41c199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:45:38.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe421374700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe41c075e10 0x7fe41c199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:45:38.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe421374700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe41c109d40 0x7fe41c1a0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:45:38.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe421374700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41c10f1d0 con 0x7fe41c074840 2024-03-04T15:45:38.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe421374700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe41c10f050 con 0x7fe41c109d40 2024-03-04T15:45:38.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe421374700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41c10f350 con 0x7fe41c075e10 2024-03-04T15:45:38.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe41b7fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe41c109d40 0x7fe41c1a0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:45:38.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe41b7fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe41c109d40 0x7fe41c1a0e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:60450/0 (socket says 172.21.15.151:60450) 2024-03-04T15:45:38.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe41b7fe700 1 -- 172.21.15.151:0/1790345401 learned_addr learned my addr 172.21.15.151:0/1790345401 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-04T15:45:38.999 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe41affd700 1 --2- 172.21.15.151:0/1790345401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe41c074840 0x7fe41c199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:45:39.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe41a7fc700 1 --2- 172.21.15.151:0/1790345401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe41c075e10 0x7fe41c199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:45:39.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe41b7fe700 1 -- 172.21.15.151:0/1790345401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe41c075e10 msgr2=0x7fe41c199d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:45:39.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe41b7fe700 1 --2- 172.21.15.151:0/1790345401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe41c075e10 0x7fe41c199d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:45:39.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe41b7fe700 1 -- 172.21.15.151:0/1790345401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe41c074840 msgr2=0x7fe41c199850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:45:39.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe41b7fe700 1 --2- 172.21.15.151:0/1790345401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe41c074840 0x7fe41c199850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:45:39.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe41b7fe700 1 -- 172.21.15.151:0/1790345401 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe41c1a14b0 con 0x7fe41c109d40 2024-03-04T15:45:39.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe41a7fc700 1 --2- 172.21.15.151:0/1790345401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe41c075e10 0x7fe41c199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T15:45:39.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe41b7fe700 1 --2- 172.21.15.151:0/1790345401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe41c109d40 0x7fe41c1a0e10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe414007ee0 tx=0x7fe41400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:45:39.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe403fff700 1 -- 172.21.15.151:0/1790345401 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe41400dbd0 con 0x7fe41c109d40 2024-03-04T15:45:39.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe403fff700 1 -- 172.21.15.151:0/1790345401 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe41400dd70 con 0x7fe41c109d40 2024-03-04T15:45:39.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe421374700 1 -- 172.21.15.151:0/1790345401 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe41c1a17a0 con 0x7fe41c109d40 2024-03-04T15:45:39.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe403fff700 1 -- 172.21.15.151:0/1790345401 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe414005890 con 0x7fe41c109d40 2024-03-04T15:45:39.000 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.991+0000 7fe421374700 1 -- 172.21.15.151:0/1790345401 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe41c1a1d30 con 0x7fe41c109d40 2024-03-04T15:45:39.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.995+0000 7fe421374700 1 -- 172.21.15.151:0/1790345401 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe41c04faa0 con 0x7fe41c109d40 2024-03-04T15:45:39.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.995+0000 7fe403fff700 1 -- 172.21.15.151:0/1790345401 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 15) v1 ==== 91397+0+0 (secure 0 0 0) 0x7fe41401a020 con 0x7fe41c109d40 2024-03-04T15:45:39.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.995+0000 7fe403fff700 1 --2- 172.21.15.151:0/1790345401 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fe40406d690 0x7fe40406fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:45:39.001 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.995+0000 7fe403fff700 1 -- 172.21.15.151:0/1790345401 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3449+0+0 (secure 0 0 0) 0x7fe41408c050 con 0x7fe41c109d40 2024-03-04T15:45:39.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.995+0000 7fe41affd700 1 --2- 172.21.15.151:0/1790345401 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fe40406d690 0x7fe40406fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:45:39.002 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.995+0000 7fe41affd700 1 --2- 172.21.15.151:0/1790345401 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fe40406d690 0x7fe40406fb50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe410005950 tx=0x7fe4100058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:45:39.005 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:38.999+0000 7fe403fff700 1 -- 172.21.15.151:0/1790345401 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe414010070 con 0x7fe41c109d40 2024-03-04T15:45:39.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:38 smithi117 bash[21480]: cluster 2024-03-04T15:45:36.770426+0000 mgr.y (mgr.14152) 204 : cluster 0 pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:39.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:38 smithi117 bash[28843]: cluster 2024-03-04T15:45:36.770426+0000 mgr.y (mgr.14152) 204 : cluster 0 pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:39.219 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:45:39.215+0000 7fe421374700 1 -- 172.21.15.151:0/1790345401 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fe41c062220 con 0x7fe40406d690 2024-03-04T15:45:39.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:39 smithi151 bash[23756]: audit 2024-03-04T15:45:39.228927+0000 mon.a (mon.0) 458 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:45:39.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:39 smithi151 bash[23756]: audit 2024-03-04T15:45:39.235718+0000 mon.a (mon.0) 459 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:45:39.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:39 smithi151 bash[23756]: audit 2024-03-04T15:45:39.237528+0000 mon.a (mon.0) 460 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:45:40.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:39 smithi117 bash[28843]: audit 2024-03-04T15:45:39.228927+0000 mon.a (mon.0) 458 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:45:40.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:39 smithi117 bash[28843]: audit 2024-03-04T15:45:39.235718+0000 mon.a (mon.0) 459 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:45:40.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:39 smithi117 bash[28843]: audit 2024-03-04T15:45:39.237528+0000 mon.a (mon.0) 460 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:45:40.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:39 smithi117 bash[21480]: audit 2024-03-04T15:45:39.228927+0000 mon.a (mon.0) 458 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:45:40.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:39 smithi117 bash[21480]: audit 2024-03-04T15:45:39.235718+0000 mon.a (mon.0) 459 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:45:40.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:39 smithi117 bash[21480]: audit 2024-03-04T15:45:39.237528+0000 mon.a (mon.0) 460 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:45:40.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:40 smithi151 bash[23756]: cluster 2024-03-04T15:45:38.771064+0000 mgr.y (mgr.14152) 205 : cluster 0 pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:40.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:40 smithi151 bash[23756]: audit 2024-03-04T15:45:39.221575+0000 mgr.y (mgr.14152) 206 : audit 0 from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:45:41.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:40 smithi117 bash[28843]: cluster 2024-03-04T15:45:38.771064+0000 mgr.y (mgr.14152) 205 : cluster 0 pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:41.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:40 smithi117 bash[28843]: audit 2024-03-04T15:45:39.221575+0000 mgr.y (mgr.14152) 206 : audit 0 from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:45:41.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:40 smithi117 bash[21480]: cluster 2024-03-04T15:45:38.771064+0000 mgr.y (mgr.14152) 205 : cluster 0 pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:41.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:40 smithi117 bash[21480]: audit 2024-03-04T15:45:39.221575+0000 mgr.y (mgr.14152) 206 : audit 0 from='client.24199 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:45:42.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:42 smithi151 bash[23756]: cluster 2024-03-04T15:45:40.771745+0000 mgr.y (mgr.14152) 207 : cluster 0 pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:43.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:42 smithi117 bash[21480]: cluster 2024-03-04T15:45:40.771745+0000 mgr.y (mgr.14152) 207 : cluster 0 pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:43.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:42 smithi117 bash[28843]: cluster 2024-03-04T15:45:40.771745+0000 mgr.y (mgr.14152) 207 : cluster 0 pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:44.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:44 smithi151 bash[23756]: cluster 2024-03-04T15:45:42.772365+0000 mgr.y (mgr.14152) 208 : cluster 0 pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:45.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:44 smithi117 bash[28843]: cluster 2024-03-04T15:45:42.772365+0000 mgr.y (mgr.14152) 208 : cluster 0 pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:45.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:44 smithi117 bash[21480]: cluster 2024-03-04T15:45:42.772365+0000 mgr.y (mgr.14152) 208 : cluster 0 pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:46.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:46 smithi151 bash[23756]: cluster 2024-03-04T15:45:44.772941+0000 mgr.y (mgr.14152) 209 : cluster 0 pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:47.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:46 smithi117 bash[21480]: cluster 2024-03-04T15:45:44.772941+0000 mgr.y (mgr.14152) 209 : cluster 0 pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:47.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:46 smithi117 bash[28843]: cluster 2024-03-04T15:45:44.772941+0000 mgr.y (mgr.14152) 209 : cluster 0 pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:48.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:48 smithi151 bash[23756]: cluster 2024-03-04T15:45:46.773518+0000 mgr.y (mgr.14152) 210 : cluster 0 pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:48.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:48 smithi151 bash[23756]: audit 2024-03-04T15:45:47.636859+0000 mon.b (mon.1) 10 : audit 1 from='client.? 172.21.15.151:0/2049957826' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35746d23-4625-45a9-81cd-c05f7ea5a98c"}]: dispatch 2024-03-04T15:45:48.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:48 smithi151 bash[23756]: audit 2024-03-04T15:45:47.637940+0000 mon.a (mon.0) 461 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35746d23-4625-45a9-81cd-c05f7ea5a98c"}]: dispatch 2024-03-04T15:45:48.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:48 smithi151 bash[23756]: audit 2024-03-04T15:45:47.641239+0000 mon.a (mon.0) 462 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35746d23-4625-45a9-81cd-c05f7ea5a98c"}]': finished 2024-03-04T15:45:48.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:48 smithi151 bash[23756]: cluster 2024-03-04T15:45:47.648146+0000 mon.a (mon.0) 463 : cluster 0 osdmap e27: 5 total, 4 up, 5 in 2024-03-04T15:45:48.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:48 smithi151 bash[23756]: audit 2024-03-04T15:45:47.648877+0000 mon.a (mon.0) 464 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:45:48.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:48 smithi151 bash[23756]: audit 2024-03-04T15:45:48.310707+0000 mon.b (mon.1) 11 : audit 0 from='client.? 172.21.15.151:0/1868146458' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:45:49.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:48 smithi117 bash[21480]: cluster 2024-03-04T15:45:46.773518+0000 mgr.y (mgr.14152) 210 : cluster 0 pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:49.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:48 smithi117 bash[21480]: audit 2024-03-04T15:45:47.636859+0000 mon.b (mon.1) 10 : audit 1 from='client.? 172.21.15.151:0/2049957826' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35746d23-4625-45a9-81cd-c05f7ea5a98c"}]: dispatch 2024-03-04T15:45:49.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:48 smithi117 bash[21480]: audit 2024-03-04T15:45:47.637940+0000 mon.a (mon.0) 461 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35746d23-4625-45a9-81cd-c05f7ea5a98c"}]: dispatch 2024-03-04T15:45:49.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:48 smithi117 bash[21480]: audit 2024-03-04T15:45:47.641239+0000 mon.a (mon.0) 462 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35746d23-4625-45a9-81cd-c05f7ea5a98c"}]': finished 2024-03-04T15:45:49.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:48 smithi117 bash[21480]: cluster 2024-03-04T15:45:47.648146+0000 mon.a (mon.0) 463 : cluster 0 osdmap e27: 5 total, 4 up, 5 in 2024-03-04T15:45:49.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:48 smithi117 bash[21480]: audit 2024-03-04T15:45:47.648877+0000 mon.a (mon.0) 464 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:45:49.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:48 smithi117 bash[21480]: audit 2024-03-04T15:45:48.310707+0000 mon.b (mon.1) 11 : audit 0 from='client.? 172.21.15.151:0/1868146458' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:45:49.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:48 smithi117 bash[28843]: cluster 2024-03-04T15:45:46.773518+0000 mgr.y (mgr.14152) 210 : cluster 0 pgmap v178: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:49.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:48 smithi117 bash[28843]: audit 2024-03-04T15:45:47.636859+0000 mon.b (mon.1) 10 : audit 1 from='client.? 172.21.15.151:0/2049957826' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35746d23-4625-45a9-81cd-c05f7ea5a98c"}]: dispatch 2024-03-04T15:45:49.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:48 smithi117 bash[28843]: audit 2024-03-04T15:45:47.637940+0000 mon.a (mon.0) 461 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "35746d23-4625-45a9-81cd-c05f7ea5a98c"}]: dispatch 2024-03-04T15:45:49.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:48 smithi117 bash[28843]: audit 2024-03-04T15:45:47.641239+0000 mon.a (mon.0) 462 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "35746d23-4625-45a9-81cd-c05f7ea5a98c"}]': finished 2024-03-04T15:45:49.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:48 smithi117 bash[28843]: cluster 2024-03-04T15:45:47.648146+0000 mon.a (mon.0) 463 : cluster 0 osdmap e27: 5 total, 4 up, 5 in 2024-03-04T15:45:49.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:48 smithi117 bash[28843]: audit 2024-03-04T15:45:47.648877+0000 mon.a (mon.0) 464 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:45:49.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:48 smithi117 bash[28843]: audit 2024-03-04T15:45:48.310707+0000 mon.b (mon.1) 11 : audit 0 from='client.? 172.21.15.151:0/1868146458' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:45:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:50 smithi151 bash[23756]: cluster 2024-03-04T15:45:48.774133+0000 mgr.y (mgr.14152) 211 : cluster 0 pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:51.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:50 smithi117 bash[21480]: cluster 2024-03-04T15:45:48.774133+0000 mgr.y (mgr.14152) 211 : cluster 0 pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:51.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:50 smithi117 bash[28843]: cluster 2024-03-04T15:45:48.774133+0000 mgr.y (mgr.14152) 211 : cluster 0 pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:52.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:52 smithi151 bash[23756]: cluster 2024-03-04T15:45:50.774812+0000 mgr.y (mgr.14152) 212 : cluster 0 pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:53.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:52 smithi117 bash[21480]: cluster 2024-03-04T15:45:50.774812+0000 mgr.y (mgr.14152) 212 : cluster 0 pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:53.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:52 smithi117 bash[28843]: cluster 2024-03-04T15:45:50.774812+0000 mgr.y (mgr.14152) 212 : cluster 0 pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:54.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:54 smithi151 bash[23756]: cluster 2024-03-04T15:45:52.775377+0000 mgr.y (mgr.14152) 213 : cluster 0 pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:55.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:54 smithi117 bash[21480]: cluster 2024-03-04T15:45:52.775377+0000 mgr.y (mgr.14152) 213 : cluster 0 pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:55.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:54 smithi117 bash[28843]: cluster 2024-03-04T15:45:52.775377+0000 mgr.y (mgr.14152) 213 : cluster 0 pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:56.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:56 smithi151 bash[23756]: cluster 2024-03-04T15:45:54.775878+0000 mgr.y (mgr.14152) 214 : cluster 0 pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:57.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:56 smithi117 bash[21480]: cluster 2024-03-04T15:45:54.775878+0000 mgr.y (mgr.14152) 214 : cluster 0 pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:57.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:56 smithi117 bash[28843]: cluster 2024-03-04T15:45:54.775878+0000 mgr.y (mgr.14152) 214 : cluster 0 pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:58.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:45:58 smithi151 bash[23756]: cluster 2024-03-04T15:45:56.776518+0000 mgr.y (mgr.14152) 215 : cluster 0 pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:59.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:45:58 smithi117 bash[21480]: cluster 2024-03-04T15:45:56.776518+0000 mgr.y (mgr.14152) 215 : cluster 0 pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:45:59.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:45:58 smithi117 bash[28843]: cluster 2024-03-04T15:45:56.776518+0000 mgr.y (mgr.14152) 215 : cluster 0 pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:00 smithi151 bash[23756]: cluster 2024-03-04T15:45:58.777204+0000 mgr.y (mgr.14152) 216 : cluster 0 pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:00 smithi151 bash[23756]: audit 2024-03-04T15:45:59.814473+0000 mon.a (mon.0) 465 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-04T15:46:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:00 smithi151 bash[23756]: audit 2024-03-04T15:45:59.816473+0000 mon.a (mon.0) 466 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:46:01.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:00 smithi117 bash[21480]: cluster 2024-03-04T15:45:58.777204+0000 mgr.y (mgr.14152) 216 : cluster 0 pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:01.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:00 smithi117 bash[21480]: audit 2024-03-04T15:45:59.814473+0000 mon.a (mon.0) 465 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-04T15:46:01.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:00 smithi117 bash[21480]: audit 2024-03-04T15:45:59.816473+0000 mon.a (mon.0) 466 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:46:01.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:00 smithi117 bash[28843]: cluster 2024-03-04T15:45:58.777204+0000 mgr.y (mgr.14152) 216 : cluster 0 pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:01.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:00 smithi117 bash[28843]: audit 2024-03-04T15:45:59.814473+0000 mon.a (mon.0) 465 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-03-04T15:46:01.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:00 smithi117 bash[28843]: audit 2024-03-04T15:45:59.816473+0000 mon.a (mon.0) 466 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:46:01.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:01 smithi151 bash[23756]: cephadm 2024-03-04T15:45:59.818141+0000 mgr.y (mgr.14152) 217 : cephadm 1 Deploying daemon osd.4 on smithi151 2024-03-04T15:46:02.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:01 smithi117 bash[28843]: cephadm 2024-03-04T15:45:59.818141+0000 mgr.y (mgr.14152) 217 : cephadm 1 Deploying daemon osd.4 on smithi151 2024-03-04T15:46:02.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:01 smithi117 bash[21480]: cephadm 2024-03-04T15:45:59.818141+0000 mgr.y (mgr.14152) 217 : cephadm 1 Deploying daemon osd.4 on smithi151 2024-03-04T15:46:02.774 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:02 smithi151 bash[23756]: cluster 2024-03-04T15:46:00.777741+0000 mgr.y (mgr.14152) 218 : cluster 0 pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:03.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:02 smithi117 bash[28843]: cluster 2024-03-04T15:46:00.777741+0000 mgr.y (mgr.14152) 218 : cluster 0 pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:03.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:02 smithi117 bash[21480]: cluster 2024-03-04T15:46:00.777741+0000 mgr.y (mgr.14152) 218 : cluster 0 pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:03.861 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:03 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:46:03.862 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:46:03 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:46:04.181 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:46:03 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:46:04.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:03 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:46:04.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:04 smithi151 bash[23756]: cluster 2024-03-04T15:46:02.778276+0000 mgr.y (mgr.14152) 219 : cluster 0 pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:04.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:04 smithi151 bash[23756]: audit 2024-03-04T15:46:04.005263+0000 mon.a (mon.0) 467 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:04.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:04 smithi151 bash[23756]: audit 2024-03-04T15:46:04.014007+0000 mon.a (mon.0) 468 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:04.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:04 smithi151 bash[23756]: audit 2024-03-04T15:46:04.038971+0000 mon.a (mon.0) 469 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:46:05.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:04 smithi117 bash[28843]: cluster 2024-03-04T15:46:02.778276+0000 mgr.y (mgr.14152) 219 : cluster 0 pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:05.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:04 smithi117 bash[28843]: audit 2024-03-04T15:46:04.005263+0000 mon.a (mon.0) 467 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:05.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:04 smithi117 bash[28843]: audit 2024-03-04T15:46:04.014007+0000 mon.a (mon.0) 468 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:05.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:04 smithi117 bash[28843]: audit 2024-03-04T15:46:04.038971+0000 mon.a (mon.0) 469 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:46:05.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:04 smithi117 bash[21480]: cluster 2024-03-04T15:46:02.778276+0000 mgr.y (mgr.14152) 219 : cluster 0 pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:05.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:04 smithi117 bash[21480]: audit 2024-03-04T15:46:04.005263+0000 mon.a (mon.0) 467 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:05.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:04 smithi117 bash[21480]: audit 2024-03-04T15:46:04.014007+0000 mon.a (mon.0) 468 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:05.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:04 smithi117 bash[21480]: audit 2024-03-04T15:46:04.038971+0000 mon.a (mon.0) 469 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:46:06.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:06 smithi151 bash[23756]: cluster 2024-03-04T15:46:04.778817+0000 mgr.y (mgr.14152) 220 : cluster 0 pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:07.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:06 smithi117 bash[21480]: cluster 2024-03-04T15:46:04.778817+0000 mgr.y (mgr.14152) 220 : cluster 0 pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:07.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:06 smithi117 bash[28843]: cluster 2024-03-04T15:46:04.778817+0000 mgr.y (mgr.14152) 220 : cluster 0 pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:08.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:08 smithi151 bash[23756]: cluster 2024-03-04T15:46:06.779363+0000 mgr.y (mgr.14152) 221 : cluster 0 pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:09.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:08 smithi117 bash[21480]: cluster 2024-03-04T15:46:06.779363+0000 mgr.y (mgr.14152) 221 : cluster 0 pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:09.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:08 smithi117 bash[28843]: cluster 2024-03-04T15:46:06.779363+0000 mgr.y (mgr.14152) 221 : cluster 0 pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:10.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:10 smithi151 bash[23756]: cluster 2024-03-04T15:46:08.779909+0000 mgr.y (mgr.14152) 222 : cluster 0 pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:10.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:10 smithi151 bash[23756]: audit 2024-03-04T15:46:10.097290+0000 mon.b (mon.1) 12 : audit 1 from='osd.4 [v2:172.21.15.151:6800/1293012079,v1:172.21.15.151:6801/1293012079]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T15:46:10.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:10 smithi151 bash[23756]: audit 2024-03-04T15:46:10.098396+0000 mon.a (mon.0) 470 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T15:46:11.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:10 smithi117 bash[21480]: cluster 2024-03-04T15:46:08.779909+0000 mgr.y (mgr.14152) 222 : cluster 0 pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:11.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:10 smithi117 bash[21480]: audit 2024-03-04T15:46:10.097290+0000 mon.b (mon.1) 12 : audit 1 from='osd.4 [v2:172.21.15.151:6800/1293012079,v1:172.21.15.151:6801/1293012079]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T15:46:11.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:10 smithi117 bash[21480]: audit 2024-03-04T15:46:10.098396+0000 mon.a (mon.0) 470 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T15:46:11.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:10 smithi117 bash[28843]: cluster 2024-03-04T15:46:08.779909+0000 mgr.y (mgr.14152) 222 : cluster 0 pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:11.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:10 smithi117 bash[28843]: audit 2024-03-04T15:46:10.097290+0000 mon.b (mon.1) 12 : audit 1 from='osd.4 [v2:172.21.15.151:6800/1293012079,v1:172.21.15.151:6801/1293012079]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T15:46:11.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:10 smithi117 bash[28843]: audit 2024-03-04T15:46:10.098396+0000 mon.a (mon.0) 470 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-03-04T15:46:11.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:11 smithi151 bash[23756]: audit 2024-03-04T15:46:10.652331+0000 mon.a (mon.0) 471 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-04T15:46:11.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:11 smithi151 bash[23756]: audit 2024-03-04T15:46:10.655403+0000 mon.b (mon.1) 13 : audit 1 from='osd.4 [v2:172.21.15.151:6800/1293012079,v1:172.21.15.151:6801/1293012079]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:46:11.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:11 smithi151 bash[23756]: cluster 2024-03-04T15:46:10.658226+0000 mon.a (mon.0) 472 : cluster 0 osdmap e28: 5 total, 4 up, 5 in 2024-03-04T15:46:11.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:11 smithi151 bash[23756]: audit 2024-03-04T15:46:10.659396+0000 mon.a (mon.0) 473 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:46:11.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:11 smithi151 bash[23756]: audit 2024-03-04T15:46:10.660155+0000 mon.a (mon.0) 474 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:46:11.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:11 smithi151 bash[23756]: audit 2024-03-04T15:46:10.748529+0000 mon.a (mon.0) 475 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:11.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:11 smithi151 bash[23756]: audit 2024-03-04T15:46:10.757065+0000 mon.a (mon.0) 476 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:11.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:11 smithi151 bash[23756]: audit 2024-03-04T15:46:11.432436+0000 mon.a (mon.0) 477 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:46:11.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:11 smithi151 bash[23756]: audit 2024-03-04T15:46:11.434276+0000 mon.a (mon.0) 478 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:46:11.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:11 smithi151 bash[23756]: audit 2024-03-04T15:46:11.446487+0000 mon.a (mon.0) 479 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:11.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:11 smithi151 bash[23756]: audit 2024-03-04T15:46:11.654936+0000 mon.a (mon.0) 480 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-03-04T15:46:12.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[21480]: audit 2024-03-04T15:46:10.652331+0000 mon.a (mon.0) 471 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-04T15:46:12.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[21480]: audit 2024-03-04T15:46:10.655403+0000 mon.b (mon.1) 13 : audit 1 from='osd.4 [v2:172.21.15.151:6800/1293012079,v1:172.21.15.151:6801/1293012079]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:46:12.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[21480]: cluster 2024-03-04T15:46:10.658226+0000 mon.a (mon.0) 472 : cluster 0 osdmap e28: 5 total, 4 up, 5 in 2024-03-04T15:46:12.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[21480]: audit 2024-03-04T15:46:10.659396+0000 mon.a (mon.0) 473 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:46:12.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[21480]: audit 2024-03-04T15:46:10.660155+0000 mon.a (mon.0) 474 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:46:12.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[21480]: audit 2024-03-04T15:46:10.748529+0000 mon.a (mon.0) 475 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:12.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[21480]: audit 2024-03-04T15:46:10.757065+0000 mon.a (mon.0) 476 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:12.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[21480]: audit 2024-03-04T15:46:11.432436+0000 mon.a (mon.0) 477 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:46:12.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[21480]: audit 2024-03-04T15:46:11.434276+0000 mon.a (mon.0) 478 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:46:12.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[21480]: audit 2024-03-04T15:46:11.446487+0000 mon.a (mon.0) 479 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:12.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[21480]: audit 2024-03-04T15:46:11.654936+0000 mon.a (mon.0) 480 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-03-04T15:46:12.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[28843]: audit 2024-03-04T15:46:10.652331+0000 mon.a (mon.0) 471 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-03-04T15:46:12.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[28843]: audit 2024-03-04T15:46:10.655403+0000 mon.b (mon.1) 13 : audit 1 from='osd.4 [v2:172.21.15.151:6800/1293012079,v1:172.21.15.151:6801/1293012079]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:46:12.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[28843]: cluster 2024-03-04T15:46:10.658226+0000 mon.a (mon.0) 472 : cluster 0 osdmap e28: 5 total, 4 up, 5 in 2024-03-04T15:46:12.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[28843]: audit 2024-03-04T15:46:10.659396+0000 mon.a (mon.0) 473 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:46:12.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[28843]: audit 2024-03-04T15:46:10.660155+0000 mon.a (mon.0) 474 : audit 1 from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:46:12.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[28843]: audit 2024-03-04T15:46:10.748529+0000 mon.a (mon.0) 475 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:12.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[28843]: audit 2024-03-04T15:46:10.757065+0000 mon.a (mon.0) 476 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:12.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[28843]: audit 2024-03-04T15:46:11.432436+0000 mon.a (mon.0) 477 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:46:12.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[28843]: audit 2024-03-04T15:46:11.434276+0000 mon.a (mon.0) 478 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:46:12.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[28843]: audit 2024-03-04T15:46:11.446487+0000 mon.a (mon.0) 479 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:12.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:11 smithi117 bash[28843]: audit 2024-03-04T15:46:11.654936+0000 mon.a (mon.0) 480 : audit 1 from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-03-04T15:46:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:12 smithi151 bash[23756]: cluster 2024-03-04T15:46:10.780441+0000 mgr.y (mgr.14152) 223 : cluster 0 pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:12 smithi151 bash[23756]: cluster 2024-03-04T15:46:11.662004+0000 mon.a (mon.0) 481 : cluster 0 osdmap e29: 5 total, 4 up, 5 in 2024-03-04T15:46:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:12 smithi151 bash[23756]: audit 2024-03-04T15:46:11.664542+0000 mon.a (mon.0) 482 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:46:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:12 smithi151 bash[23756]: audit 2024-03-04T15:46:11.675375+0000 mon.a (mon.0) 483 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:46:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:12 smithi151 bash[23756]: cluster 2024-03-04T15:46:11.758896+0000 mon.a (mon.0) 484 : cluster 0 osdmap e30: 5 total, 4 up, 5 in 2024-03-04T15:46:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:12 smithi151 bash[23756]: audit 2024-03-04T15:46:11.760152+0000 mon.a (mon.0) 485 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:46:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:12 smithi151 bash[23756]: audit 2024-03-04T15:46:12.161343+0000 mon.a (mon.0) 486 : audit 1 from='osd.4 ' entity='osd.4' 2024-03-04T15:46:13.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:12 smithi117 bash[21480]: cluster 2024-03-04T15:46:10.780441+0000 mgr.y (mgr.14152) 223 : cluster 0 pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:13.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:12 smithi117 bash[21480]: cluster 2024-03-04T15:46:11.662004+0000 mon.a (mon.0) 481 : cluster 0 osdmap e29: 5 total, 4 up, 5 in 2024-03-04T15:46:13.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:12 smithi117 bash[21480]: audit 2024-03-04T15:46:11.664542+0000 mon.a (mon.0) 482 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:46:13.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:12 smithi117 bash[21480]: audit 2024-03-04T15:46:11.675375+0000 mon.a (mon.0) 483 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:46:13.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:12 smithi117 bash[21480]: cluster 2024-03-04T15:46:11.758896+0000 mon.a (mon.0) 484 : cluster 0 osdmap e30: 5 total, 4 up, 5 in 2024-03-04T15:46:13.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:12 smithi117 bash[21480]: audit 2024-03-04T15:46:11.760152+0000 mon.a (mon.0) 485 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:46:13.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:12 smithi117 bash[21480]: audit 2024-03-04T15:46:12.161343+0000 mon.a (mon.0) 486 : audit 1 from='osd.4 ' entity='osd.4' 2024-03-04T15:46:13.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:12 smithi117 bash[28843]: cluster 2024-03-04T15:46:10.780441+0000 mgr.y (mgr.14152) 223 : cluster 0 pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:13.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:12 smithi117 bash[28843]: cluster 2024-03-04T15:46:11.662004+0000 mon.a (mon.0) 481 : cluster 0 osdmap e29: 5 total, 4 up, 5 in 2024-03-04T15:46:13.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:12 smithi117 bash[28843]: audit 2024-03-04T15:46:11.664542+0000 mon.a (mon.0) 482 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:46:13.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:12 smithi117 bash[28843]: audit 2024-03-04T15:46:11.675375+0000 mon.a (mon.0) 483 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:46:13.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:12 smithi117 bash[28843]: cluster 2024-03-04T15:46:11.758896+0000 mon.a (mon.0) 484 : cluster 0 osdmap e30: 5 total, 4 up, 5 in 2024-03-04T15:46:13.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:12 smithi117 bash[28843]: audit 2024-03-04T15:46:11.760152+0000 mon.a (mon.0) 485 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:46:13.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:12 smithi117 bash[28843]: audit 2024-03-04T15:46:12.161343+0000 mon.a (mon.0) 486 : audit 1 from='osd.4 ' entity='osd.4' 2024-03-04T15:46:13.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:13 smithi151 bash[23756]: cluster 2024-03-04T15:46:11.142475+0000 osd.4 (osd.4) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:46:13.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:13 smithi151 bash[23756]: cluster 2024-03-04T15:46:11.142614+0000 osd.4 (osd.4) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:46:13.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:13 smithi151 bash[23756]: audit 2024-03-04T15:46:12.675170+0000 mon.a (mon.0) 487 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:46:13.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:13 smithi151 bash[23756]: cluster 2024-03-04T15:46:12.807296+0000 mon.a (mon.0) 488 : cluster 1 osd.4 [v2:172.21.15.151:6800/1293012079,v1:172.21.15.151:6801/1293012079] boot 2024-03-04T15:46:13.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:13 smithi151 bash[23756]: cluster 2024-03-04T15:46:12.807428+0000 mon.a (mon.0) 489 : cluster 0 osdmap e31: 5 total, 5 up, 5 in 2024-03-04T15:46:13.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:13 smithi151 bash[23756]: audit 2024-03-04T15:46:12.807814+0000 mon.a (mon.0) 490 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:46:14.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:13 smithi117 bash[21480]: cluster 2024-03-04T15:46:11.142475+0000 osd.4 (osd.4) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:46:14.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:13 smithi117 bash[21480]: cluster 2024-03-04T15:46:11.142614+0000 osd.4 (osd.4) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:46:14.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:13 smithi117 bash[21480]: audit 2024-03-04T15:46:12.675170+0000 mon.a (mon.0) 487 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:46:14.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:13 smithi117 bash[21480]: cluster 2024-03-04T15:46:12.807296+0000 mon.a (mon.0) 488 : cluster 1 osd.4 [v2:172.21.15.151:6800/1293012079,v1:172.21.15.151:6801/1293012079] boot 2024-03-04T15:46:14.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:13 smithi117 bash[21480]: cluster 2024-03-04T15:46:12.807428+0000 mon.a (mon.0) 489 : cluster 0 osdmap e31: 5 total, 5 up, 5 in 2024-03-04T15:46:14.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:13 smithi117 bash[21480]: audit 2024-03-04T15:46:12.807814+0000 mon.a (mon.0) 490 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:46:14.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:13 smithi117 bash[28843]: cluster 2024-03-04T15:46:11.142475+0000 osd.4 (osd.4) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:46:14.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:13 smithi117 bash[28843]: cluster 2024-03-04T15:46:11.142614+0000 osd.4 (osd.4) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:46:14.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:13 smithi117 bash[28843]: audit 2024-03-04T15:46:12.675170+0000 mon.a (mon.0) 487 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:46:14.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:13 smithi117 bash[28843]: cluster 2024-03-04T15:46:12.807296+0000 mon.a (mon.0) 488 : cluster 1 osd.4 [v2:172.21.15.151:6800/1293012079,v1:172.21.15.151:6801/1293012079] boot 2024-03-04T15:46:14.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:13 smithi117 bash[28843]: cluster 2024-03-04T15:46:12.807428+0000 mon.a (mon.0) 489 : cluster 0 osdmap e31: 5 total, 5 up, 5 in 2024-03-04T15:46:14.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:13 smithi117 bash[28843]: audit 2024-03-04T15:46:12.807814+0000 mon.a (mon.0) 490 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:46:14.366 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:14.359+0000 7fe403fff700 1 -- 172.21.15.151:0/1790345401 <== mgr.14152 v2:172.21.15.117:6800/2865911351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe41c062220 con 0x7fe40406d690 2024-03-04T15:46:14.366 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 4 on host 'smithi151' 2024-03-04T15:46:14.368 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:14.363+0000 7fe421374700 1 -- 172.21.15.151:0/1790345401 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fe40406d690 msgr2=0x7fe40406fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:46:14.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:14.363+0000 7fe421374700 1 --2- 172.21.15.151:0/1790345401 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fe40406d690 0x7fe40406fb50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe410005950 tx=0x7fe4100058e0 comp rx=0 tx=0).stop 2024-03-04T15:46:14.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:14.363+0000 7fe421374700 1 -- 172.21.15.151:0/1790345401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe41c109d40 msgr2=0x7fe41c1a0e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:46:14.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:14.363+0000 7fe421374700 1 --2- 172.21.15.151:0/1790345401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe41c109d40 0x7fe41c1a0e10 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe414007ee0 tx=0x7fe41400d3b0 comp rx=0 tx=0).stop 2024-03-04T15:46:14.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:14.363+0000 7fe421374700 1 -- 172.21.15.151:0/1790345401 shutdown_connections 2024-03-04T15:46:14.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:14.363+0000 7fe421374700 1 --2- 172.21.15.151:0/1790345401 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe41c074840 0x7fe41c199850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:46:14.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:14.363+0000 7fe421374700 1 --2- 172.21.15.151:0/1790345401 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fe40406d690 0x7fe40406fb50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:46:14.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:14.363+0000 7fe421374700 1 --2- 172.21.15.151:0/1790345401 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe41c109d40 0x7fe41c1a0e10 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:46:14.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:14.363+0000 7fe421374700 1 --2- 172.21.15.151:0/1790345401 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe41c075e10 0x7fe41c199d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:46:14.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:14.363+0000 7fe421374700 1 -- 172.21.15.151:0/1790345401 >> 172.21.15.151:0/1790345401 conn(0x7fe41c0fd410 msgr2=0x7fe41c105c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:46:14.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:14.363+0000 7fe421374700 1 -- 172.21.15.151:0/1790345401 shutdown_connections 2024-03-04T15:46:14.369 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:14.363+0000 7fe421374700 1 -- 172.21.15.151:0/1790345401 wait complete. 2024-03-04T15:46:14.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:14 smithi151 bash[23756]: cluster 2024-03-04T15:46:12.780970+0000 mgr.y (mgr.14152) 224 : cluster 0 pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:14.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:14 smithi151 bash[23756]: cluster 2024-03-04T15:46:13.808933+0000 mon.a (mon.0) 491 : cluster 0 osdmap e32: 5 total, 5 up, 5 in 2024-03-04T15:46:14.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:14 smithi151 bash[23756]: audit 2024-03-04T15:46:14.336852+0000 mon.a (mon.0) 492 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:14.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:14 smithi151 bash[23756]: audit 2024-03-04T15:46:14.340516+0000 mon.a (mon.0) 493 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:46:14.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:14 smithi151 bash[23756]: audit 2024-03-04T15:46:14.346244+0000 mon.a (mon.0) 494 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:14.970 DEBUG:teuthology.orchestra.run.smithi151:osd.4> sudo journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.4.service 2024-03-04T15:46:14.972 INFO:tasks.cephadm:Deploying osd.5 on smithi151 with /dev/vg_nvme/lv_3... 2024-03-04T15:46:14.972 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-03-04T15:46:15.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:14 smithi117 bash[21480]: cluster 2024-03-04T15:46:12.780970+0000 mgr.y (mgr.14152) 224 : cluster 0 pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:15.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:14 smithi117 bash[21480]: cluster 2024-03-04T15:46:13.808933+0000 mon.a (mon.0) 491 : cluster 0 osdmap e32: 5 total, 5 up, 5 in 2024-03-04T15:46:15.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:14 smithi117 bash[21480]: audit 2024-03-04T15:46:14.336852+0000 mon.a (mon.0) 492 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:15.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:14 smithi117 bash[21480]: audit 2024-03-04T15:46:14.340516+0000 mon.a (mon.0) 493 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:46:15.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:14 smithi117 bash[21480]: audit 2024-03-04T15:46:14.346244+0000 mon.a (mon.0) 494 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:15.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:14 smithi117 bash[28843]: cluster 2024-03-04T15:46:12.780970+0000 mgr.y (mgr.14152) 224 : cluster 0 pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-03-04T15:46:15.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:14 smithi117 bash[28843]: cluster 2024-03-04T15:46:13.808933+0000 mon.a (mon.0) 491 : cluster 0 osdmap e32: 5 total, 5 up, 5 in 2024-03-04T15:46:15.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:14 smithi117 bash[28843]: audit 2024-03-04T15:46:14.336852+0000 mon.a (mon.0) 492 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:15.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:14 smithi117 bash[28843]: audit 2024-03-04T15:46:14.340516+0000 mon.a (mon.0) 493 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:46:15.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:14 smithi117 bash[28843]: audit 2024-03-04T15:46:14.346244+0000 mon.a (mon.0) 494 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:16.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:15 smithi117 bash[21480]: cluster 2024-03-04T15:46:14.781512+0000 mgr.y (mgr.14152) 225 : cluster 0 pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:16.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:15 smithi117 bash[21480]: cluster 2024-03-04T15:46:14.809478+0000 mon.a (mon.0) 495 : cluster 0 osdmap e33: 5 total, 5 up, 5 in 2024-03-04T15:46:16.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:15 smithi117 bash[28843]: cluster 2024-03-04T15:46:14.781512+0000 mgr.y (mgr.14152) 225 : cluster 0 pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:16.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:15 smithi117 bash[28843]: cluster 2024-03-04T15:46:14.809478+0000 mon.a (mon.0) 495 : cluster 0 osdmap e33: 5 total, 5 up, 5 in 2024-03-04T15:46:16.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:15 smithi151 bash[23756]: cluster 2024-03-04T15:46:14.781512+0000 mgr.y (mgr.14152) 225 : cluster 0 pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:16.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:15 smithi151 bash[23756]: cluster 2024-03-04T15:46:14.809478+0000 mon.a (mon.0) 495 : cluster 0 osdmap e33: 5 total, 5 up, 5 in 2024-03-04T15:46:17.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:16 smithi151 bash[23756]: cluster 2024-03-04T15:46:15.812081+0000 mon.a (mon.0) 496 : cluster 0 osdmap e34: 5 total, 5 up, 5 in 2024-03-04T15:46:17.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:16 smithi117 bash[21480]: cluster 2024-03-04T15:46:15.812081+0000 mon.a (mon.0) 496 : cluster 0 osdmap e34: 5 total, 5 up, 5 in 2024-03-04T15:46:17.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:16 smithi117 bash[28843]: cluster 2024-03-04T15:46:15.812081+0000 mon.a (mon.0) 496 : cluster 0 osdmap e34: 5 total, 5 up, 5 in 2024-03-04T15:46:18.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:17 smithi151 bash[23756]: cluster 2024-03-04T15:46:16.782026+0000 mgr.y (mgr.14152) 226 : cluster 0 pgmap v201: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T15:46:18.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:17 smithi151 bash[23756]: cluster 2024-03-04T15:46:16.832621+0000 mon.a (mon.0) 497 : cluster 0 osdmap e35: 5 total, 5 up, 5 in 2024-03-04T15:46:18.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:17 smithi117 bash[21480]: cluster 2024-03-04T15:46:16.782026+0000 mgr.y (mgr.14152) 226 : cluster 0 pgmap v201: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T15:46:18.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:17 smithi117 bash[21480]: cluster 2024-03-04T15:46:16.832621+0000 mon.a (mon.0) 497 : cluster 0 osdmap e35: 5 total, 5 up, 5 in 2024-03-04T15:46:18.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:17 smithi117 bash[28843]: cluster 2024-03-04T15:46:16.782026+0000 mgr.y (mgr.14152) 226 : cluster 0 pgmap v201: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-03-04T15:46:18.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:17 smithi117 bash[28843]: cluster 2024-03-04T15:46:16.832621+0000 mon.a (mon.0) 497 : cluster 0 osdmap e35: 5 total, 5 up, 5 in 2024-03-04T15:46:19.315 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T15:46:20.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:19 smithi151 bash[23756]: cluster 2024-03-04T15:46:18.782670+0000 mgr.y (mgr.14152) 227 : cluster 0 pgmap v203: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-04T15:46:20.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:19 smithi117 bash[21480]: cluster 2024-03-04T15:46:18.782670+0000 mgr.y (mgr.14152) 227 : cluster 0 pgmap v203: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-04T15:46:20.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:19 smithi117 bash[28843]: cluster 2024-03-04T15:46:18.782670+0000 mgr.y (mgr.14152) 227 : cluster 0 pgmap v203: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 152 MiB used, 447 GiB / 447 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-03-04T15:46:21.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:21 smithi151 bash[23756]: cluster 2024-03-04T15:46:20.783281+0000 mgr.y (mgr.14152) 228 : cluster 0 pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-04T15:46:22.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:21 smithi117 bash[21480]: cluster 2024-03-04T15:46:20.783281+0000 mgr.y (mgr.14152) 228 : cluster 0 pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-04T15:46:22.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:21 smithi117 bash[28843]: cluster 2024-03-04T15:46:20.783281+0000 mgr.y (mgr.14152) 228 : cluster 0 pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-04T15:46:23.493 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-03-04T15:46:23.493 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-03-04T15:46:23.493 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-03-04T15:46:23.493 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-03-04T15:46:23.493 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0131587 s, 797 MB/s 2024-03-04T15:46:23.494 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-03-04T15:46:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:23 smithi151 bash[23756]: cluster 2024-03-04T15:46:22.783790+0000 mgr.y (mgr.14152) 229 : cluster 0 pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T15:46:24.291 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi151:vg_nvme/lv_3 2024-03-04T15:46:24.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:23 smithi117 bash[21480]: cluster 2024-03-04T15:46:22.783790+0000 mgr.y (mgr.14152) 229 : cluster 0 pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T15:46:24.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:23 smithi117 bash[28843]: cluster 2024-03-04T15:46:22.783790+0000 mgr.y (mgr.14152) 229 : cluster 0 pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T15:46:26.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:25 smithi117 bash[21480]: cephadm 2024-03-04T15:46:24.694100+0000 mgr.y (mgr.14152) 230 : cephadm 1 Detected new or changed devices on smithi151 2024-03-04T15:46:26.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:25 smithi117 bash[21480]: audit 2024-03-04T15:46:24.704800+0000 mon.a (mon.0) 498 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:26.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:25 smithi117 bash[21480]: audit 2024-03-04T15:46:24.713628+0000 mon.a (mon.0) 499 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:26.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:25 smithi117 bash[21480]: audit 2024-03-04T15:46:24.715696+0000 mon.a (mon.0) 500 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:46:26.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:25 smithi117 bash[21480]: cephadm 2024-03-04T15:46:24.717209+0000 mgr.y (mgr.14152) 231 : cephadm 1 Adjusting osd_memory_target on smithi151 to 17265M 2024-03-04T15:46:26.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:25 smithi117 bash[21480]: audit 2024-03-04T15:46:24.723884+0000 mon.a (mon.0) 501 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:26.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:25 smithi117 bash[21480]: audit 2024-03-04T15:46:24.728154+0000 mon.a (mon.0) 502 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:46:26.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:25 smithi117 bash[21480]: audit 2024-03-04T15:46:24.729790+0000 mon.a (mon.0) 503 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:46:26.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:25 smithi117 bash[21480]: audit 2024-03-04T15:46:24.739824+0000 mon.a (mon.0) 504 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:26.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:25 smithi117 bash[28843]: cephadm 2024-03-04T15:46:24.694100+0000 mgr.y (mgr.14152) 230 : cephadm 1 Detected new or changed devices on smithi151 2024-03-04T15:46:26.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:25 smithi117 bash[28843]: audit 2024-03-04T15:46:24.704800+0000 mon.a (mon.0) 498 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:26.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:25 smithi117 bash[28843]: audit 2024-03-04T15:46:24.713628+0000 mon.a (mon.0) 499 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:26.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:25 smithi117 bash[28843]: audit 2024-03-04T15:46:24.715696+0000 mon.a (mon.0) 500 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:46:26.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:25 smithi117 bash[28843]: cephadm 2024-03-04T15:46:24.717209+0000 mgr.y (mgr.14152) 231 : cephadm 1 Adjusting osd_memory_target on smithi151 to 17265M 2024-03-04T15:46:26.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:25 smithi117 bash[28843]: audit 2024-03-04T15:46:24.723884+0000 mon.a (mon.0) 501 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:26.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:25 smithi117 bash[28843]: audit 2024-03-04T15:46:24.728154+0000 mon.a (mon.0) 502 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:46:26.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:25 smithi117 bash[28843]: audit 2024-03-04T15:46:24.729790+0000 mon.a (mon.0) 503 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:46:26.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:25 smithi117 bash[28843]: audit 2024-03-04T15:46:24.739824+0000 mon.a (mon.0) 504 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:25 smithi151 bash[23756]: cephadm 2024-03-04T15:46:24.694100+0000 mgr.y (mgr.14152) 230 : cephadm 1 Detected new or changed devices on smithi151 2024-03-04T15:46:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:25 smithi151 bash[23756]: audit 2024-03-04T15:46:24.704800+0000 mon.a (mon.0) 498 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:25 smithi151 bash[23756]: audit 2024-03-04T15:46:24.713628+0000 mon.a (mon.0) 499 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:25 smithi151 bash[23756]: audit 2024-03-04T15:46:24.715696+0000 mon.a (mon.0) 500 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:46:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:25 smithi151 bash[23756]: cephadm 2024-03-04T15:46:24.717209+0000 mgr.y (mgr.14152) 231 : cephadm 1 Adjusting osd_memory_target on smithi151 to 17265M 2024-03-04T15:46:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:25 smithi151 bash[23756]: audit 2024-03-04T15:46:24.723884+0000 mon.a (mon.0) 501 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:25 smithi151 bash[23756]: audit 2024-03-04T15:46:24.728154+0000 mon.a (mon.0) 502 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:46:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:25 smithi151 bash[23756]: audit 2024-03-04T15:46:24.729790+0000 mon.a (mon.0) 503 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:46:26.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:25 smithi151 bash[23756]: audit 2024-03-04T15:46:24.739824+0000 mon.a (mon.0) 504 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:27.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:26 smithi117 bash[21480]: cluster 2024-03-04T15:46:24.784507+0000 mgr.y (mgr.14152) 232 : cluster 0 pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:27.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:26 smithi117 bash[28843]: cluster 2024-03-04T15:46:24.784507+0000 mgr.y (mgr.14152) 232 : cluster 0 pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:26 smithi151 bash[23756]: cluster 2024-03-04T15:46:24.784507+0000 mgr.y (mgr.14152) 232 : cluster 0 pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:29.000 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T15:46:29.021 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:28 smithi151 bash[23756]: cluster 2024-03-04T15:46:26.785202+0000 mgr.y (mgr.14152) 233 : cluster 0 pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:29.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:28 smithi117 bash[21480]: cluster 2024-03-04T15:46:26.785202+0000 mgr.y (mgr.14152) 233 : cluster 0 pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:29.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:28 smithi117 bash[28843]: cluster 2024-03-04T15:46:26.785202+0000 mgr.y (mgr.14152) 233 : cluster 0 pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:30.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:29 smithi117 bash[21480]: cluster 2024-03-04T15:46:28.785705+0000 mgr.y (mgr.14152) 234 : cluster 0 pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:30.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:29 smithi117 bash[28843]: cluster 2024-03-04T15:46:28.785705+0000 mgr.y (mgr.14152) 234 : cluster 0 pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:30.125 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:29 smithi151 bash[23756]: cluster 2024-03-04T15:46:28.785705+0000 mgr.y (mgr.14152) 234 : cluster 0 pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:31.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.519+0000 7fc09baad700 1 -- 172.21.15.151:0/1452284617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc094105c30 msgr2=0x7fc09410c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:46:31.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.519+0000 7fc09baad700 1 --2- 172.21.15.151:0/1452284617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc094105c30 0x7fc09410c4e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fc08c00b0d0 tx=0x7fc08c00b3e0 comp rx=0 tx=0).stop 2024-03-04T15:46:31.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.519+0000 7fc09baad700 1 -- 172.21.15.151:0/1452284617 shutdown_connections 2024-03-04T15:46:31.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.519+0000 7fc09baad700 1 --2- 172.21.15.151:0/1452284617 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc094105c30 0x7fc09410c4e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:46:31.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.519+0000 7fc09baad700 1 --2- 172.21.15.151:0/1452284617 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc094104090 0x7fc0941044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:46:31.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.519+0000 7fc09baad700 1 --2- 172.21.15.151:0/1452284617 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc094105290 0x7fc0941056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:46:31.522 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.519+0000 7fc09baad700 1 -- 172.21.15.151:0/1452284617 >> 172.21.15.151:0/1452284617 conn(0x7fc0940ff670 msgr2=0x7fc094101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:46:31.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.519+0000 7fc09baad700 1 -- 172.21.15.151:0/1452284617 shutdown_connections 2024-03-04T15:46:31.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.519+0000 7fc09baad700 1 -- 172.21.15.151:0/1452284617 wait complete. 2024-03-04T15:46:31.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.519+0000 7fc09baad700 1 Processor -- start 2024-03-04T15:46:31.523 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.519+0000 7fc09baad700 1 -- start start 2024-03-04T15:46:31.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.519+0000 7fc09baad700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc094104090 0x7fc0941999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:46:31.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.519+0000 7fc09baad700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc094105290 0x7fc094199ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:46:31.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.519+0000 7fc09baad700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc094105c30 0x7fc0941a0f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:46:31.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.519+0000 7fc09baad700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09410f1e0 con 0x7fc094105290 2024-03-04T15:46:31.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.519+0000 7fc09baad700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09410f060 con 0x7fc094105c30 2024-03-04T15:46:31.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.519+0000 7fc09baad700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc09410f360 con 0x7fc094104090 2024-03-04T15:46:31.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc09a04a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc094105c30 0x7fc0941a0f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:46:31.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc09a04a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc094105c30 0x7fc0941a0f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:33208/0 (socket says 172.21.15.151:33208) 2024-03-04T15:46:31.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc09a04a700 1 -- 172.21.15.151:0/3464096426 learned_addr learned my addr 172.21.15.151:0/3464096426 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-04T15:46:31.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc099849700 1 --2- 172.21.15.151:0/3464096426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc094104090 0x7fc0941999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:46:31.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc099048700 1 --2- 172.21.15.151:0/3464096426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc094105290 0x7fc094199ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:46:31.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc09a04a700 1 -- 172.21.15.151:0/3464096426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc094104090 msgr2=0x7fc0941999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:46:31.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc09a04a700 1 --2- 172.21.15.151:0/3464096426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc094104090 0x7fc0941999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:46:31.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc09a04a700 1 -- 172.21.15.151:0/3464096426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc094105290 msgr2=0x7fc094199ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:46:31.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc09a04a700 1 --2- 172.21.15.151:0/3464096426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc094105290 0x7fc094199ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:46:31.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc09a04a700 1 -- 172.21.15.151:0/3464096426 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0941a15b0 con 0x7fc094105c30 2024-03-04T15:46:31.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc09a04a700 1 --2- 172.21.15.151:0/3464096426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc094105c30 0x7fc0941a0f10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc08c007d90 tx=0x7fc08c005b80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:46:31.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc092ffd700 1 -- 172.21.15.151:0/3464096426 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c022070 con 0x7fc094105c30 2024-03-04T15:46:31.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc092ffd700 1 -- 172.21.15.151:0/3464096426 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fc08c013d10 con 0x7fc094105c30 2024-03-04T15:46:31.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc09baad700 1 -- 172.21.15.151:0/3464096426 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0941a1840 con 0x7fc094105c30 2024-03-04T15:46:31.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc092ffd700 1 -- 172.21.15.151:0/3464096426 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c01bd40 con 0x7fc094105c30 2024-03-04T15:46:31.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc09baad700 1 -- 172.21.15.151:0/3464096426 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0941a1d70 con 0x7fc094105c30 2024-03-04T15:46:31.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc09baad700 1 -- 172.21.15.151:0/3464096426 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc09404faa0 con 0x7fc094105c30 2024-03-04T15:46:31.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc092ffd700 1 -- 172.21.15.151:0/3464096426 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 15) v1 ==== 91397+0+0 (secure 0 0 0) 0x7fc08c025070 con 0x7fc094105c30 2024-03-04T15:46:31.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc092ffd700 1 --2- 172.21.15.151:0/3464096426 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fc07c06d690 0x7fc07c06fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:46:31.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc092ffd700 1 -- 172.21.15.151:0/3464096426 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 3991+0+0 (secure 0 0 0) 0x7fc08c00f020 con 0x7fc094105c30 2024-03-04T15:46:31.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.523+0000 7fc099849700 1 --2- 172.21.15.151:0/3464096426 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fc07c06d690 0x7fc07c06fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:46:31.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.527+0000 7fc099849700 1 --2- 172.21.15.151:0/3464096426 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fc07c06d690 0x7fc07c06fb50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc084006fd0 tx=0x7fc084008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:46:31.530 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.527+0000 7fc092ffd700 1 -- 172.21.15.151:0/3464096426 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc08c02c020 con 0x7fc094105c30 2024-03-04T15:46:31.744 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:46:31.743+0000 7fc09baad700 1 -- 172.21.15.151:0/3464096426 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fc094062220 con 0x7fc07c06d690 2024-03-04T15:46:32.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:31 smithi117 bash[28843]: cluster 2024-03-04T15:46:30.786367+0000 mgr.y (mgr.14152) 235 : cluster 0 pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:32.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:31 smithi117 bash[28843]: audit 2024-03-04T15:46:31.753964+0000 mon.a (mon.0) 505 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:46:32.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:31 smithi117 bash[28843]: audit 2024-03-04T15:46:31.761136+0000 mon.a (mon.0) 506 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:46:32.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:31 smithi117 bash[28843]: audit 2024-03-04T15:46:31.763091+0000 mon.a (mon.0) 507 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:46:32.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:31 smithi117 bash[21480]: cluster 2024-03-04T15:46:30.786367+0000 mgr.y (mgr.14152) 235 : cluster 0 pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:32.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:31 smithi117 bash[21480]: audit 2024-03-04T15:46:31.753964+0000 mon.a (mon.0) 505 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:46:32.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:31 smithi117 bash[21480]: audit 2024-03-04T15:46:31.761136+0000 mon.a (mon.0) 506 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:46:32.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:31 smithi117 bash[21480]: audit 2024-03-04T15:46:31.763091+0000 mon.a (mon.0) 507 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:46:32.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:31 smithi151 bash[23756]: cluster 2024-03-04T15:46:30.786367+0000 mgr.y (mgr.14152) 235 : cluster 0 pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:32.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:31 smithi151 bash[23756]: audit 2024-03-04T15:46:31.753964+0000 mon.a (mon.0) 505 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:46:32.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:31 smithi151 bash[23756]: audit 2024-03-04T15:46:31.761136+0000 mon.a (mon.0) 506 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:46:32.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:31 smithi151 bash[23756]: audit 2024-03-04T15:46:31.763091+0000 mon.a (mon.0) 507 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:46:33.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:32 smithi117 bash[28843]: audit 2024-03-04T15:46:31.746874+0000 mgr.y (mgr.14152) 236 : audit 0 from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:46:33.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:32 smithi117 bash[21480]: audit 2024-03-04T15:46:31.746874+0000 mgr.y (mgr.14152) 236 : audit 0 from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:46:33.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:32 smithi151 bash[23756]: audit 2024-03-04T15:46:31.746874+0000 mgr.y (mgr.14152) 236 : audit 0 from='client.24226 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:46:34.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:33 smithi117 bash[28843]: cluster 2024-03-04T15:46:32.786917+0000 mgr.y (mgr.14152) 237 : cluster 0 pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:34.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:33 smithi117 bash[21480]: cluster 2024-03-04T15:46:32.786917+0000 mgr.y (mgr.14152) 237 : cluster 0 pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:34.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:33 smithi151 bash[23756]: cluster 2024-03-04T15:46:32.786917+0000 mgr.y (mgr.14152) 237 : cluster 0 pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:36.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:35 smithi117 bash[21480]: cluster 2024-03-04T15:46:34.787459+0000 mgr.y (mgr.14152) 238 : cluster 0 pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:36.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:35 smithi117 bash[28843]: cluster 2024-03-04T15:46:34.787459+0000 mgr.y (mgr.14152) 238 : cluster 0 pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:36.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:35 smithi151 bash[23756]: cluster 2024-03-04T15:46:34.787459+0000 mgr.y (mgr.14152) 238 : cluster 0 pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:37.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:37 smithi151 bash[23756]: cluster 2024-03-04T15:46:36.788050+0000 mgr.y (mgr.14152) 239 : cluster 0 pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:38.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:37 smithi117 bash[28843]: cluster 2024-03-04T15:46:36.788050+0000 mgr.y (mgr.14152) 239 : cluster 0 pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:38.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:37 smithi117 bash[21480]: cluster 2024-03-04T15:46:36.788050+0000 mgr.y (mgr.14152) 239 : cluster 0 pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:40.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:39 smithi151 bash[23756]: cluster 2024-03-04T15:46:38.788583+0000 mgr.y (mgr.14152) 240 : cluster 0 pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:40.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:39 smithi117 bash[21480]: cluster 2024-03-04T15:46:38.788583+0000 mgr.y (mgr.14152) 240 : cluster 0 pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:40.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:39 smithi117 bash[28843]: cluster 2024-03-04T15:46:38.788583+0000 mgr.y (mgr.14152) 240 : cluster 0 pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:41.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:40 smithi151 bash[23756]: audit 2024-03-04T15:46:40.057775+0000 mon.b (mon.1) 14 : audit 1 from='client.? 172.21.15.151:0/2835722818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9660e52c-72e0-4f9c-8cf5-bf5868080e5b"}]: dispatch 2024-03-04T15:46:41.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:40 smithi151 bash[23756]: audit 2024-03-04T15:46:40.058993+0000 mon.a (mon.0) 508 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9660e52c-72e0-4f9c-8cf5-bf5868080e5b"}]: dispatch 2024-03-04T15:46:41.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:40 smithi151 bash[23756]: audit 2024-03-04T15:46:40.062384+0000 mon.a (mon.0) 509 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9660e52c-72e0-4f9c-8cf5-bf5868080e5b"}]': finished 2024-03-04T15:46:41.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:40 smithi151 bash[23756]: cluster 2024-03-04T15:46:40.067947+0000 mon.a (mon.0) 510 : cluster 0 osdmap e36: 6 total, 5 up, 6 in 2024-03-04T15:46:41.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:40 smithi151 bash[23756]: audit 2024-03-04T15:46:40.068489+0000 mon.a (mon.0) 511 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T15:46:41.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:40 smithi151 bash[23756]: audit 2024-03-04T15:46:40.801608+0000 mon.b (mon.1) 15 : audit 0 from='client.? 172.21.15.151:0/2966893061' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:46:41.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:40 smithi117 bash[21480]: audit 2024-03-04T15:46:40.057775+0000 mon.b (mon.1) 14 : audit 1 from='client.? 172.21.15.151:0/2835722818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9660e52c-72e0-4f9c-8cf5-bf5868080e5b"}]: dispatch 2024-03-04T15:46:41.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:40 smithi117 bash[21480]: audit 2024-03-04T15:46:40.058993+0000 mon.a (mon.0) 508 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9660e52c-72e0-4f9c-8cf5-bf5868080e5b"}]: dispatch 2024-03-04T15:46:41.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:40 smithi117 bash[21480]: audit 2024-03-04T15:46:40.062384+0000 mon.a (mon.0) 509 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9660e52c-72e0-4f9c-8cf5-bf5868080e5b"}]': finished 2024-03-04T15:46:41.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:40 smithi117 bash[21480]: cluster 2024-03-04T15:46:40.067947+0000 mon.a (mon.0) 510 : cluster 0 osdmap e36: 6 total, 5 up, 6 in 2024-03-04T15:46:41.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:40 smithi117 bash[21480]: audit 2024-03-04T15:46:40.068489+0000 mon.a (mon.0) 511 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T15:46:41.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:40 smithi117 bash[21480]: audit 2024-03-04T15:46:40.801608+0000 mon.b (mon.1) 15 : audit 0 from='client.? 172.21.15.151:0/2966893061' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:46:41.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:40 smithi117 bash[28843]: audit 2024-03-04T15:46:40.057775+0000 mon.b (mon.1) 14 : audit 1 from='client.? 172.21.15.151:0/2835722818' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9660e52c-72e0-4f9c-8cf5-bf5868080e5b"}]: dispatch 2024-03-04T15:46:41.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:40 smithi117 bash[28843]: audit 2024-03-04T15:46:40.058993+0000 mon.a (mon.0) 508 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9660e52c-72e0-4f9c-8cf5-bf5868080e5b"}]: dispatch 2024-03-04T15:46:41.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:40 smithi117 bash[28843]: audit 2024-03-04T15:46:40.062384+0000 mon.a (mon.0) 509 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9660e52c-72e0-4f9c-8cf5-bf5868080e5b"}]': finished 2024-03-04T15:46:41.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:40 smithi117 bash[28843]: cluster 2024-03-04T15:46:40.067947+0000 mon.a (mon.0) 510 : cluster 0 osdmap e36: 6 total, 5 up, 6 in 2024-03-04T15:46:41.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:40 smithi117 bash[28843]: audit 2024-03-04T15:46:40.068489+0000 mon.a (mon.0) 511 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T15:46:41.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:40 smithi117 bash[28843]: audit 2024-03-04T15:46:40.801608+0000 mon.b (mon.1) 15 : audit 0 from='client.? 172.21.15.151:0/2966893061' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:46:42.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:42 smithi117 bash[21480]: cluster 2024-03-04T15:46:40.789204+0000 mgr.y (mgr.14152) 241 : cluster 0 pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:42.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:42 smithi117 bash[28843]: cluster 2024-03-04T15:46:40.789204+0000 mgr.y (mgr.14152) 241 : cluster 0 pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:42.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:42 smithi151 bash[23756]: cluster 2024-03-04T15:46:40.789204+0000 mgr.y (mgr.14152) 241 : cluster 0 pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:44.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:44 smithi117 bash[21480]: cluster 2024-03-04T15:46:42.789689+0000 mgr.y (mgr.14152) 242 : cluster 0 pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:44.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:44 smithi117 bash[28843]: cluster 2024-03-04T15:46:42.789689+0000 mgr.y (mgr.14152) 242 : cluster 0 pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:44.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:44 smithi151 bash[23756]: cluster 2024-03-04T15:46:42.789689+0000 mgr.y (mgr.14152) 242 : cluster 0 pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:46.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:46 smithi151 bash[23756]: cluster 2024-03-04T15:46:44.790189+0000 mgr.y (mgr.14152) 243 : cluster 0 pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:46.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:46 smithi117 bash[21480]: cluster 2024-03-04T15:46:44.790189+0000 mgr.y (mgr.14152) 243 : cluster 0 pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:46.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:46 smithi117 bash[28843]: cluster 2024-03-04T15:46:44.790189+0000 mgr.y (mgr.14152) 243 : cluster 0 pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:48.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:48 smithi151 bash[23756]: cluster 2024-03-04T15:46:46.790745+0000 mgr.y (mgr.14152) 244 : cluster 0 pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:48.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:48 smithi117 bash[21480]: cluster 2024-03-04T15:46:46.790745+0000 mgr.y (mgr.14152) 244 : cluster 0 pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:48.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:48 smithi117 bash[28843]: cluster 2024-03-04T15:46:46.790745+0000 mgr.y (mgr.14152) 244 : cluster 0 pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:50.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:50 smithi151 bash[23756]: cluster 2024-03-04T15:46:48.791306+0000 mgr.y (mgr.14152) 245 : cluster 0 pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:50.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:50 smithi117 bash[28843]: cluster 2024-03-04T15:46:48.791306+0000 mgr.y (mgr.14152) 245 : cluster 0 pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:50.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:50 smithi117 bash[21480]: cluster 2024-03-04T15:46:48.791306+0000 mgr.y (mgr.14152) 245 : cluster 0 pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:52.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:52 smithi151 bash[23756]: cluster 2024-03-04T15:46:50.791982+0000 mgr.y (mgr.14152) 246 : cluster 0 pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:52.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:52 smithi117 bash[28843]: cluster 2024-03-04T15:46:50.791982+0000 mgr.y (mgr.14152) 246 : cluster 0 pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:52.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:52 smithi117 bash[21480]: cluster 2024-03-04T15:46:50.791982+0000 mgr.y (mgr.14152) 246 : cluster 0 pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:53.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:53 smithi151 bash[23756]: audit 2024-03-04T15:46:52.570377+0000 mon.a (mon.0) 512 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-04T15:46:53.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:53 smithi151 bash[23756]: audit 2024-03-04T15:46:52.572240+0000 mon.a (mon.0) 513 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:46:53.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:53 smithi151 bash[23756]: cephadm 2024-03-04T15:46:52.573820+0000 mgr.y (mgr.14152) 247 : cephadm 1 Deploying daemon osd.5 on smithi151 2024-03-04T15:46:53.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:53 smithi117 bash[21480]: audit 2024-03-04T15:46:52.570377+0000 mon.a (mon.0) 512 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-04T15:46:53.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:53 smithi117 bash[21480]: audit 2024-03-04T15:46:52.572240+0000 mon.a (mon.0) 513 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:46:53.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:53 smithi117 bash[21480]: cephadm 2024-03-04T15:46:52.573820+0000 mgr.y (mgr.14152) 247 : cephadm 1 Deploying daemon osd.5 on smithi151 2024-03-04T15:46:53.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:53 smithi117 bash[28843]: audit 2024-03-04T15:46:52.570377+0000 mon.a (mon.0) 512 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-03-04T15:46:53.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:53 smithi117 bash[28843]: audit 2024-03-04T15:46:52.572240+0000 mon.a (mon.0) 513 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:46:53.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:53 smithi117 bash[28843]: cephadm 2024-03-04T15:46:52.573820+0000 mgr.y (mgr.14152) 247 : cephadm 1 Deploying daemon osd.5 on smithi151 2024-03-04T15:46:54.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:54 smithi151 bash[23756]: cluster 2024-03-04T15:46:52.792552+0000 mgr.y (mgr.14152) 248 : cluster 0 pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:54.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:54 smithi117 bash[21480]: cluster 2024-03-04T15:46:52.792552+0000 mgr.y (mgr.14152) 248 : cluster 0 pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:54.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:54 smithi117 bash[28843]: cluster 2024-03-04T15:46:52.792552+0000 mgr.y (mgr.14152) 248 : cluster 0 pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:56.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:56 smithi151 bash[23756]: cluster 2024-03-04T15:46:54.793102+0000 mgr.y (mgr.14152) 249 : cluster 0 pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:56.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:56 smithi117 bash[21480]: cluster 2024-03-04T15:46:54.793102+0000 mgr.y (mgr.14152) 249 : cluster 0 pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:56.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:56 smithi117 bash[28843]: cluster 2024-03-04T15:46:54.793102+0000 mgr.y (mgr.14152) 249 : cluster 0 pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:56.906 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:56 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:46:56.906 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 15:46:56 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:46:56.907 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:46:56 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:46:57.181 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:46:57 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:46:57.182 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 15:46:57 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:46:57.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:57 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:46:58.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:58 smithi151 bash[23756]: cluster 2024-03-04T15:46:56.793676+0000 mgr.y (mgr.14152) 250 : cluster 0 pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:58.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:58 smithi151 bash[23756]: audit 2024-03-04T15:46:57.161518+0000 mon.a (mon.0) 514 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:58.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:58 smithi151 bash[23756]: audit 2024-03-04T15:46:57.162453+0000 mon.a (mon.0) 515 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:46:58.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:46:58 smithi151 bash[23756]: audit 2024-03-04T15:46:57.170844+0000 mon.a (mon.0) 516 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:58.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:58 smithi117 bash[21480]: cluster 2024-03-04T15:46:56.793676+0000 mgr.y (mgr.14152) 250 : cluster 0 pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:58.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:58 smithi117 bash[21480]: audit 2024-03-04T15:46:57.161518+0000 mon.a (mon.0) 514 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:58.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:58 smithi117 bash[21480]: audit 2024-03-04T15:46:57.162453+0000 mon.a (mon.0) 515 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:46:58.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:46:58 smithi117 bash[21480]: audit 2024-03-04T15:46:57.170844+0000 mon.a (mon.0) 516 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:58.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:58 smithi117 bash[28843]: cluster 2024-03-04T15:46:56.793676+0000 mgr.y (mgr.14152) 250 : cluster 0 pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:46:58.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:58 smithi117 bash[28843]: audit 2024-03-04T15:46:57.161518+0000 mon.a (mon.0) 514 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:46:58.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:58 smithi117 bash[28843]: audit 2024-03-04T15:46:57.162453+0000 mon.a (mon.0) 515 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:46:58.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:46:58 smithi117 bash[28843]: audit 2024-03-04T15:46:57.170844+0000 mon.a (mon.0) 516 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:00.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:00 smithi151 bash[23756]: cluster 2024-03-04T15:46:58.794225+0000 mgr.y (mgr.14152) 251 : cluster 0 pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:47:00.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:00 smithi117 bash[28843]: cluster 2024-03-04T15:46:58.794225+0000 mgr.y (mgr.14152) 251 : cluster 0 pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:47:00.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:00 smithi117 bash[21480]: cluster 2024-03-04T15:46:58.794225+0000 mgr.y (mgr.14152) 251 : cluster 0 pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:47:02.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:02 smithi151 bash[23756]: cluster 2024-03-04T15:47:00.794787+0000 mgr.y (mgr.14152) 252 : cluster 0 pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:47:02.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:02 smithi117 bash[21480]: cluster 2024-03-04T15:47:00.794787+0000 mgr.y (mgr.14152) 252 : cluster 0 pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:47:02.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:02 smithi117 bash[28843]: cluster 2024-03-04T15:47:00.794787+0000 mgr.y (mgr.14152) 252 : cluster 0 pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:47:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:04 smithi151 bash[23756]: cluster 2024-03-04T15:47:02.795404+0000 mgr.y (mgr.14152) 253 : cluster 0 pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:47:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:04 smithi151 bash[23756]: audit 2024-03-04T15:47:03.602372+0000 mon.b (mon.1) 16 : audit 1 from='osd.5 [v2:172.21.15.151:6808/3277640682,v1:172.21.15.151:6809/3277640682]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T15:47:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:04 smithi151 bash[23756]: audit 2024-03-04T15:47:03.603605+0000 mon.a (mon.0) 517 : audit 1 from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T15:47:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:04 smithi151 bash[23756]: audit 2024-03-04T15:47:04.062769+0000 mon.a (mon.0) 518 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:04 smithi151 bash[23756]: audit 2024-03-04T15:47:04.074669+0000 mon.a (mon.0) 519 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:04 smithi151 bash[23756]: audit 2024-03-04T15:47:04.076871+0000 mon.a (mon.0) 520 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:47:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:04 smithi151 bash[23756]: audit 2024-03-04T15:47:04.078853+0000 mon.a (mon.0) 521 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:47:04.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:04 smithi151 bash[23756]: audit 2024-03-04T15:47:04.092880+0000 mon.a (mon.0) 522 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:04.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:04 smithi117 bash[21480]: cluster 2024-03-04T15:47:02.795404+0000 mgr.y (mgr.14152) 253 : cluster 0 pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:47:04.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:04 smithi117 bash[21480]: audit 2024-03-04T15:47:03.602372+0000 mon.b (mon.1) 16 : audit 1 from='osd.5 [v2:172.21.15.151:6808/3277640682,v1:172.21.15.151:6809/3277640682]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T15:47:04.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:04 smithi117 bash[21480]: audit 2024-03-04T15:47:03.603605+0000 mon.a (mon.0) 517 : audit 1 from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T15:47:04.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:04 smithi117 bash[21480]: audit 2024-03-04T15:47:04.062769+0000 mon.a (mon.0) 518 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:04.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:04 smithi117 bash[21480]: audit 2024-03-04T15:47:04.074669+0000 mon.a (mon.0) 519 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:04.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:04 smithi117 bash[21480]: audit 2024-03-04T15:47:04.076871+0000 mon.a (mon.0) 520 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:47:04.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:04 smithi117 bash[21480]: audit 2024-03-04T15:47:04.078853+0000 mon.a (mon.0) 521 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:47:04.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:04 smithi117 bash[21480]: audit 2024-03-04T15:47:04.092880+0000 mon.a (mon.0) 522 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:04.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:04 smithi117 bash[28843]: cluster 2024-03-04T15:47:02.795404+0000 mgr.y (mgr.14152) 253 : cluster 0 pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:47:04.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:04 smithi117 bash[28843]: audit 2024-03-04T15:47:03.602372+0000 mon.b (mon.1) 16 : audit 1 from='osd.5 [v2:172.21.15.151:6808/3277640682,v1:172.21.15.151:6809/3277640682]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T15:47:04.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:04 smithi117 bash[28843]: audit 2024-03-04T15:47:03.603605+0000 mon.a (mon.0) 517 : audit 1 from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-03-04T15:47:04.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:04 smithi117 bash[28843]: audit 2024-03-04T15:47:04.062769+0000 mon.a (mon.0) 518 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:04.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:04 smithi117 bash[28843]: audit 2024-03-04T15:47:04.074669+0000 mon.a (mon.0) 519 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:04.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:04 smithi117 bash[28843]: audit 2024-03-04T15:47:04.076871+0000 mon.a (mon.0) 520 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:47:04.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:04 smithi117 bash[28843]: audit 2024-03-04T15:47:04.078853+0000 mon.a (mon.0) 521 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:47:04.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:04 smithi117 bash[28843]: audit 2024-03-04T15:47:04.092880+0000 mon.a (mon.0) 522 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:05.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:05 smithi151 bash[23756]: audit 2024-03-04T15:47:04.187078+0000 mon.a (mon.0) 523 : audit 1 from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-04T15:47:05.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:05 smithi151 bash[23756]: audit 2024-03-04T15:47:04.193830+0000 mon.b (mon.1) 17 : audit 1 from='osd.5 [v2:172.21.15.151:6808/3277640682,v1:172.21.15.151:6809/3277640682]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:47:05.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:05 smithi151 bash[23756]: cluster 2024-03-04T15:47:04.196676+0000 mon.a (mon.0) 524 : cluster 0 osdmap e37: 6 total, 5 up, 6 in 2024-03-04T15:47:05.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:05 smithi151 bash[23756]: audit 2024-03-04T15:47:04.197344+0000 mon.a (mon.0) 525 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T15:47:05.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:05 smithi151 bash[23756]: audit 2024-03-04T15:47:04.197977+0000 mon.a (mon.0) 526 : audit 1 from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:47:05.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:05 smithi117 bash[21480]: audit 2024-03-04T15:47:04.187078+0000 mon.a (mon.0) 523 : audit 1 from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-04T15:47:05.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:05 smithi117 bash[21480]: audit 2024-03-04T15:47:04.193830+0000 mon.b (mon.1) 17 : audit 1 from='osd.5 [v2:172.21.15.151:6808/3277640682,v1:172.21.15.151:6809/3277640682]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:47:05.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:05 smithi117 bash[21480]: cluster 2024-03-04T15:47:04.196676+0000 mon.a (mon.0) 524 : cluster 0 osdmap e37: 6 total, 5 up, 6 in 2024-03-04T15:47:05.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:05 smithi117 bash[21480]: audit 2024-03-04T15:47:04.197344+0000 mon.a (mon.0) 525 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T15:47:05.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:05 smithi117 bash[21480]: audit 2024-03-04T15:47:04.197977+0000 mon.a (mon.0) 526 : audit 1 from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:47:05.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:05 smithi117 bash[28843]: audit 2024-03-04T15:47:04.187078+0000 mon.a (mon.0) 523 : audit 1 from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-03-04T15:47:05.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:05 smithi117 bash[28843]: audit 2024-03-04T15:47:04.193830+0000 mon.b (mon.1) 17 : audit 1 from='osd.5 [v2:172.21.15.151:6808/3277640682,v1:172.21.15.151:6809/3277640682]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:47:05.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:05 smithi117 bash[28843]: cluster 2024-03-04T15:47:04.196676+0000 mon.a (mon.0) 524 : cluster 0 osdmap e37: 6 total, 5 up, 6 in 2024-03-04T15:47:05.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:05 smithi117 bash[28843]: audit 2024-03-04T15:47:04.197344+0000 mon.a (mon.0) 525 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T15:47:05.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:05 smithi117 bash[28843]: audit 2024-03-04T15:47:04.197977+0000 mon.a (mon.0) 526 : audit 1 from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:47:06.491 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:06 smithi151 bash[23756]: cluster 2024-03-04T15:47:04.795959+0000 mgr.y (mgr.14152) 254 : cluster 0 pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:47:06.491 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:06 smithi151 bash[23756]: audit 2024-03-04T15:47:05.189502+0000 mon.a (mon.0) 527 : audit 1 from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-03-04T15:47:06.491 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:06 smithi151 bash[23756]: cluster 2024-03-04T15:47:05.196873+0000 mon.a (mon.0) 528 : cluster 0 osdmap e38: 6 total, 5 up, 6 in 2024-03-04T15:47:06.491 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:06 smithi151 bash[23756]: audit 2024-03-04T15:47:05.198086+0000 mon.a (mon.0) 529 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T15:47:06.491 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:06 smithi151 bash[23756]: audit 2024-03-04T15:47:05.211639+0000 mon.a (mon.0) 530 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T15:47:06.491 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:06 smithi151 bash[23756]: audit 2024-03-04T15:47:05.688220+0000 mon.a (mon.0) 531 : audit 1 from='osd.5 ' entity='osd.5' 2024-03-04T15:47:06.491 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:06 smithi151 bash[23756]: cluster 2024-03-04T15:47:06.198445+0000 mon.a (mon.0) 532 : cluster 1 osd.5 [v2:172.21.15.151:6808/3277640682,v1:172.21.15.151:6809/3277640682] boot 2024-03-04T15:47:06.492 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:06 smithi151 bash[23756]: cluster 2024-03-04T15:47:06.198551+0000 mon.a (mon.0) 533 : cluster 0 osdmap e39: 6 total, 6 up, 6 in 2024-03-04T15:47:06.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:06 smithi117 bash[28843]: cluster 2024-03-04T15:47:04.795959+0000 mgr.y (mgr.14152) 254 : cluster 0 pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:47:06.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:06 smithi117 bash[28843]: audit 2024-03-04T15:47:05.189502+0000 mon.a (mon.0) 527 : audit 1 from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-03-04T15:47:06.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:06 smithi117 bash[28843]: cluster 2024-03-04T15:47:05.196873+0000 mon.a (mon.0) 528 : cluster 0 osdmap e38: 6 total, 5 up, 6 in 2024-03-04T15:47:06.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:06 smithi117 bash[28843]: audit 2024-03-04T15:47:05.198086+0000 mon.a (mon.0) 529 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T15:47:06.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:06 smithi117 bash[28843]: audit 2024-03-04T15:47:05.211639+0000 mon.a (mon.0) 530 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T15:47:06.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:06 smithi117 bash[28843]: audit 2024-03-04T15:47:05.688220+0000 mon.a (mon.0) 531 : audit 1 from='osd.5 ' entity='osd.5' 2024-03-04T15:47:06.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:06 smithi117 bash[28843]: cluster 2024-03-04T15:47:06.198445+0000 mon.a (mon.0) 532 : cluster 1 osd.5 [v2:172.21.15.151:6808/3277640682,v1:172.21.15.151:6809/3277640682] boot 2024-03-04T15:47:06.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:06 smithi117 bash[28843]: cluster 2024-03-04T15:47:06.198551+0000 mon.a (mon.0) 533 : cluster 0 osdmap e39: 6 total, 6 up, 6 in 2024-03-04T15:47:06.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:06 smithi117 bash[21480]: cluster 2024-03-04T15:47:04.795959+0000 mgr.y (mgr.14152) 254 : cluster 0 pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:47:06.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:06 smithi117 bash[21480]: audit 2024-03-04T15:47:05.189502+0000 mon.a (mon.0) 527 : audit 1 from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-03-04T15:47:06.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:06 smithi117 bash[21480]: cluster 2024-03-04T15:47:05.196873+0000 mon.a (mon.0) 528 : cluster 0 osdmap e38: 6 total, 5 up, 6 in 2024-03-04T15:47:06.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:06 smithi117 bash[21480]: audit 2024-03-04T15:47:05.198086+0000 mon.a (mon.0) 529 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T15:47:06.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:06 smithi117 bash[21480]: audit 2024-03-04T15:47:05.211639+0000 mon.a (mon.0) 530 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T15:47:06.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:06 smithi117 bash[21480]: audit 2024-03-04T15:47:05.688220+0000 mon.a (mon.0) 531 : audit 1 from='osd.5 ' entity='osd.5' 2024-03-04T15:47:06.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:06 smithi117 bash[21480]: cluster 2024-03-04T15:47:06.198445+0000 mon.a (mon.0) 532 : cluster 1 osd.5 [v2:172.21.15.151:6808/3277640682,v1:172.21.15.151:6809/3277640682] boot 2024-03-04T15:47:06.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:06 smithi117 bash[21480]: cluster 2024-03-04T15:47:06.198551+0000 mon.a (mon.0) 533 : cluster 0 osdmap e39: 6 total, 6 up, 6 in 2024-03-04T15:47:07.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:07 smithi151 bash[23756]: cluster 2024-03-04T15:47:04.619905+0000 osd.5 (osd.5) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:47:07.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:07 smithi151 bash[23756]: cluster 2024-03-04T15:47:04.620031+0000 osd.5 (osd.5) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:47:07.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:07 smithi151 bash[23756]: audit 2024-03-04T15:47:06.211728+0000 mon.a (mon.0) 534 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T15:47:07.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:07 smithi151 bash[23756]: cluster 2024-03-04T15:47:06.755331+0000 mon.a (mon.0) 535 : cluster 0 osdmap e40: 6 total, 6 up, 6 in 2024-03-04T15:47:07.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:07 smithi117 bash[21480]: cluster 2024-03-04T15:47:04.619905+0000 osd.5 (osd.5) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:47:07.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:07 smithi117 bash[21480]: cluster 2024-03-04T15:47:04.620031+0000 osd.5 (osd.5) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:47:07.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:07 smithi117 bash[21480]: audit 2024-03-04T15:47:06.211728+0000 mon.a (mon.0) 534 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T15:47:07.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:07 smithi117 bash[21480]: cluster 2024-03-04T15:47:06.755331+0000 mon.a (mon.0) 535 : cluster 0 osdmap e40: 6 total, 6 up, 6 in 2024-03-04T15:47:07.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:07 smithi117 bash[28843]: cluster 2024-03-04T15:47:04.619905+0000 osd.5 (osd.5) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:47:07.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:07 smithi117 bash[28843]: cluster 2024-03-04T15:47:04.620031+0000 osd.5 (osd.5) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:47:07.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:07 smithi117 bash[28843]: audit 2024-03-04T15:47:06.211728+0000 mon.a (mon.0) 534 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T15:47:07.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:07 smithi117 bash[28843]: cluster 2024-03-04T15:47:06.755331+0000 mon.a (mon.0) 535 : cluster 0 osdmap e40: 6 total, 6 up, 6 in 2024-03-04T15:47:07.767 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:07.766+0000 7fc092ffd700 1 -- 172.21.15.151:0/3464096426 <== mgr.14152 v2:172.21.15.117:6800/2865911351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc094062220 con 0x7fc07c06d690 2024-03-04T15:47:07.768 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 5 on host 'smithi151' 2024-03-04T15:47:07.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:07.766+0000 7fc09baad700 1 -- 172.21.15.151:0/3464096426 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fc07c06d690 msgr2=0x7fc07c06fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:47:07.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:07.766+0000 7fc09baad700 1 --2- 172.21.15.151:0/3464096426 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fc07c06d690 0x7fc07c06fb50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc084006fd0 tx=0x7fc084008040 comp rx=0 tx=0).stop 2024-03-04T15:47:07.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:07.766+0000 7fc09baad700 1 -- 172.21.15.151:0/3464096426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc094105c30 msgr2=0x7fc0941a0f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:47:07.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:07.766+0000 7fc09baad700 1 --2- 172.21.15.151:0/3464096426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc094105c30 0x7fc0941a0f10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fc08c007d90 tx=0x7fc08c005b80 comp rx=0 tx=0).stop 2024-03-04T15:47:07.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:07.770+0000 7fc09baad700 1 -- 172.21.15.151:0/3464096426 shutdown_connections 2024-03-04T15:47:07.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:07.770+0000 7fc09baad700 1 --2- 172.21.15.151:0/3464096426 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fc07c06d690 0x7fc07c06fb50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:47:07.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:07.770+0000 7fc09baad700 1 --2- 172.21.15.151:0/3464096426 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc094105c30 0x7fc0941a0f10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:47:07.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:07.770+0000 7fc09baad700 1 --2- 172.21.15.151:0/3464096426 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc094104090 0x7fc0941999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:47:07.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:07.770+0000 7fc09baad700 1 --2- 172.21.15.151:0/3464096426 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc094105290 0x7fc094199ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:47:07.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:07.770+0000 7fc09baad700 1 -- 172.21.15.151:0/3464096426 >> 172.21.15.151:0/3464096426 conn(0x7fc0940ff670 msgr2=0x7fc094101170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:47:07.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:07.770+0000 7fc09baad700 1 -- 172.21.15.151:0/3464096426 shutdown_connections 2024-03-04T15:47:07.771 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:07.770+0000 7fc09baad700 1 -- 172.21.15.151:0/3464096426 wait complete. 2024-03-04T15:47:08.461 DEBUG:teuthology.orchestra.run.smithi151:osd.5> sudo journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.5.service 2024-03-04T15:47:08.467 INFO:tasks.cephadm:Deploying osd.6 on smithi151 with /dev/vg_nvme/lv_2... 2024-03-04T15:47:08.468 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-03-04T15:47:09.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:08 smithi117 bash[21480]: cluster 2024-03-04T15:47:06.796489+0000 mgr.y (mgr.14152) 255 : cluster 0 pgmap v232: 1 pgs: 1 unknown; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:47:09.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:08 smithi117 bash[21480]: audit 2024-03-04T15:47:07.724295+0000 mon.a (mon.0) 536 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:09.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:08 smithi117 bash[21480]: audit 2024-03-04T15:47:07.733568+0000 mon.a (mon.0) 537 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:09.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:08 smithi117 bash[21480]: audit 2024-03-04T15:47:07.737994+0000 mon.a (mon.0) 538 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:47:09.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:08 smithi117 bash[21480]: cluster 2024-03-04T15:47:07.764215+0000 mon.a (mon.0) 539 : cluster 0 osdmap e41: 6 total, 6 up, 6 in 2024-03-04T15:47:09.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:08 smithi117 bash[28843]: cluster 2024-03-04T15:47:06.796489+0000 mgr.y (mgr.14152) 255 : cluster 0 pgmap v232: 1 pgs: 1 unknown; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:47:09.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:08 smithi117 bash[28843]: audit 2024-03-04T15:47:07.724295+0000 mon.a (mon.0) 536 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:09.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:08 smithi117 bash[28843]: audit 2024-03-04T15:47:07.733568+0000 mon.a (mon.0) 537 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:09.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:08 smithi117 bash[28843]: audit 2024-03-04T15:47:07.737994+0000 mon.a (mon.0) 538 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:47:09.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:08 smithi117 bash[28843]: cluster 2024-03-04T15:47:07.764215+0000 mon.a (mon.0) 539 : cluster 0 osdmap e41: 6 total, 6 up, 6 in 2024-03-04T15:47:09.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:08 smithi151 bash[23756]: cluster 2024-03-04T15:47:06.796489+0000 mgr.y (mgr.14152) 255 : cluster 0 pgmap v232: 1 pgs: 1 unknown; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-03-04T15:47:09.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:08 smithi151 bash[23756]: audit 2024-03-04T15:47:07.724295+0000 mon.a (mon.0) 536 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:09.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:08 smithi151 bash[23756]: audit 2024-03-04T15:47:07.733568+0000 mon.a (mon.0) 537 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:09.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:08 smithi151 bash[23756]: audit 2024-03-04T15:47:07.737994+0000 mon.a (mon.0) 538 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:47:09.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:08 smithi151 bash[23756]: cluster 2024-03-04T15:47:07.764215+0000 mon.a (mon.0) 539 : cluster 0 osdmap e41: 6 total, 6 up, 6 in 2024-03-04T15:47:10.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:09 smithi117 bash[21480]: cluster 2024-03-04T15:47:08.774409+0000 mon.a (mon.0) 540 : cluster 0 osdmap e42: 6 total, 6 up, 6 in 2024-03-04T15:47:10.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:09 smithi117 bash[21480]: cluster 2024-03-04T15:47:08.797052+0000 mgr.y (mgr.14152) 256 : cluster 0 pgmap v235: 1 pgs: 1 unknown; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:10.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:09 smithi117 bash[28843]: cluster 2024-03-04T15:47:08.774409+0000 mon.a (mon.0) 540 : cluster 0 osdmap e42: 6 total, 6 up, 6 in 2024-03-04T15:47:10.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:09 smithi117 bash[28843]: cluster 2024-03-04T15:47:08.797052+0000 mgr.y (mgr.14152) 256 : cluster 0 pgmap v235: 1 pgs: 1 unknown; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:10.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:09 smithi151 bash[23756]: cluster 2024-03-04T15:47:08.774409+0000 mon.a (mon.0) 540 : cluster 0 osdmap e42: 6 total, 6 up, 6 in 2024-03-04T15:47:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:09 smithi151 bash[23756]: cluster 2024-03-04T15:47:08.797052+0000 mgr.y (mgr.14152) 256 : cluster 0 pgmap v235: 1 pgs: 1 unknown; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:11.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:10 smithi117 bash[21480]: cluster 2024-03-04T15:47:09.789857+0000 mon.a (mon.0) 541 : cluster 0 osdmap e43: 6 total, 6 up, 6 in 2024-03-04T15:47:11.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:10 smithi117 bash[28843]: cluster 2024-03-04T15:47:09.789857+0000 mon.a (mon.0) 541 : cluster 0 osdmap e43: 6 total, 6 up, 6 in 2024-03-04T15:47:11.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:10 smithi151 bash[23756]: cluster 2024-03-04T15:47:09.789857+0000 mon.a (mon.0) 541 : cluster 0 osdmap e43: 6 total, 6 up, 6 in 2024-03-04T15:47:12.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:11 smithi117 bash[21480]: cluster 2024-03-04T15:47:10.797579+0000 mgr.y (mgr.14152) 257 : cluster 0 pgmap v237: 1 pgs: 1 unknown; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:12.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:11 smithi117 bash[28843]: cluster 2024-03-04T15:47:10.797579+0000 mgr.y (mgr.14152) 257 : cluster 0 pgmap v237: 1 pgs: 1 unknown; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:12.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:11 smithi151 bash[23756]: cluster 2024-03-04T15:47:10.797579+0000 mgr.y (mgr.14152) 257 : cluster 0 pgmap v237: 1 pgs: 1 unknown; 577 KiB data, 178 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:12.751 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T15:47:14.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:13 smithi151 bash[23756]: cluster 2024-03-04T15:47:12.798129+0000 mgr.y (mgr.14152) 258 : cluster 0 pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-04T15:47:14.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:13 smithi117 bash[21480]: cluster 2024-03-04T15:47:12.798129+0000 mgr.y (mgr.14152) 258 : cluster 0 pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-04T15:47:14.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:13 smithi117 bash[28843]: cluster 2024-03-04T15:47:12.798129+0000 mgr.y (mgr.14152) 258 : cluster 0 pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-03-04T15:47:16.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:15 smithi151 bash[23756]: cluster 2024-03-04T15:47:14.798698+0000 mgr.y (mgr.14152) 259 : cluster 0 pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-04T15:47:16.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:15 smithi117 bash[21480]: cluster 2024-03-04T15:47:14.798698+0000 mgr.y (mgr.14152) 259 : cluster 0 pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-04T15:47:16.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:15 smithi117 bash[28843]: cluster 2024-03-04T15:47:14.798698+0000 mgr.y (mgr.14152) 259 : cluster 0 pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 82 KiB/s, 0 objects/s recovering 2024-03-04T15:47:17.021 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-03-04T15:47:17.021 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-03-04T15:47:17.021 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-03-04T15:47:17.021 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-03-04T15:47:17.021 INFO:teuthology.orchestra.run.smithi151.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.013704 s, 765 MB/s 2024-03-04T15:47:17.021 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-03-04T15:47:17.943 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi151:vg_nvme/lv_2 2024-03-04T15:47:18.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:17 smithi117 bash[21480]: cluster 2024-03-04T15:47:16.799273+0000 mgr.y (mgr.14152) 260 : cluster 0 pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T15:47:18.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:17 smithi117 bash[28843]: cluster 2024-03-04T15:47:16.799273+0000 mgr.y (mgr.14152) 260 : cluster 0 pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T15:47:18.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:17 smithi151 bash[23756]: cluster 2024-03-04T15:47:16.799273+0000 mgr.y (mgr.14152) 260 : cluster 0 pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T15:47:19.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:19 smithi151 bash[23756]: cephadm 2024-03-04T15:47:18.395676+0000 mgr.y (mgr.14152) 261 : cephadm 1 Detected new or changed devices on smithi151 2024-03-04T15:47:19.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:19 smithi151 bash[23756]: audit 2024-03-04T15:47:18.404769+0000 mon.a (mon.0) 542 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:19.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:19 smithi151 bash[23756]: audit 2024-03-04T15:47:18.414402+0000 mon.a (mon.0) 543 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:19.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:19 smithi151 bash[23756]: audit 2024-03-04T15:47:18.416809+0000 mon.a (mon.0) 544 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:47:19.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:19 smithi151 bash[23756]: audit 2024-03-04T15:47:18.418561+0000 mon.a (mon.0) 545 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:47:19.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:19 smithi151 bash[23756]: cephadm 2024-03-04T15:47:18.419882+0000 mgr.y (mgr.14152) 262 : cephadm 1 Adjusting osd_memory_target on smithi151 to 8632M 2024-03-04T15:47:19.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:19 smithi151 bash[23756]: audit 2024-03-04T15:47:18.428340+0000 mon.a (mon.0) 546 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:19.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:19 smithi151 bash[23756]: audit 2024-03-04T15:47:18.433010+0000 mon.a (mon.0) 547 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:47:19.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:19 smithi151 bash[23756]: audit 2024-03-04T15:47:18.435041+0000 mon.a (mon.0) 548 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:47:19.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:19 smithi151 bash[23756]: audit 2024-03-04T15:47:18.447661+0000 mon.a (mon.0) 549 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:19.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:19 smithi117 bash[21480]: cephadm 2024-03-04T15:47:18.395676+0000 mgr.y (mgr.14152) 261 : cephadm 1 Detected new or changed devices on smithi151 2024-03-04T15:47:19.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:19 smithi117 bash[21480]: audit 2024-03-04T15:47:18.404769+0000 mon.a (mon.0) 542 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:19.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:19 smithi117 bash[21480]: audit 2024-03-04T15:47:18.414402+0000 mon.a (mon.0) 543 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:19.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:19 smithi117 bash[21480]: audit 2024-03-04T15:47:18.416809+0000 mon.a (mon.0) 544 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:47:19.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:19 smithi117 bash[21480]: audit 2024-03-04T15:47:18.418561+0000 mon.a (mon.0) 545 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:47:19.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:19 smithi117 bash[21480]: cephadm 2024-03-04T15:47:18.419882+0000 mgr.y (mgr.14152) 262 : cephadm 1 Adjusting osd_memory_target on smithi151 to 8632M 2024-03-04T15:47:19.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:19 smithi117 bash[21480]: audit 2024-03-04T15:47:18.428340+0000 mon.a (mon.0) 546 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:19.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:19 smithi117 bash[21480]: audit 2024-03-04T15:47:18.433010+0000 mon.a (mon.0) 547 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:47:19.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:19 smithi117 bash[21480]: audit 2024-03-04T15:47:18.435041+0000 mon.a (mon.0) 548 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:47:19.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:19 smithi117 bash[21480]: audit 2024-03-04T15:47:18.447661+0000 mon.a (mon.0) 549 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:19.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:19 smithi117 bash[28843]: cephadm 2024-03-04T15:47:18.395676+0000 mgr.y (mgr.14152) 261 : cephadm 1 Detected new or changed devices on smithi151 2024-03-04T15:47:19.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:19 smithi117 bash[28843]: audit 2024-03-04T15:47:18.404769+0000 mon.a (mon.0) 542 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:19.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:19 smithi117 bash[28843]: audit 2024-03-04T15:47:18.414402+0000 mon.a (mon.0) 543 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:19.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:19 smithi117 bash[28843]: audit 2024-03-04T15:47:18.416809+0000 mon.a (mon.0) 544 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:47:19.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:19 smithi117 bash[28843]: audit 2024-03-04T15:47:18.418561+0000 mon.a (mon.0) 545 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:47:19.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:19 smithi117 bash[28843]: cephadm 2024-03-04T15:47:18.419882+0000 mgr.y (mgr.14152) 262 : cephadm 1 Adjusting osd_memory_target on smithi151 to 8632M 2024-03-04T15:47:19.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:19 smithi117 bash[28843]: audit 2024-03-04T15:47:18.428340+0000 mon.a (mon.0) 546 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:19.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:19 smithi117 bash[28843]: audit 2024-03-04T15:47:18.433010+0000 mon.a (mon.0) 547 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:47:19.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:19 smithi117 bash[28843]: audit 2024-03-04T15:47:18.435041+0000 mon.a (mon.0) 548 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:47:19.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:19 smithi117 bash[28843]: audit 2024-03-04T15:47:18.447661+0000 mon.a (mon.0) 549 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:20.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:20 smithi117 bash[21480]: cluster 2024-03-04T15:47:18.799899+0000 mgr.y (mgr.14152) 263 : cluster 0 pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-04T15:47:20.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:20 smithi117 bash[28843]: cluster 2024-03-04T15:47:18.799899+0000 mgr.y (mgr.14152) 263 : cluster 0 pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-04T15:47:20.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:20 smithi151 bash[23756]: cluster 2024-03-04T15:47:18.799899+0000 mgr.y (mgr.14152) 263 : cluster 0 pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-04T15:47:22.669 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T15:47:22.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:22 smithi117 bash[21480]: cluster 2024-03-04T15:47:20.800456+0000 mgr.y (mgr.14152) 264 : cluster 0 pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-04T15:47:22.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:22 smithi117 bash[28843]: cluster 2024-03-04T15:47:20.800456+0000 mgr.y (mgr.14152) 264 : cluster 0 pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-04T15:47:22.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:22 smithi151 bash[23756]: cluster 2024-03-04T15:47:20.800456+0000 mgr.y (mgr.14152) 264 : cluster 0 pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-04T15:47:24.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:24 smithi151 bash[23756]: cluster 2024-03-04T15:47:22.801104+0000 mgr.y (mgr.14152) 265 : cluster 0 pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-04T15:47:24.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:24 smithi117 bash[21480]: cluster 2024-03-04T15:47:22.801104+0000 mgr.y (mgr.14152) 265 : cluster 0 pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-04T15:47:24.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:24 smithi117 bash[28843]: cluster 2024-03-04T15:47:22.801104+0000 mgr.y (mgr.14152) 265 : cluster 0 pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-04T15:47:25.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddde99d700 1 -- 172.21.15.151:0/926367109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddd8105c30 msgr2=0x7fddd810c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:47:25.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddde99d700 1 --2- 172.21.15.151:0/926367109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddd8105c30 0x7fddd810c4e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fddd00099c0 tx=0x7fddd0009cd0 comp rx=0 tx=0).stop 2024-03-04T15:47:25.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddde99d700 1 -- 172.21.15.151:0/926367109 shutdown_connections 2024-03-04T15:47:25.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddde99d700 1 --2- 172.21.15.151:0/926367109 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddd8104090 0x7fddd81044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:47:25.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddde99d700 1 --2- 172.21.15.151:0/926367109 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddd8105c30 0x7fddd810c4e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:47:25.220 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddde99d700 1 --2- 172.21.15.151:0/926367109 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddd8105290 0x7fddd81056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:47:25.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddde99d700 1 -- 172.21.15.151:0/926367109 >> 172.21.15.151:0/926367109 conn(0x7fddd80ff670 msgr2=0x7fddd8101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:47:25.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddde99d700 1 -- 172.21.15.151:0/926367109 shutdown_connections 2024-03-04T15:47:25.221 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddde99d700 1 -- 172.21.15.151:0/926367109 wait complete. 2024-03-04T15:47:25.222 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddde99d700 1 Processor -- start 2024-03-04T15:47:25.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddde99d700 1 -- start start 2024-03-04T15:47:25.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddde99d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddd8104090 0x7fddd8199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:47:25.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddde99d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddd8105290 0x7fddd8199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:47:25.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddde99d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddd8105c30 0x7fddd81a0cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:47:25.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddde99d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd810f180 con 0x7fddd8105c30 2024-03-04T15:47:25.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddde99d700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddd810f000 con 0x7fddd8105290 2024-03-04T15:47:25.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddde99d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddd810f300 con 0x7fddd8104090 2024-03-04T15:47:25.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddd77fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddd8105290 0x7fddd8199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:47:25.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddd77fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddd8105290 0x7fddd8199c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:52696/0 (socket says 172.21.15.151:52696) 2024-03-04T15:47:25.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddd77fe700 1 -- 172.21.15.151:0/238845587 learned_addr learned my addr 172.21.15.151:0/238845587 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-04T15:47:25.223 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddd7fff700 1 --2- 172.21.15.151:0/238845587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddd8104090 0x7fddd8199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:47:25.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fdddcf3a700 1 --2- 172.21.15.151:0/238845587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddd8105c30 0x7fddd81a0cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:47:25.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddd77fe700 1 -- 172.21.15.151:0/238845587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddd8104090 msgr2=0x7fddd8199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:47:25.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddd77fe700 1 --2- 172.21.15.151:0/238845587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddd8104090 0x7fddd8199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:47:25.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddd77fe700 1 -- 172.21.15.151:0/238845587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddd8105c30 msgr2=0x7fddd81a0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:47:25.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddd77fe700 1 --2- 172.21.15.151:0/238845587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddd8105c30 0x7fddd81a0cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:47:25.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddd77fe700 1 -- 172.21.15.151:0/238845587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddd81a1390 con 0x7fddd8105290 2024-03-04T15:47:25.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddd77fe700 1 --2- 172.21.15.151:0/238845587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddd8105290 0x7fddd8199c70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fddc800d980 tx=0x7fddc800dd40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:47:25.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddd57fa700 1 -- 172.21.15.151:0/238845587 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc8017070 con 0x7fddd8105290 2024-03-04T15:47:25.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddd57fa700 1 -- 172.21.15.151:0/238845587 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fddc8004830 con 0x7fddd8105290 2024-03-04T15:47:25.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddde99d700 1 -- 172.21.15.151:0/238845587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddd81a1680 con 0x7fddd8105290 2024-03-04T15:47:25.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddd57fa700 1 -- 172.21.15.151:0/238845587 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddc8005390 con 0x7fddd8105290 2024-03-04T15:47:25.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.218+0000 7fddde99d700 1 -- 172.21.15.151:0/238845587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddd81a1b70 con 0x7fddd8105290 2024-03-04T15:47:25.224 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.222+0000 7fddde99d700 1 -- 172.21.15.151:0/238845587 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddd804faa0 con 0x7fddd8105290 2024-03-04T15:47:25.225 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.222+0000 7fddd57fa700 1 -- 172.21.15.151:0/238845587 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 15) v1 ==== 91397+0+0 (secure 0 0 0) 0x7fddc8005530 con 0x7fddd8105290 2024-03-04T15:47:25.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.222+0000 7fddd57fa700 1 --2- 172.21.15.151:0/238845587 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fddb806d690 0x7fddb806fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:47:25.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.222+0000 7fddd57fa700 1 -- 172.21.15.151:0/238845587 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 4423+0+0 (secure 0 0 0) 0x7fddc808cc20 con 0x7fddd8105290 2024-03-04T15:47:25.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.222+0000 7fddd7fff700 1 --2- 172.21.15.151:0/238845587 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fddb806d690 0x7fddb806fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:47:25.226 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.222+0000 7fddd7fff700 1 --2- 172.21.15.151:0/238845587 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fddb806d690 0x7fddb806fb50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fddd81050f0 tx=0x7fddc0009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:47:25.229 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.226+0000 7fddd57fa700 1 -- 172.21.15.151:0/238845587 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fddc8020020 con 0x7fddd8105290 2024-03-04T15:47:25.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:47:25.446+0000 7fddde99d700 1 -- 172.21.15.151:0/238845587 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fddd8062220 con 0x7fddb806d690 2024-03-04T15:47:25.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:25 smithi117 bash[21480]: audit 2024-03-04T15:47:25.458354+0000 mon.a (mon.0) 550 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:47:25.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:25 smithi117 bash[21480]: audit 2024-03-04T15:47:25.465458+0000 mon.a (mon.0) 551 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:47:25.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:25 smithi117 bash[21480]: audit 2024-03-04T15:47:25.467074+0000 mon.a (mon.0) 552 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:47:25.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:25 smithi117 bash[28843]: audit 2024-03-04T15:47:25.458354+0000 mon.a (mon.0) 550 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:47:25.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:25 smithi117 bash[28843]: audit 2024-03-04T15:47:25.465458+0000 mon.a (mon.0) 551 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:47:25.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:25 smithi117 bash[28843]: audit 2024-03-04T15:47:25.467074+0000 mon.a (mon.0) 552 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:47:25.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:25 smithi151 bash[23756]: audit 2024-03-04T15:47:25.458354+0000 mon.a (mon.0) 550 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:47:25.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:25 smithi151 bash[23756]: audit 2024-03-04T15:47:25.465458+0000 mon.a (mon.0) 551 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:47:25.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:25 smithi151 bash[23756]: audit 2024-03-04T15:47:25.467074+0000 mon.a (mon.0) 552 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:47:26.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:26 smithi117 bash[21480]: cluster 2024-03-04T15:47:24.801673+0000 mgr.y (mgr.14152) 266 : cluster 0 pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:26.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:26 smithi117 bash[21480]: audit 2024-03-04T15:47:25.451217+0000 mgr.y (mgr.14152) 267 : audit 0 from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:47:26.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:26 smithi117 bash[28843]: cluster 2024-03-04T15:47:24.801673+0000 mgr.y (mgr.14152) 266 : cluster 0 pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:26.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:26 smithi117 bash[28843]: audit 2024-03-04T15:47:25.451217+0000 mgr.y (mgr.14152) 267 : audit 0 from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:47:26.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:26 smithi151 bash[23756]: cluster 2024-03-04T15:47:24.801673+0000 mgr.y (mgr.14152) 266 : cluster 0 pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:26.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:26 smithi151 bash[23756]: audit 2024-03-04T15:47:25.451217+0000 mgr.y (mgr.14152) 267 : audit 0 from='client.24253 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:47:28.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:28 smithi117 bash[21480]: cluster 2024-03-04T15:47:26.802220+0000 mgr.y (mgr.14152) 268 : cluster 0 pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:28.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:28 smithi117 bash[28843]: cluster 2024-03-04T15:47:26.802220+0000 mgr.y (mgr.14152) 268 : cluster 0 pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:28.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:28 smithi151 bash[23756]: cluster 2024-03-04T15:47:26.802220+0000 mgr.y (mgr.14152) 268 : cluster 0 pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:30.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:30 smithi117 bash[21480]: cluster 2024-03-04T15:47:28.802787+0000 mgr.y (mgr.14152) 269 : cluster 0 pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:30.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:30 smithi117 bash[28843]: cluster 2024-03-04T15:47:28.802787+0000 mgr.y (mgr.14152) 269 : cluster 0 pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:30.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:30 smithi151 bash[23756]: cluster 2024-03-04T15:47:28.802787+0000 mgr.y (mgr.14152) 269 : cluster 0 pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:32 smithi117 bash[21480]: cluster 2024-03-04T15:47:30.803347+0000 mgr.y (mgr.14152) 270 : cluster 0 pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:32 smithi117 bash[28843]: cluster 2024-03-04T15:47:30.803347+0000 mgr.y (mgr.14152) 270 : cluster 0 pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:32.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:32 smithi151 bash[23756]: cluster 2024-03-04T15:47:30.803347+0000 mgr.y (mgr.14152) 270 : cluster 0 pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:34.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:34 smithi117 bash[21480]: cluster 2024-03-04T15:47:32.803901+0000 mgr.y (mgr.14152) 271 : cluster 0 pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:34.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:34 smithi117 bash[21480]: audit 2024-03-04T15:47:33.887241+0000 mon.b (mon.1) 18 : audit 1 from='client.? 172.21.15.151:0/3380112624' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "706cd130-5cc0-41f3-9e78-85922c29a01c"}]: dispatch 2024-03-04T15:47:34.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:34 smithi117 bash[21480]: audit 2024-03-04T15:47:33.888585+0000 mon.a (mon.0) 553 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "706cd130-5cc0-41f3-9e78-85922c29a01c"}]: dispatch 2024-03-04T15:47:34.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:34 smithi117 bash[21480]: audit 2024-03-04T15:47:33.892226+0000 mon.a (mon.0) 554 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "706cd130-5cc0-41f3-9e78-85922c29a01c"}]': finished 2024-03-04T15:47:34.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:34 smithi117 bash[21480]: cluster 2024-03-04T15:47:33.898048+0000 mon.a (mon.0) 555 : cluster 0 osdmap e44: 7 total, 6 up, 7 in 2024-03-04T15:47:34.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:34 smithi117 bash[21480]: audit 2024-03-04T15:47:33.898851+0000 mon.a (mon.0) 556 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T15:47:34.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:34 smithi117 bash[28843]: cluster 2024-03-04T15:47:32.803901+0000 mgr.y (mgr.14152) 271 : cluster 0 pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:34.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:34 smithi117 bash[28843]: audit 2024-03-04T15:47:33.887241+0000 mon.b (mon.1) 18 : audit 1 from='client.? 172.21.15.151:0/3380112624' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "706cd130-5cc0-41f3-9e78-85922c29a01c"}]: dispatch 2024-03-04T15:47:34.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:34 smithi117 bash[28843]: audit 2024-03-04T15:47:33.888585+0000 mon.a (mon.0) 553 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "706cd130-5cc0-41f3-9e78-85922c29a01c"}]: dispatch 2024-03-04T15:47:34.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:34 smithi117 bash[28843]: audit 2024-03-04T15:47:33.892226+0000 mon.a (mon.0) 554 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "706cd130-5cc0-41f3-9e78-85922c29a01c"}]': finished 2024-03-04T15:47:34.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:34 smithi117 bash[28843]: cluster 2024-03-04T15:47:33.898048+0000 mon.a (mon.0) 555 : cluster 0 osdmap e44: 7 total, 6 up, 7 in 2024-03-04T15:47:34.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:34 smithi117 bash[28843]: audit 2024-03-04T15:47:33.898851+0000 mon.a (mon.0) 556 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T15:47:34.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:34 smithi151 bash[23756]: cluster 2024-03-04T15:47:32.803901+0000 mgr.y (mgr.14152) 271 : cluster 0 pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:34.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:34 smithi151 bash[23756]: audit 2024-03-04T15:47:33.887241+0000 mon.b (mon.1) 18 : audit 1 from='client.? 172.21.15.151:0/3380112624' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "706cd130-5cc0-41f3-9e78-85922c29a01c"}]: dispatch 2024-03-04T15:47:34.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:34 smithi151 bash[23756]: audit 2024-03-04T15:47:33.888585+0000 mon.a (mon.0) 553 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "706cd130-5cc0-41f3-9e78-85922c29a01c"}]: dispatch 2024-03-04T15:47:34.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:34 smithi151 bash[23756]: audit 2024-03-04T15:47:33.892226+0000 mon.a (mon.0) 554 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "706cd130-5cc0-41f3-9e78-85922c29a01c"}]': finished 2024-03-04T15:47:34.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:34 smithi151 bash[23756]: cluster 2024-03-04T15:47:33.898048+0000 mon.a (mon.0) 555 : cluster 0 osdmap e44: 7 total, 6 up, 7 in 2024-03-04T15:47:34.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:34 smithi151 bash[23756]: audit 2024-03-04T15:47:33.898851+0000 mon.a (mon.0) 556 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T15:47:35.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:35 smithi117 bash[21480]: audit 2024-03-04T15:47:34.589193+0000 mon.b (mon.1) 19 : audit 0 from='client.? 172.21.15.151:0/1923300134' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:47:35.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:35 smithi117 bash[28843]: audit 2024-03-04T15:47:34.589193+0000 mon.b (mon.1) 19 : audit 0 from='client.? 172.21.15.151:0/1923300134' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:47:35.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:35 smithi151 bash[23756]: audit 2024-03-04T15:47:34.589193+0000 mon.b (mon.1) 19 : audit 0 from='client.? 172.21.15.151:0/1923300134' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:47:36.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:36 smithi117 bash[21480]: cluster 2024-03-04T15:47:34.804441+0000 mgr.y (mgr.14152) 272 : cluster 0 pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:36.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:36 smithi117 bash[28843]: cluster 2024-03-04T15:47:34.804441+0000 mgr.y (mgr.14152) 272 : cluster 0 pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:36.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:36 smithi151 bash[23756]: cluster 2024-03-04T15:47:34.804441+0000 mgr.y (mgr.14152) 272 : cluster 0 pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:38.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:38 smithi117 bash[21480]: cluster 2024-03-04T15:47:36.805127+0000 mgr.y (mgr.14152) 273 : cluster 0 pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:38.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:38 smithi117 bash[28843]: cluster 2024-03-04T15:47:36.805127+0000 mgr.y (mgr.14152) 273 : cluster 0 pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:38.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:38 smithi151 bash[23756]: cluster 2024-03-04T15:47:36.805127+0000 mgr.y (mgr.14152) 273 : cluster 0 pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:40.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:40 smithi117 bash[21480]: cluster 2024-03-04T15:47:38.805704+0000 mgr.y (mgr.14152) 274 : cluster 0 pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:40.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:40 smithi117 bash[28843]: cluster 2024-03-04T15:47:38.805704+0000 mgr.y (mgr.14152) 274 : cluster 0 pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:40.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:40 smithi151 bash[23756]: cluster 2024-03-04T15:47:38.805704+0000 mgr.y (mgr.14152) 274 : cluster 0 pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:42.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:42 smithi117 bash[21480]: cluster 2024-03-04T15:47:40.806240+0000 mgr.y (mgr.14152) 275 : cluster 0 pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:42.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:42 smithi117 bash[28843]: cluster 2024-03-04T15:47:40.806240+0000 mgr.y (mgr.14152) 275 : cluster 0 pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:42.825 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:42 smithi151 bash[23756]: cluster 2024-03-04T15:47:40.806240+0000 mgr.y (mgr.14152) 275 : cluster 0 pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:44.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:44 smithi117 bash[21480]: cluster 2024-03-04T15:47:42.806796+0000 mgr.y (mgr.14152) 276 : cluster 0 pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:44.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:44 smithi117 bash[28843]: cluster 2024-03-04T15:47:42.806796+0000 mgr.y (mgr.14152) 276 : cluster 0 pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:44.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:44 smithi151 bash[23756]: cluster 2024-03-04T15:47:42.806796+0000 mgr.y (mgr.14152) 276 : cluster 0 pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:46.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:46 smithi117 bash[21480]: cluster 2024-03-04T15:47:44.807330+0000 mgr.y (mgr.14152) 277 : cluster 0 pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:46.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:46 smithi117 bash[28843]: cluster 2024-03-04T15:47:44.807330+0000 mgr.y (mgr.14152) 277 : cluster 0 pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:46.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:46 smithi151 bash[23756]: cluster 2024-03-04T15:47:44.807330+0000 mgr.y (mgr.14152) 277 : cluster 0 pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:47.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:47 smithi117 bash[21480]: audit 2024-03-04T15:47:46.570753+0000 mon.a (mon.0) 557 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-04T15:47:47.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:47 smithi117 bash[21480]: audit 2024-03-04T15:47:46.572596+0000 mon.a (mon.0) 558 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:47:47.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:47 smithi117 bash[21480]: cephadm 2024-03-04T15:47:46.574338+0000 mgr.y (mgr.14152) 278 : cephadm 1 Deploying daemon osd.6 on smithi151 2024-03-04T15:47:47.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:47 smithi117 bash[28843]: audit 2024-03-04T15:47:46.570753+0000 mon.a (mon.0) 557 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-04T15:47:47.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:47 smithi117 bash[28843]: audit 2024-03-04T15:47:46.572596+0000 mon.a (mon.0) 558 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:47:47.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:47 smithi117 bash[28843]: cephadm 2024-03-04T15:47:46.574338+0000 mgr.y (mgr.14152) 278 : cephadm 1 Deploying daemon osd.6 on smithi151 2024-03-04T15:47:47.924 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:47 smithi151 bash[23756]: audit 2024-03-04T15:47:46.570753+0000 mon.a (mon.0) 557 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-03-04T15:47:47.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:47 smithi151 bash[23756]: audit 2024-03-04T15:47:46.572596+0000 mon.a (mon.0) 558 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:47:47.925 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:47 smithi151 bash[23756]: cephadm 2024-03-04T15:47:46.574338+0000 mgr.y (mgr.14152) 278 : cephadm 1 Deploying daemon osd.6 on smithi151 2024-03-04T15:47:48.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:48 smithi117 bash[21480]: cluster 2024-03-04T15:47:46.807812+0000 mgr.y (mgr.14152) 279 : cluster 0 pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:48.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:48 smithi117 bash[28843]: cluster 2024-03-04T15:47:46.807812+0000 mgr.y (mgr.14152) 279 : cluster 0 pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:48.867 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:48 smithi151 bash[23756]: cluster 2024-03-04T15:47:46.807812+0000 mgr.y (mgr.14152) 279 : cluster 0 pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:50.879 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:50 smithi151 bash[23756]: cluster 2024-03-04T15:47:48.808371+0000 mgr.y (mgr.14152) 280 : cluster 0 pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:51.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:50 smithi117 bash[21480]: cluster 2024-03-04T15:47:48.808371+0000 mgr.y (mgr.14152) 280 : cluster 0 pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:51.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:50 smithi117 bash[28843]: cluster 2024-03-04T15:47:48.808371+0000 mgr.y (mgr.14152) 280 : cluster 0 pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:51.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:50 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:47:51.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:51 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:47:51.182 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 15:47:50 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:47:51.182 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 15:47:51 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:47:51.182 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:47:50 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:47:51.183 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:47:51 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:47:51.183 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 15:47:50 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:47:51.183 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 15:47:51 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:47:52.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:52 smithi151 bash[23756]: cluster 2024-03-04T15:47:50.808914+0000 mgr.y (mgr.14152) 281 : cluster 0 pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:52.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:52 smithi151 bash[23756]: audit 2024-03-04T15:47:51.256696+0000 mon.a (mon.0) 559 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:52.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:52 smithi151 bash[23756]: audit 2024-03-04T15:47:51.266029+0000 mon.a (mon.0) 560 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:52.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:52 smithi151 bash[23756]: audit 2024-03-04T15:47:51.278876+0000 mon.a (mon.0) 561 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:47:52.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:52 smithi117 bash[21480]: cluster 2024-03-04T15:47:50.808914+0000 mgr.y (mgr.14152) 281 : cluster 0 pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:52.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:52 smithi117 bash[21480]: audit 2024-03-04T15:47:51.256696+0000 mon.a (mon.0) 559 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:52.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:52 smithi117 bash[21480]: audit 2024-03-04T15:47:51.266029+0000 mon.a (mon.0) 560 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:52.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:52 smithi117 bash[21480]: audit 2024-03-04T15:47:51.278876+0000 mon.a (mon.0) 561 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:47:52.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:52 smithi117 bash[28843]: cluster 2024-03-04T15:47:50.808914+0000 mgr.y (mgr.14152) 281 : cluster 0 pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:52.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:52 smithi117 bash[28843]: audit 2024-03-04T15:47:51.256696+0000 mon.a (mon.0) 559 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:52.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:52 smithi117 bash[28843]: audit 2024-03-04T15:47:51.266029+0000 mon.a (mon.0) 560 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:52.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:52 smithi117 bash[28843]: audit 2024-03-04T15:47:51.278876+0000 mon.a (mon.0) 561 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:47:54.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:54 smithi117 bash[28843]: cluster 2024-03-04T15:47:52.809493+0000 mgr.y (mgr.14152) 282 : cluster 0 pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:54.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:54 smithi117 bash[21480]: cluster 2024-03-04T15:47:52.809493+0000 mgr.y (mgr.14152) 282 : cluster 0 pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:54.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:54 smithi151 bash[23756]: cluster 2024-03-04T15:47:52.809493+0000 mgr.y (mgr.14152) 282 : cluster 0 pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:56.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:56 smithi117 bash[21480]: cluster 2024-03-04T15:47:54.809983+0000 mgr.y (mgr.14152) 283 : cluster 0 pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:56.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:56 smithi117 bash[28843]: cluster 2024-03-04T15:47:54.809983+0000 mgr.y (mgr.14152) 283 : cluster 0 pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:56.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:56 smithi151 bash[23756]: cluster 2024-03-04T15:47:54.809983+0000 mgr.y (mgr.14152) 283 : cluster 0 pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:57.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:57 smithi151 bash[23756]: audit 2024-03-04T15:47:56.626376+0000 mon.a (mon.0) 562 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:57.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:57 smithi151 bash[23756]: audit 2024-03-04T15:47:56.651028+0000 mon.a (mon.0) 563 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:57.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:57 smithi151 bash[23756]: audit 2024-03-04T15:47:57.278055+0000 mon.a (mon.0) 564 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:47:57.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:57 smithi151 bash[23756]: audit 2024-03-04T15:47:57.280040+0000 mon.a (mon.0) 565 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:47:57.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:57 smithi151 bash[23756]: audit 2024-03-04T15:47:57.290296+0000 mon.a (mon.0) 566 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:58.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:57 smithi117 bash[21480]: audit 2024-03-04T15:47:56.626376+0000 mon.a (mon.0) 562 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:58.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:57 smithi117 bash[21480]: audit 2024-03-04T15:47:56.651028+0000 mon.a (mon.0) 563 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:58.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:57 smithi117 bash[21480]: audit 2024-03-04T15:47:57.278055+0000 mon.a (mon.0) 564 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:47:58.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:57 smithi117 bash[21480]: audit 2024-03-04T15:47:57.280040+0000 mon.a (mon.0) 565 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:47:58.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:57 smithi117 bash[21480]: audit 2024-03-04T15:47:57.290296+0000 mon.a (mon.0) 566 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:58.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:57 smithi117 bash[28843]: audit 2024-03-04T15:47:56.626376+0000 mon.a (mon.0) 562 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:58.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:57 smithi117 bash[28843]: audit 2024-03-04T15:47:56.651028+0000 mon.a (mon.0) 563 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:58.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:57 smithi117 bash[28843]: audit 2024-03-04T15:47:57.278055+0000 mon.a (mon.0) 564 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:47:58.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:57 smithi117 bash[28843]: audit 2024-03-04T15:47:57.280040+0000 mon.a (mon.0) 565 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:47:58.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:57 smithi117 bash[28843]: audit 2024-03-04T15:47:57.290296+0000 mon.a (mon.0) 566 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:47:58.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:58 smithi151 bash[23756]: cluster 2024-03-04T15:47:56.810470+0000 mgr.y (mgr.14152) 284 : cluster 0 pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:58.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:58 smithi151 bash[23756]: audit 2024-03-04T15:47:57.873891+0000 mon.b (mon.1) 20 : audit 1 from='osd.6 [v2:172.21.15.151:6816/3501440615,v1:172.21.15.151:6817/3501440615]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-04T15:47:58.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:58 smithi151 bash[23756]: audit 2024-03-04T15:47:57.875084+0000 mon.a (mon.0) 567 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-04T15:47:59.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:58 smithi117 bash[21480]: cluster 2024-03-04T15:47:56.810470+0000 mgr.y (mgr.14152) 284 : cluster 0 pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:59.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:58 smithi117 bash[21480]: audit 2024-03-04T15:47:57.873891+0000 mon.b (mon.1) 20 : audit 1 from='osd.6 [v2:172.21.15.151:6816/3501440615,v1:172.21.15.151:6817/3501440615]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-04T15:47:59.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:58 smithi117 bash[21480]: audit 2024-03-04T15:47:57.875084+0000 mon.a (mon.0) 567 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-04T15:47:59.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:58 smithi117 bash[28843]: cluster 2024-03-04T15:47:56.810470+0000 mgr.y (mgr.14152) 284 : cluster 0 pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:47:59.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:58 smithi117 bash[28843]: audit 2024-03-04T15:47:57.873891+0000 mon.b (mon.1) 20 : audit 1 from='osd.6 [v2:172.21.15.151:6816/3501440615,v1:172.21.15.151:6817/3501440615]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-04T15:47:59.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:58 smithi117 bash[28843]: audit 2024-03-04T15:47:57.875084+0000 mon.a (mon.0) 567 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-03-04T15:47:59.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:59 smithi151 bash[23756]: audit 2024-03-04T15:47:58.655876+0000 mon.a (mon.0) 568 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-04T15:47:59.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:59 smithi151 bash[23756]: cluster 2024-03-04T15:47:58.661381+0000 mon.a (mon.0) 569 : cluster 0 osdmap e45: 7 total, 6 up, 7 in 2024-03-04T15:47:59.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:59 smithi151 bash[23756]: audit 2024-03-04T15:47:58.661919+0000 mon.a (mon.0) 570 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T15:47:59.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:59 smithi151 bash[23756]: audit 2024-03-04T15:47:58.664762+0000 mon.b (mon.1) 21 : audit 1 from='osd.6 [v2:172.21.15.151:6816/3501440615,v1:172.21.15.151:6817/3501440615]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:47:59.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:47:59 smithi151 bash[23756]: audit 2024-03-04T15:47:58.666156+0000 mon.a (mon.0) 571 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:48:00.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:59 smithi117 bash[21480]: audit 2024-03-04T15:47:58.655876+0000 mon.a (mon.0) 568 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-04T15:48:00.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:59 smithi117 bash[21480]: cluster 2024-03-04T15:47:58.661381+0000 mon.a (mon.0) 569 : cluster 0 osdmap e45: 7 total, 6 up, 7 in 2024-03-04T15:48:00.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:59 smithi117 bash[21480]: audit 2024-03-04T15:47:58.661919+0000 mon.a (mon.0) 570 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T15:48:00.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:59 smithi117 bash[21480]: audit 2024-03-04T15:47:58.664762+0000 mon.b (mon.1) 21 : audit 1 from='osd.6 [v2:172.21.15.151:6816/3501440615,v1:172.21.15.151:6817/3501440615]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:48:00.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:47:59 smithi117 bash[21480]: audit 2024-03-04T15:47:58.666156+0000 mon.a (mon.0) 571 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:48:00.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:59 smithi117 bash[28843]: audit 2024-03-04T15:47:58.655876+0000 mon.a (mon.0) 568 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-03-04T15:48:00.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:59 smithi117 bash[28843]: cluster 2024-03-04T15:47:58.661381+0000 mon.a (mon.0) 569 : cluster 0 osdmap e45: 7 total, 6 up, 7 in 2024-03-04T15:48:00.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:59 smithi117 bash[28843]: audit 2024-03-04T15:47:58.661919+0000 mon.a (mon.0) 570 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T15:48:00.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:59 smithi117 bash[28843]: audit 2024-03-04T15:47:58.664762+0000 mon.b (mon.1) 21 : audit 1 from='osd.6 [v2:172.21.15.151:6816/3501440615,v1:172.21.15.151:6817/3501440615]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:48:00.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:47:59 smithi117 bash[28843]: audit 2024-03-04T15:47:58.666156+0000 mon.a (mon.0) 571 : audit 1 from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:48:00.253 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:00.250+0000 7fddd57fa700 1 -- 172.21.15.151:0/238845587 <== mgr.14152 v2:172.21.15.117:6800/2865911351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fddd8062220 con 0x7fddb806d690 2024-03-04T15:48:00.254 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 6 on host 'smithi151' 2024-03-04T15:48:00.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:00.254+0000 7fddde99d700 1 -- 172.21.15.151:0/238845587 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fddb806d690 msgr2=0x7fddb806fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:48:00.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:00.254+0000 7fddde99d700 1 --2- 172.21.15.151:0/238845587 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fddb806d690 0x7fddb806fb50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fddd81050f0 tx=0x7fddc0009040 comp rx=0 tx=0).stop 2024-03-04T15:48:00.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:00.254+0000 7fddde99d700 1 -- 172.21.15.151:0/238845587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddd8105290 msgr2=0x7fddd8199c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:48:00.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:00.254+0000 7fddde99d700 1 --2- 172.21.15.151:0/238845587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddd8105290 0x7fddd8199c70 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fddc800d980 tx=0x7fddc800dd40 comp rx=0 tx=0).stop 2024-03-04T15:48:00.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:00.254+0000 7fddde99d700 1 -- 172.21.15.151:0/238845587 shutdown_connections 2024-03-04T15:48:00.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:00.254+0000 7fddde99d700 1 --2- 172.21.15.151:0/238845587 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fddd8105c30 0x7fddd81a0cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:48:00.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:00.254+0000 7fddde99d700 1 --2- 172.21.15.151:0/238845587 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fddb806d690 0x7fddb806fb50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:48:00.256 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:00.254+0000 7fddde99d700 1 --2- 172.21.15.151:0/238845587 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fddd8104090 0x7fddd8199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:48:00.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:00.254+0000 7fddde99d700 1 --2- 172.21.15.151:0/238845587 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fddd8105290 0x7fddd8199c70 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:48:00.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:00.254+0000 7fddde99d700 1 -- 172.21.15.151:0/238845587 >> 172.21.15.151:0/238845587 conn(0x7fddd80ff670 msgr2=0x7fddd8108660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:48:00.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:00.254+0000 7fddde99d700 1 -- 172.21.15.151:0/238845587 shutdown_connections 2024-03-04T15:48:00.257 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:00.254+0000 7fddde99d700 1 -- 172.21.15.151:0/238845587 wait complete. 2024-03-04T15:48:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:00 smithi151 bash[23756]: cluster 2024-03-04T15:47:58.810990+0000 mgr.y (mgr.14152) 285 : cluster 0 pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:48:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:00 smithi151 bash[23756]: audit 2024-03-04T15:47:59.664522+0000 mon.a (mon.0) 572 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-03-04T15:48:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:00 smithi151 bash[23756]: cluster 2024-03-04T15:47:59.672715+0000 mon.a (mon.0) 573 : cluster 0 osdmap e46: 7 total, 6 up, 7 in 2024-03-04T15:48:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:00 smithi151 bash[23756]: audit 2024-03-04T15:47:59.673570+0000 mon.a (mon.0) 574 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T15:48:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:00 smithi151 bash[23756]: audit 2024-03-04T15:47:59.681283+0000 mon.a (mon.0) 575 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T15:48:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:00 smithi151 bash[23756]: audit 2024-03-04T15:48:00.197401+0000 mon.a (mon.0) 576 : audit 1 from='osd.6 ' entity='osd.6' 2024-03-04T15:48:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:00 smithi151 bash[23756]: audit 2024-03-04T15:48:00.230662+0000 mon.a (mon.0) 577 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:00 smithi151 bash[23756]: audit 2024-03-04T15:48:00.235003+0000 mon.a (mon.0) 578 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:48:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:00 smithi151 bash[23756]: audit 2024-03-04T15:48:00.241094+0000 mon.a (mon.0) 579 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:00.962 DEBUG:teuthology.orchestra.run.smithi151:osd.6> sudo journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.6.service 2024-03-04T15:48:00.965 INFO:tasks.cephadm:Deploying osd.7 on smithi151 with /dev/vg_nvme/lv_1... 2024-03-04T15:48:00.965 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-03-04T15:48:01.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:00 smithi117 bash[21480]: cluster 2024-03-04T15:47:58.810990+0000 mgr.y (mgr.14152) 285 : cluster 0 pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:48:01.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:00 smithi117 bash[21480]: audit 2024-03-04T15:47:59.664522+0000 mon.a (mon.0) 572 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-03-04T15:48:01.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:00 smithi117 bash[21480]: cluster 2024-03-04T15:47:59.672715+0000 mon.a (mon.0) 573 : cluster 0 osdmap e46: 7 total, 6 up, 7 in 2024-03-04T15:48:01.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:00 smithi117 bash[21480]: audit 2024-03-04T15:47:59.673570+0000 mon.a (mon.0) 574 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T15:48:01.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:00 smithi117 bash[21480]: audit 2024-03-04T15:47:59.681283+0000 mon.a (mon.0) 575 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T15:48:01.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:00 smithi117 bash[21480]: audit 2024-03-04T15:48:00.197401+0000 mon.a (mon.0) 576 : audit 1 from='osd.6 ' entity='osd.6' 2024-03-04T15:48:01.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:00 smithi117 bash[21480]: audit 2024-03-04T15:48:00.230662+0000 mon.a (mon.0) 577 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:01.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:00 smithi117 bash[21480]: audit 2024-03-04T15:48:00.235003+0000 mon.a (mon.0) 578 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:48:01.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:00 smithi117 bash[21480]: audit 2024-03-04T15:48:00.241094+0000 mon.a (mon.0) 579 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:01.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:00 smithi117 bash[28843]: cluster 2024-03-04T15:47:58.810990+0000 mgr.y (mgr.14152) 285 : cluster 0 pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:48:01.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:00 smithi117 bash[28843]: audit 2024-03-04T15:47:59.664522+0000 mon.a (mon.0) 572 : audit 1 from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-03-04T15:48:01.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:00 smithi117 bash[28843]: cluster 2024-03-04T15:47:59.672715+0000 mon.a (mon.0) 573 : cluster 0 osdmap e46: 7 total, 6 up, 7 in 2024-03-04T15:48:01.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:00 smithi117 bash[28843]: audit 2024-03-04T15:47:59.673570+0000 mon.a (mon.0) 574 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T15:48:01.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:00 smithi117 bash[28843]: audit 2024-03-04T15:47:59.681283+0000 mon.a (mon.0) 575 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T15:48:01.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:00 smithi117 bash[28843]: audit 2024-03-04T15:48:00.197401+0000 mon.a (mon.0) 576 : audit 1 from='osd.6 ' entity='osd.6' 2024-03-04T15:48:01.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:00 smithi117 bash[28843]: audit 2024-03-04T15:48:00.230662+0000 mon.a (mon.0) 577 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:01.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:00 smithi117 bash[28843]: audit 2024-03-04T15:48:00.235003+0000 mon.a (mon.0) 578 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:48:01.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:00 smithi117 bash[28843]: audit 2024-03-04T15:48:00.241094+0000 mon.a (mon.0) 579 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:02.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:01 smithi117 bash[21480]: cluster 2024-03-04T15:47:58.871477+0000 osd.6 (osd.6) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:48:02.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:01 smithi117 bash[21480]: cluster 2024-03-04T15:47:58.871583+0000 osd.6 (osd.6) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:48:02.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:01 smithi117 bash[21480]: cluster 2024-03-04T15:48:00.680223+0000 mon.a (mon.0) 580 : cluster 1 osd.6 [v2:172.21.15.151:6816/3501440615,v1:172.21.15.151:6817/3501440615] boot 2024-03-04T15:48:02.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:01 smithi117 bash[21480]: cluster 2024-03-04T15:48:00.680320+0000 mon.a (mon.0) 581 : cluster 0 osdmap e47: 7 total, 7 up, 7 in 2024-03-04T15:48:02.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:01 smithi117 bash[21480]: audit 2024-03-04T15:48:00.680631+0000 mon.a (mon.0) 582 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T15:48:02.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:01 smithi117 bash[28843]: cluster 2024-03-04T15:47:58.871477+0000 osd.6 (osd.6) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:48:02.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:01 smithi117 bash[28843]: cluster 2024-03-04T15:47:58.871583+0000 osd.6 (osd.6) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:48:02.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:01 smithi117 bash[28843]: cluster 2024-03-04T15:48:00.680223+0000 mon.a (mon.0) 580 : cluster 1 osd.6 [v2:172.21.15.151:6816/3501440615,v1:172.21.15.151:6817/3501440615] boot 2024-03-04T15:48:02.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:01 smithi117 bash[28843]: cluster 2024-03-04T15:48:00.680320+0000 mon.a (mon.0) 581 : cluster 0 osdmap e47: 7 total, 7 up, 7 in 2024-03-04T15:48:02.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:01 smithi117 bash[28843]: audit 2024-03-04T15:48:00.680631+0000 mon.a (mon.0) 582 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T15:48:02.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:01 smithi151 bash[23756]: cluster 2024-03-04T15:47:58.871477+0000 osd.6 (osd.6) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:48:02.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:01 smithi151 bash[23756]: cluster 2024-03-04T15:47:58.871583+0000 osd.6 (osd.6) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:48:02.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:01 smithi151 bash[23756]: cluster 2024-03-04T15:48:00.680223+0000 mon.a (mon.0) 580 : cluster 1 osd.6 [v2:172.21.15.151:6816/3501440615,v1:172.21.15.151:6817/3501440615] boot 2024-03-04T15:48:02.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:01 smithi151 bash[23756]: cluster 2024-03-04T15:48:00.680320+0000 mon.a (mon.0) 581 : cluster 0 osdmap e47: 7 total, 7 up, 7 in 2024-03-04T15:48:02.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:01 smithi151 bash[23756]: audit 2024-03-04T15:48:00.680631+0000 mon.a (mon.0) 582 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T15:48:03.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:02 smithi117 bash[21480]: cluster 2024-03-04T15:48:00.811625+0000 mgr.y (mgr.14152) 286 : cluster 0 pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:48:03.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:02 smithi117 bash[21480]: cluster 2024-03-04T15:48:01.691294+0000 mon.a (mon.0) 583 : cluster 0 osdmap e48: 7 total, 7 up, 7 in 2024-03-04T15:48:03.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:02 smithi117 bash[28843]: cluster 2024-03-04T15:48:00.811625+0000 mgr.y (mgr.14152) 286 : cluster 0 pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:48:03.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:02 smithi117 bash[28843]: cluster 2024-03-04T15:48:01.691294+0000 mon.a (mon.0) 583 : cluster 0 osdmap e48: 7 total, 7 up, 7 in 2024-03-04T15:48:03.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:02 smithi151 bash[23756]: cluster 2024-03-04T15:48:00.811625+0000 mgr.y (mgr.14152) 286 : cluster 0 pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-03-04T15:48:03.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:02 smithi151 bash[23756]: cluster 2024-03-04T15:48:01.691294+0000 mon.a (mon.0) 583 : cluster 0 osdmap e48: 7 total, 7 up, 7 in 2024-03-04T15:48:04.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:03 smithi117 bash[21480]: cluster 2024-03-04T15:48:02.692319+0000 mon.a (mon.0) 584 : cluster 0 osdmap e49: 7 total, 7 up, 7 in 2024-03-04T15:48:04.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:03 smithi117 bash[28843]: cluster 2024-03-04T15:48:02.692319+0000 mon.a (mon.0) 584 : cluster 0 osdmap e49: 7 total, 7 up, 7 in 2024-03-04T15:48:04.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:03 smithi151 bash[23756]: cluster 2024-03-04T15:48:02.692319+0000 mon.a (mon.0) 584 : cluster 0 osdmap e49: 7 total, 7 up, 7 in 2024-03-04T15:48:05.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:04 smithi117 bash[21480]: cluster 2024-03-04T15:48:02.812209+0000 mgr.y (mgr.14152) 287 : cluster 0 pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:05.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:04 smithi117 bash[21480]: cluster 2024-03-04T15:48:03.714139+0000 mon.a (mon.0) 585 : cluster 0 osdmap e50: 7 total, 7 up, 7 in 2024-03-04T15:48:05.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:04 smithi117 bash[28843]: cluster 2024-03-04T15:48:02.812209+0000 mgr.y (mgr.14152) 287 : cluster 0 pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:05.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:04 smithi117 bash[28843]: cluster 2024-03-04T15:48:03.714139+0000 mon.a (mon.0) 585 : cluster 0 osdmap e50: 7 total, 7 up, 7 in 2024-03-04T15:48:05.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:04 smithi151 bash[23756]: cluster 2024-03-04T15:48:02.812209+0000 mgr.y (mgr.14152) 287 : cluster 0 pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:05.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:04 smithi151 bash[23756]: cluster 2024-03-04T15:48:03.714139+0000 mon.a (mon.0) 585 : cluster 0 osdmap e50: 7 total, 7 up, 7 in 2024-03-04T15:48:05.214 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T15:48:06.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:05 smithi117 bash[21480]: cluster 2024-03-04T15:48:04.713719+0000 mon.a (mon.0) 586 : cluster 0 osdmap e51: 7 total, 7 up, 7 in 2024-03-04T15:48:06.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:05 smithi117 bash[28843]: cluster 2024-03-04T15:48:04.713719+0000 mon.a (mon.0) 586 : cluster 0 osdmap e51: 7 total, 7 up, 7 in 2024-03-04T15:48:06.196 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:05 smithi151 bash[23756]: cluster 2024-03-04T15:48:04.713719+0000 mon.a (mon.0) 586 : cluster 0 osdmap e51: 7 total, 7 up, 7 in 2024-03-04T15:48:07.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:06 smithi117 bash[21480]: cluster 2024-03-04T15:48:04.812679+0000 mgr.y (mgr.14152) 288 : cluster 0 pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:07.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:06 smithi117 bash[28843]: cluster 2024-03-04T15:48:04.812679+0000 mgr.y (mgr.14152) 288 : cluster 0 pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:07.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:06 smithi151 bash[23756]: cluster 2024-03-04T15:48:04.812679+0000 mgr.y (mgr.14152) 288 : cluster 0 pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:09.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:08 smithi117 bash[28843]: cluster 2024-03-04T15:48:06.813299+0000 mgr.y (mgr.14152) 289 : cluster 0 pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-03-04T15:48:09.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:08 smithi117 bash[21480]: cluster 2024-03-04T15:48:06.813299+0000 mgr.y (mgr.14152) 289 : cluster 0 pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-03-04T15:48:09.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:08 smithi151 bash[23756]: cluster 2024-03-04T15:48:06.813299+0000 mgr.y (mgr.14152) 289 : cluster 0 pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail; 112 KiB/s, 0 objects/s recovering 2024-03-04T15:48:09.900 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-03-04T15:48:09.900 INFO:teuthology.orchestra.run.smithi151.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-03-04T15:48:09.900 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10+0 records in 2024-03-04T15:48:09.900 INFO:teuthology.orchestra.run.smithi151.stderr:10+0 records out 2024-03-04T15:48:09.900 INFO:teuthology.orchestra.run.smithi151.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0141801 s, 739 MB/s 2024-03-04T15:48:09.900 INFO:teuthology.orchestra.run.smithi151.stderr:--> Zapping successful for: 2024-03-04T15:48:10.907 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph orch daemon add osd smithi151:vg_nvme/lv_1 2024-03-04T15:48:11.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:10 smithi117 bash[21480]: cluster 2024-03-04T15:48:08.814000+0000 mgr.y (mgr.14152) 290 : cluster 0 pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-04T15:48:11.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:10 smithi117 bash[28843]: cluster 2024-03-04T15:48:08.814000+0000 mgr.y (mgr.14152) 290 : cluster 0 pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-04T15:48:11.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:10 smithi151 bash[23756]: cluster 2024-03-04T15:48:08.814000+0000 mgr.y (mgr.14152) 290 : cluster 0 pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail; 94 KiB/s, 0 objects/s recovering 2024-03-04T15:48:12.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:12 smithi151 bash[23756]: cluster 2024-03-04T15:48:10.814681+0000 mgr.y (mgr.14152) 291 : cluster 0 pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T15:48:12.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:12 smithi151 bash[23756]: cephadm 2024-03-04T15:48:11.331939+0000 mgr.y (mgr.14152) 292 : cephadm 1 Detected new or changed devices on smithi151 2024-03-04T15:48:12.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:12 smithi151 bash[23756]: audit 2024-03-04T15:48:11.340826+0000 mon.a (mon.0) 587 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:12.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:12 smithi151 bash[23756]: audit 2024-03-04T15:48:11.350359+0000 mon.a (mon.0) 588 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:12.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:12 smithi151 bash[23756]: audit 2024-03-04T15:48:11.352794+0000 mon.a (mon.0) 589 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:48:12.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:12 smithi151 bash[23756]: audit 2024-03-04T15:48:11.354572+0000 mon.a (mon.0) 590 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:48:12.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:12 smithi151 bash[23756]: audit 2024-03-04T15:48:11.356164+0000 mon.a (mon.0) 591 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:48:12.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:12 smithi151 bash[23756]: cephadm 2024-03-04T15:48:11.357479+0000 mgr.y (mgr.14152) 293 : cephadm 1 Adjusting osd_memory_target on smithi151 to 5755M 2024-03-04T15:48:12.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:12 smithi151 bash[23756]: audit 2024-03-04T15:48:11.363652+0000 mon.a (mon.0) 592 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:12.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:12 smithi151 bash[23756]: audit 2024-03-04T15:48:11.367911+0000 mon.a (mon.0) 593 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:48:12.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:12 smithi151 bash[23756]: audit 2024-03-04T15:48:11.369561+0000 mon.a (mon.0) 594 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:48:12.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:12 smithi151 bash[23756]: audit 2024-03-04T15:48:11.379432+0000 mon.a (mon.0) 595 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:12.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[28843]: cluster 2024-03-04T15:48:10.814681+0000 mgr.y (mgr.14152) 291 : cluster 0 pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T15:48:12.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[28843]: cephadm 2024-03-04T15:48:11.331939+0000 mgr.y (mgr.14152) 292 : cephadm 1 Detected new or changed devices on smithi151 2024-03-04T15:48:12.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[28843]: audit 2024-03-04T15:48:11.340826+0000 mon.a (mon.0) 587 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:12.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[28843]: audit 2024-03-04T15:48:11.350359+0000 mon.a (mon.0) 588 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:12.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[28843]: audit 2024-03-04T15:48:11.352794+0000 mon.a (mon.0) 589 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:48:12.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[28843]: audit 2024-03-04T15:48:11.354572+0000 mon.a (mon.0) 590 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:48:12.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[28843]: audit 2024-03-04T15:48:11.356164+0000 mon.a (mon.0) 591 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:48:12.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[28843]: cephadm 2024-03-04T15:48:11.357479+0000 mgr.y (mgr.14152) 293 : cephadm 1 Adjusting osd_memory_target on smithi151 to 5755M 2024-03-04T15:48:12.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[28843]: audit 2024-03-04T15:48:11.363652+0000 mon.a (mon.0) 592 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:12.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[28843]: audit 2024-03-04T15:48:11.367911+0000 mon.a (mon.0) 593 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:48:12.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[28843]: audit 2024-03-04T15:48:11.369561+0000 mon.a (mon.0) 594 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:48:12.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[28843]: audit 2024-03-04T15:48:11.379432+0000 mon.a (mon.0) 595 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:12.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[21480]: cluster 2024-03-04T15:48:10.814681+0000 mgr.y (mgr.14152) 291 : cluster 0 pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T15:48:12.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[21480]: cephadm 2024-03-04T15:48:11.331939+0000 mgr.y (mgr.14152) 292 : cephadm 1 Detected new or changed devices on smithi151 2024-03-04T15:48:12.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[21480]: audit 2024-03-04T15:48:11.340826+0000 mon.a (mon.0) 587 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:12.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[21480]: audit 2024-03-04T15:48:11.350359+0000 mon.a (mon.0) 588 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:12.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[21480]: audit 2024-03-04T15:48:11.352794+0000 mon.a (mon.0) 589 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:48:12.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[21480]: audit 2024-03-04T15:48:11.354572+0000 mon.a (mon.0) 590 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:48:12.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[21480]: audit 2024-03-04T15:48:11.356164+0000 mon.a (mon.0) 591 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:48:12.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[21480]: cephadm 2024-03-04T15:48:11.357479+0000 mgr.y (mgr.14152) 293 : cephadm 1 Adjusting osd_memory_target on smithi151 to 5755M 2024-03-04T15:48:12.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[21480]: audit 2024-03-04T15:48:11.363652+0000 mon.a (mon.0) 592 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:12.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[21480]: audit 2024-03-04T15:48:11.367911+0000 mon.a (mon.0) 593 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:48:12.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[21480]: audit 2024-03-04T15:48:11.369561+0000 mon.a (mon.0) 594 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:48:12.823 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:12 smithi117 bash[21480]: audit 2024-03-04T15:48:11.379432+0000 mon.a (mon.0) 595 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:14.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:14 smithi151 bash[23756]: cluster 2024-03-04T15:48:12.815353+0000 mgr.y (mgr.14152) 294 : cluster 0 pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-04T15:48:14.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:14 smithi117 bash[21480]: cluster 2024-03-04T15:48:12.815353+0000 mgr.y (mgr.14152) 294 : cluster 0 pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-04T15:48:14.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:14 smithi117 bash[28843]: cluster 2024-03-04T15:48:12.815353+0000 mgr.y (mgr.14152) 294 : cluster 0 pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-03-04T15:48:15.647 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T15:48:16.631 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:16 smithi151 bash[23756]: cluster 2024-03-04T15:48:14.815957+0000 mgr.y (mgr.14152) 295 : cluster 0 pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-04T15:48:16.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:16 smithi117 bash[21480]: cluster 2024-03-04T15:48:14.815957+0000 mgr.y (mgr.14152) 295 : cluster 0 pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-04T15:48:16.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:16 smithi117 bash[28843]: cluster 2024-03-04T15:48:14.815957+0000 mgr.y (mgr.14152) 295 : cluster 0 pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-03-04T15:48:18.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9ca39700 1 -- 172.21.15.151:0/1723096352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a98101f70 msgr2=0x7f4a981023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:48:18.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9ca39700 1 --2- 172.21.15.151:0/1723096352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a98101f70 0x7f4a981023d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c0099c0 tx=0x7f4a8c009cd0 comp rx=0 tx=0).stop 2024-03-04T15:48:18.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9ca39700 1 -- 172.21.15.151:0/1723096352 shutdown_connections 2024-03-04T15:48:18.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9ca39700 1 --2- 172.21.15.151:0/1723096352 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a98100d70 0x7f4a98101190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:48:18.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9ca39700 1 --2- 172.21.15.151:0/1723096352 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a98102910 0x7f4a981091c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:48:18.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9ca39700 1 --2- 172.21.15.151:0/1723096352 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a98101f70 0x7f4a981023d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:48:18.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9ca39700 1 -- 172.21.15.151:0/1723096352 >> 172.21.15.151:0/1723096352 conn(0x7f4a980fc330 msgr2=0x7f4a980fe750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:48:18.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9ca39700 1 -- 172.21.15.151:0/1723096352 shutdown_connections 2024-03-04T15:48:18.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9ca39700 1 -- 172.21.15.151:0/1723096352 wait complete. 2024-03-04T15:48:18.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9ca39700 1 Processor -- start 2024-03-04T15:48:18.405 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9ca39700 1 -- start start 2024-03-04T15:48:18.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9ca39700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a98100d70 0x7f4a9806b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:48:18.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9ca39700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a98101f70 0x7f4a9819c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:48:18.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9ca39700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a98102910 0x7f4a9819e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:48:18.407 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9ca39700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9810beb0 con 0x7f4a98102910 2024-03-04T15:48:18.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9ca39700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9810bd30 con 0x7f4a98101f70 2024-03-04T15:48:18.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9ca39700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a9810c030 con 0x7f4a98100d70 2024-03-04T15:48:18.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a95d9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a98101f70 0x7f4a9819c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:48:18.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a95d9b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a98101f70 0x7f4a9819c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:44338/0 (socket says 172.21.15.151:44338) 2024-03-04T15:48:18.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a95d9b700 1 -- 172.21.15.151:0/1390650023 learned_addr learned my addr 172.21.15.151:0/1390650023 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-04T15:48:18.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9659c700 1 --2- 172.21.15.151:0/1390650023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a98100d70 0x7f4a9806b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:48:18.408 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a96d9d700 1 --2- 172.21.15.151:0/1390650023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a98102910 0x7f4a9819e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:48:18.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a95d9b700 1 -- 172.21.15.151:0/1390650023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a98100d70 msgr2=0x7f4a9806b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:48:18.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a95d9b700 1 --2- 172.21.15.151:0/1390650023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a98100d70 0x7f4a9806b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:48:18.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a95d9b700 1 -- 172.21.15.151:0/1390650023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a98102910 msgr2=0x7f4a9819e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:48:18.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a95d9b700 1 --2- 172.21.15.151:0/1390650023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a98102910 0x7f4a9819e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:48:18.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a95d9b700 1 -- 172.21.15.151:0/1390650023 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a9819f2a0 con 0x7f4a98101f70 2024-03-04T15:48:18.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9659c700 1 --2- 172.21.15.151:0/1390650023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a98100d70 0x7f4a9806b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T15:48:18.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a96d9d700 1 --2- 172.21.15.151:0/1390650023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a98102910 0x7f4a9819e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T15:48:18.409 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a95d9b700 1 --2- 172.21.15.151:0/1390650023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a98101f70 0x7f4a9819c340 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c006010 tx=0x7f4a8c003950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:48:18.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a877fe700 1 -- 172.21.15.151:0/1390650023 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c003e00 con 0x7f4a98101f70 2024-03-04T15:48:18.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9ca39700 1 -- 172.21.15.151:0/1390650023 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a9819f530 con 0x7f4a98101f70 2024-03-04T15:48:18.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a877fe700 1 -- 172.21.15.151:0/1390650023 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f4a8c004cb0 con 0x7f4a98101f70 2024-03-04T15:48:18.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a877fe700 1 -- 172.21.15.151:0/1390650023 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a8c005590 con 0x7f4a98101f70 2024-03-04T15:48:18.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.402+0000 7f4a9ca39700 1 -- 172.21.15.151:0/1390650023 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a9819f920 con 0x7f4a98101f70 2024-03-04T15:48:18.410 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.406+0000 7f4a9ca39700 1 -- 172.21.15.151:0/1390650023 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a9804faa0 con 0x7f4a98101f70 2024-03-04T15:48:18.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.406+0000 7f4a877fe700 1 -- 172.21.15.151:0/1390650023 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 15) v1 ==== 91397+0+0 (secure 0 0 0) 0x7f4a8c029060 con 0x7f4a98101f70 2024-03-04T15:48:18.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.406+0000 7f4a877fe700 1 --2- 172.21.15.151:0/1390650023 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f4a8006d690 0x7f4a8006fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:48:18.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.406+0000 7f4a877fe700 1 -- 172.21.15.151:0/1390650023 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 4855+0+0 (secure 0 0 0) 0x7f4a8c08ddc0 con 0x7f4a98101f70 2024-03-04T15:48:18.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.406+0000 7f4a9659c700 1 --2- 172.21.15.151:0/1390650023 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f4a8006d690 0x7f4a8006fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:48:18.412 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.406+0000 7f4a9659c700 1 --2- 172.21.15.151:0/1390650023 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f4a8006d690 0x7f4a8006fb50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4a98101dd0 tx=0x7f4a880076a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:48:18.413 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.410+0000 7f4a877fe700 1 -- 172.21.15.151:0/1390650023 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4a8c013070 con 0x7f4a98101f70 2024-03-04T15:48:18.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:18.622+0000 7f4a9ca39700 1 -- 172.21.15.151:0/1390650023 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f4a98062220 con 0x7f4a8006d690 2024-03-04T15:48:18.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:18 smithi151 bash[23756]: cluster 2024-03-04T15:48:16.816576+0000 mgr.y (mgr.14152) 296 : cluster 0 pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-04T15:48:18.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:18 smithi117 bash[28843]: cluster 2024-03-04T15:48:16.816576+0000 mgr.y (mgr.14152) 296 : cluster 0 pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-04T15:48:18.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:18 smithi117 bash[21480]: cluster 2024-03-04T15:48:16.816576+0000 mgr.y (mgr.14152) 296 : cluster 0 pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-04T15:48:19.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:19 smithi151 bash[23756]: audit 2024-03-04T15:48:18.626032+0000 mgr.y (mgr.14152) 297 : audit 0 from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:48:19.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:19 smithi151 bash[23756]: audit 2024-03-04T15:48:18.632986+0000 mon.a (mon.0) 596 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:48:19.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:19 smithi151 bash[23756]: audit 2024-03-04T15:48:18.639614+0000 mon.a (mon.0) 597 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:48:19.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:19 smithi151 bash[23756]: audit 2024-03-04T15:48:18.641353+0000 mon.a (mon.0) 598 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:48:19.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:19 smithi117 bash[21480]: audit 2024-03-04T15:48:18.626032+0000 mgr.y (mgr.14152) 297 : audit 0 from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:48:19.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:19 smithi117 bash[21480]: audit 2024-03-04T15:48:18.632986+0000 mon.a (mon.0) 596 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:48:19.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:19 smithi117 bash[21480]: audit 2024-03-04T15:48:18.639614+0000 mon.a (mon.0) 597 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:48:19.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:19 smithi117 bash[21480]: audit 2024-03-04T15:48:18.641353+0000 mon.a (mon.0) 598 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:48:19.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:19 smithi117 bash[28843]: audit 2024-03-04T15:48:18.626032+0000 mgr.y (mgr.14152) 297 : audit 0 from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi151:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:48:19.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:19 smithi117 bash[28843]: audit 2024-03-04T15:48:18.632986+0000 mon.a (mon.0) 596 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-03-04T15:48:19.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:19 smithi117 bash[28843]: audit 2024-03-04T15:48:18.639614+0000 mon.a (mon.0) 597 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-03-04T15:48:19.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:19 smithi117 bash[28843]: audit 2024-03-04T15:48:18.641353+0000 mon.a (mon.0) 598 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:48:20.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:20 smithi151 bash[23756]: cluster 2024-03-04T15:48:18.817105+0000 mgr.y (mgr.14152) 298 : cluster 0 pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:20.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:20 smithi117 bash[21480]: cluster 2024-03-04T15:48:18.817105+0000 mgr.y (mgr.14152) 298 : cluster 0 pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:20.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:20 smithi117 bash[28843]: cluster 2024-03-04T15:48:18.817105+0000 mgr.y (mgr.14152) 298 : cluster 0 pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:22.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:22 smithi151 bash[23756]: cluster 2024-03-04T15:48:20.817644+0000 mgr.y (mgr.14152) 299 : cluster 0 pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:22.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:22 smithi117 bash[21480]: cluster 2024-03-04T15:48:20.817644+0000 mgr.y (mgr.14152) 299 : cluster 0 pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:22.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:22 smithi117 bash[28843]: cluster 2024-03-04T15:48:20.817644+0000 mgr.y (mgr.14152) 299 : cluster 0 pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:24.675 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:24 smithi151 bash[23756]: cluster 2024-03-04T15:48:22.818212+0000 mgr.y (mgr.14152) 300 : cluster 0 pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:24.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:24 smithi117 bash[21480]: cluster 2024-03-04T15:48:22.818212+0000 mgr.y (mgr.14152) 300 : cluster 0 pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:24.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:24 smithi117 bash[28843]: cluster 2024-03-04T15:48:22.818212+0000 mgr.y (mgr.14152) 300 : cluster 0 pgmap v281: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:26.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:26 smithi151 bash[23756]: cluster 2024-03-04T15:48:24.818797+0000 mgr.y (mgr.14152) 301 : cluster 0 pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:26.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:26 smithi117 bash[21480]: cluster 2024-03-04T15:48:24.818797+0000 mgr.y (mgr.14152) 301 : cluster 0 pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:26.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:26 smithi117 bash[28843]: cluster 2024-03-04T15:48:24.818797+0000 mgr.y (mgr.14152) 301 : cluster 0 pgmap v282: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:27.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:27 smithi151 bash[23756]: audit 2024-03-04T15:48:27.156344+0000 mon.b (mon.1) 22 : audit 1 from='client.? 172.21.15.151:0/3264517034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f49ea27-0d78-4e68-8d06-b3c565e37a58"}]: dispatch 2024-03-04T15:48:27.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:27 smithi151 bash[23756]: audit 2024-03-04T15:48:27.157709+0000 mon.a (mon.0) 599 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f49ea27-0d78-4e68-8d06-b3c565e37a58"}]: dispatch 2024-03-04T15:48:27.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:27 smithi151 bash[23756]: audit 2024-03-04T15:48:27.161239+0000 mon.a (mon.0) 600 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f49ea27-0d78-4e68-8d06-b3c565e37a58"}]': finished 2024-03-04T15:48:27.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:27 smithi151 bash[23756]: cluster 2024-03-04T15:48:27.169938+0000 mon.a (mon.0) 601 : cluster 0 osdmap e52: 8 total, 7 up, 8 in 2024-03-04T15:48:27.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:27 smithi151 bash[23756]: audit 2024-03-04T15:48:27.170635+0000 mon.a (mon.0) 602 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T15:48:27.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:27 smithi117 bash[21480]: audit 2024-03-04T15:48:27.156344+0000 mon.b (mon.1) 22 : audit 1 from='client.? 172.21.15.151:0/3264517034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f49ea27-0d78-4e68-8d06-b3c565e37a58"}]: dispatch 2024-03-04T15:48:27.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:27 smithi117 bash[21480]: audit 2024-03-04T15:48:27.157709+0000 mon.a (mon.0) 599 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f49ea27-0d78-4e68-8d06-b3c565e37a58"}]: dispatch 2024-03-04T15:48:27.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:27 smithi117 bash[21480]: audit 2024-03-04T15:48:27.161239+0000 mon.a (mon.0) 600 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f49ea27-0d78-4e68-8d06-b3c565e37a58"}]': finished 2024-03-04T15:48:27.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:27 smithi117 bash[21480]: cluster 2024-03-04T15:48:27.169938+0000 mon.a (mon.0) 601 : cluster 0 osdmap e52: 8 total, 7 up, 8 in 2024-03-04T15:48:27.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:27 smithi117 bash[21480]: audit 2024-03-04T15:48:27.170635+0000 mon.a (mon.0) 602 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T15:48:27.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:27 smithi117 bash[28843]: audit 2024-03-04T15:48:27.156344+0000 mon.b (mon.1) 22 : audit 1 from='client.? 172.21.15.151:0/3264517034' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f49ea27-0d78-4e68-8d06-b3c565e37a58"}]: dispatch 2024-03-04T15:48:27.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:27 smithi117 bash[28843]: audit 2024-03-04T15:48:27.157709+0000 mon.a (mon.0) 599 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "0f49ea27-0d78-4e68-8d06-b3c565e37a58"}]: dispatch 2024-03-04T15:48:27.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:27 smithi117 bash[28843]: audit 2024-03-04T15:48:27.161239+0000 mon.a (mon.0) 600 : audit 1 from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "0f49ea27-0d78-4e68-8d06-b3c565e37a58"}]': finished 2024-03-04T15:48:27.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:27 smithi117 bash[28843]: cluster 2024-03-04T15:48:27.169938+0000 mon.a (mon.0) 601 : cluster 0 osdmap e52: 8 total, 7 up, 8 in 2024-03-04T15:48:27.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:27 smithi117 bash[28843]: audit 2024-03-04T15:48:27.170635+0000 mon.a (mon.0) 602 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T15:48:28.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:28 smithi151 bash[23756]: cluster 2024-03-04T15:48:26.819379+0000 mgr.y (mgr.14152) 302 : cluster 0 pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:28.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:28 smithi151 bash[23756]: audit 2024-03-04T15:48:27.907693+0000 mon.b (mon.1) 23 : audit 0 from='client.? 172.21.15.151:0/3527428921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:48:28.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:28 smithi117 bash[21480]: cluster 2024-03-04T15:48:26.819379+0000 mgr.y (mgr.14152) 302 : cluster 0 pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:28.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:28 smithi117 bash[21480]: audit 2024-03-04T15:48:27.907693+0000 mon.b (mon.1) 23 : audit 0 from='client.? 172.21.15.151:0/3527428921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:48:28.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:28 smithi117 bash[28843]: cluster 2024-03-04T15:48:26.819379+0000 mgr.y (mgr.14152) 302 : cluster 0 pgmap v283: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:28.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:28 smithi117 bash[28843]: audit 2024-03-04T15:48:27.907693+0000 mon.b (mon.1) 23 : audit 0 from='client.? 172.21.15.151:0/3527428921' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-03-04T15:48:30.657 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:30 smithi151 bash[23756]: cluster 2024-03-04T15:48:28.819930+0000 mgr.y (mgr.14152) 303 : cluster 0 pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:30.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:30 smithi117 bash[21480]: cluster 2024-03-04T15:48:28.819930+0000 mgr.y (mgr.14152) 303 : cluster 0 pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:30.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:30 smithi117 bash[28843]: cluster 2024-03-04T15:48:28.819930+0000 mgr.y (mgr.14152) 303 : cluster 0 pgmap v285: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:32.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:32 smithi151 bash[23756]: cluster 2024-03-04T15:48:30.820471+0000 mgr.y (mgr.14152) 304 : cluster 0 pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:32.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:32 smithi117 bash[28843]: cluster 2024-03-04T15:48:30.820471+0000 mgr.y (mgr.14152) 304 : cluster 0 pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:32 smithi117 bash[21480]: cluster 2024-03-04T15:48:30.820471+0000 mgr.y (mgr.14152) 304 : cluster 0 pgmap v286: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:34.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:34 smithi151 bash[23756]: cluster 2024-03-04T15:48:32.821051+0000 mgr.y (mgr.14152) 305 : cluster 0 pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:34.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:34 smithi117 bash[28843]: cluster 2024-03-04T15:48:32.821051+0000 mgr.y (mgr.14152) 305 : cluster 0 pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:34.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:34 smithi117 bash[21480]: cluster 2024-03-04T15:48:32.821051+0000 mgr.y (mgr.14152) 305 : cluster 0 pgmap v287: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:36.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:36 smithi151 bash[23756]: cluster 2024-03-04T15:48:34.821795+0000 mgr.y (mgr.14152) 306 : cluster 0 pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:36.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:36 smithi117 bash[21480]: cluster 2024-03-04T15:48:34.821795+0000 mgr.y (mgr.14152) 306 : cluster 0 pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:36.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:36 smithi117 bash[28843]: cluster 2024-03-04T15:48:34.821795+0000 mgr.y (mgr.14152) 306 : cluster 0 pgmap v288: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:38.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:38 smithi151 bash[23756]: cluster 2024-03-04T15:48:36.822423+0000 mgr.y (mgr.14152) 307 : cluster 0 pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:38.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:38 smithi117 bash[21480]: cluster 2024-03-04T15:48:36.822423+0000 mgr.y (mgr.14152) 307 : cluster 0 pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:38.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:38 smithi117 bash[28843]: cluster 2024-03-04T15:48:36.822423+0000 mgr.y (mgr.14152) 307 : cluster 0 pgmap v289: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:40.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:40 smithi151 bash[23756]: cluster 2024-03-04T15:48:38.823112+0000 mgr.y (mgr.14152) 308 : cluster 0 pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:40.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:40 smithi151 bash[23756]: audit 2024-03-04T15:48:40.012529+0000 mon.a (mon.0) 603 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-04T15:48:40.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:40 smithi151 bash[23756]: audit 2024-03-04T15:48:40.014179+0000 mon.a (mon.0) 604 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:48:40.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:40 smithi117 bash[21480]: cluster 2024-03-04T15:48:38.823112+0000 mgr.y (mgr.14152) 308 : cluster 0 pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:40.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:40 smithi117 bash[21480]: audit 2024-03-04T15:48:40.012529+0000 mon.a (mon.0) 603 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-04T15:48:40.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:40 smithi117 bash[21480]: audit 2024-03-04T15:48:40.014179+0000 mon.a (mon.0) 604 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:48:40.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:40 smithi117 bash[28843]: cluster 2024-03-04T15:48:38.823112+0000 mgr.y (mgr.14152) 308 : cluster 0 pgmap v290: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:40.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:40 smithi117 bash[28843]: audit 2024-03-04T15:48:40.012529+0000 mon.a (mon.0) 603 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-03-04T15:48:40.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:40 smithi117 bash[28843]: audit 2024-03-04T15:48:40.014179+0000 mon.a (mon.0) 604 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:48:41.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:41 smithi151 bash[23756]: cephadm 2024-03-04T15:48:40.015691+0000 mgr.y (mgr.14152) 309 : cephadm 1 Deploying daemon osd.7 on smithi151 2024-03-04T15:48:41.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:41 smithi117 bash[21480]: cephadm 2024-03-04T15:48:40.015691+0000 mgr.y (mgr.14152) 309 : cephadm 1 Deploying daemon osd.7 on smithi151 2024-03-04T15:48:41.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:41 smithi117 bash[28843]: cephadm 2024-03-04T15:48:40.015691+0000 mgr.y (mgr.14152) 309 : cephadm 1 Deploying daemon osd.7 on smithi151 2024-03-04T15:48:42.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:42 smithi151 bash[23756]: cluster 2024-03-04T15:48:40.823721+0000 mgr.y (mgr.14152) 310 : cluster 0 pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:42.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:42 smithi117 bash[21480]: cluster 2024-03-04T15:48:40.823721+0000 mgr.y (mgr.14152) 310 : cluster 0 pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:42.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:42 smithi117 bash[28843]: cluster 2024-03-04T15:48:40.823721+0000 mgr.y (mgr.14152) 310 : cluster 0 pgmap v291: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:44.504 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 15:48:44 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:48:44.505 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:44 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:48:44.505 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:44 smithi151 bash[23756]: cluster 2024-03-04T15:48:42.824486+0000 mgr.y (mgr.14152) 311 : cluster 0 pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:44.505 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:48:44 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:48:44.505 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 15:48:44 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:48:44.506 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 15:48:44 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:48:44.781 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:44 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:48:44.781 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:48:44 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:48:44.781 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 15:48:44 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:48:44.782 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 15:48:44 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:48:44.782 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 15:48:44 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:48:44.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:44 smithi117 bash[28843]: cluster 2024-03-04T15:48:42.824486+0000 mgr.y (mgr.14152) 311 : cluster 0 pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:44.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:44 smithi117 bash[21480]: cluster 2024-03-04T15:48:42.824486+0000 mgr.y (mgr.14152) 311 : cluster 0 pgmap v292: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:46.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:45 smithi117 bash[21480]: audit 2024-03-04T15:48:44.792527+0000 mon.a (mon.0) 605 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:46.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:45 smithi117 bash[21480]: audit 2024-03-04T15:48:44.804640+0000 mon.a (mon.0) 606 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:46.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:45 smithi117 bash[21480]: audit 2024-03-04T15:48:44.816130+0000 mon.a (mon.0) 607 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:48:46.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:45 smithi117 bash[21480]: cluster 2024-03-04T15:48:44.825108+0000 mgr.y (mgr.14152) 312 : cluster 0 pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:46.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:45 smithi117 bash[28843]: audit 2024-03-04T15:48:44.792527+0000 mon.a (mon.0) 605 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:46.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:45 smithi117 bash[28843]: audit 2024-03-04T15:48:44.804640+0000 mon.a (mon.0) 606 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:46.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:45 smithi117 bash[28843]: audit 2024-03-04T15:48:44.816130+0000 mon.a (mon.0) 607 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:48:46.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:45 smithi117 bash[28843]: cluster 2024-03-04T15:48:44.825108+0000 mgr.y (mgr.14152) 312 : cluster 0 pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:46.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:45 smithi151 bash[23756]: audit 2024-03-04T15:48:44.792527+0000 mon.a (mon.0) 605 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:46.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:45 smithi151 bash[23756]: audit 2024-03-04T15:48:44.804640+0000 mon.a (mon.0) 606 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:46.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:45 smithi151 bash[23756]: audit 2024-03-04T15:48:44.816130+0000 mon.a (mon.0) 607 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:48:46.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:45 smithi151 bash[23756]: cluster 2024-03-04T15:48:44.825108+0000 mgr.y (mgr.14152) 312 : cluster 0 pgmap v293: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:48.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:47 smithi151 bash[23756]: cluster 2024-03-04T15:48:46.825676+0000 mgr.y (mgr.14152) 313 : cluster 0 pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:48.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:47 smithi117 bash[28843]: cluster 2024-03-04T15:48:46.825676+0000 mgr.y (mgr.14152) 313 : cluster 0 pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:48.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:47 smithi117 bash[21480]: cluster 2024-03-04T15:48:46.825676+0000 mgr.y (mgr.14152) 313 : cluster 0 pgmap v294: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:50.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:49 smithi151 bash[23756]: cluster 2024-03-04T15:48:48.826188+0000 mgr.y (mgr.14152) 314 : cluster 0 pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:50.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:49 smithi117 bash[21480]: cluster 2024-03-04T15:48:48.826188+0000 mgr.y (mgr.14152) 314 : cluster 0 pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:50.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:49 smithi117 bash[28843]: cluster 2024-03-04T15:48:48.826188+0000 mgr.y (mgr.14152) 314 : cluster 0 pgmap v295: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:51.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:51 smithi117 bash[21480]: audit 2024-03-04T15:48:50.221263+0000 mon.a (mon.0) 608 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:51.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:51 smithi117 bash[21480]: audit 2024-03-04T15:48:50.231702+0000 mon.a (mon.0) 609 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:51.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:51 smithi117 bash[21480]: audit 2024-03-04T15:48:50.234000+0000 mon.a (mon.0) 610 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:48:51.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:51 smithi117 bash[21480]: audit 2024-03-04T15:48:50.235649+0000 mon.a (mon.0) 611 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:48:51.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:51 smithi117 bash[21480]: audit 2024-03-04T15:48:50.246102+0000 mon.a (mon.0) 612 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:51.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:51 smithi117 bash[28843]: audit 2024-03-04T15:48:50.221263+0000 mon.a (mon.0) 608 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:51.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:51 smithi117 bash[28843]: audit 2024-03-04T15:48:50.231702+0000 mon.a (mon.0) 609 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:51.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:51 smithi117 bash[28843]: audit 2024-03-04T15:48:50.234000+0000 mon.a (mon.0) 610 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:48:51.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:51 smithi117 bash[28843]: audit 2024-03-04T15:48:50.235649+0000 mon.a (mon.0) 611 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:48:51.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:51 smithi117 bash[28843]: audit 2024-03-04T15:48:50.246102+0000 mon.a (mon.0) 612 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:51.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:51 smithi151 bash[23756]: audit 2024-03-04T15:48:50.221263+0000 mon.a (mon.0) 608 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:51.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:51 smithi151 bash[23756]: audit 2024-03-04T15:48:50.231702+0000 mon.a (mon.0) 609 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:51.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:51 smithi151 bash[23756]: audit 2024-03-04T15:48:50.234000+0000 mon.a (mon.0) 610 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:48:51.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:51 smithi151 bash[23756]: audit 2024-03-04T15:48:50.235649+0000 mon.a (mon.0) 611 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:48:51.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:51 smithi151 bash[23756]: audit 2024-03-04T15:48:50.246102+0000 mon.a (mon.0) 612 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:52.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:52 smithi117 bash[21480]: cluster 2024-03-04T15:48:50.826689+0000 mgr.y (mgr.14152) 315 : cluster 0 pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:52.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:52 smithi117 bash[21480]: audit 2024-03-04T15:48:51.457107+0000 mon.b (mon.1) 24 : audit 1 from='osd.7 [v2:172.21.15.151:6824/3965450917,v1:172.21.15.151:6825/3965450917]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-04T15:48:52.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:52 smithi117 bash[21480]: audit 2024-03-04T15:48:51.458774+0000 mon.a (mon.0) 613 : audit 1 from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-04T15:48:52.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:52 smithi117 bash[28843]: cluster 2024-03-04T15:48:50.826689+0000 mgr.y (mgr.14152) 315 : cluster 0 pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:52.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:52 smithi117 bash[28843]: audit 2024-03-04T15:48:51.457107+0000 mon.b (mon.1) 24 : audit 1 from='osd.7 [v2:172.21.15.151:6824/3965450917,v1:172.21.15.151:6825/3965450917]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-04T15:48:52.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:52 smithi117 bash[28843]: audit 2024-03-04T15:48:51.458774+0000 mon.a (mon.0) 613 : audit 1 from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-04T15:48:52.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:52 smithi151 bash[23756]: cluster 2024-03-04T15:48:50.826689+0000 mgr.y (mgr.14152) 315 : cluster 0 pgmap v296: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:52 smithi151 bash[23756]: audit 2024-03-04T15:48:51.457107+0000 mon.b (mon.1) 24 : audit 1 from='osd.7 [v2:172.21.15.151:6824/3965450917,v1:172.21.15.151:6825/3965450917]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-04T15:48:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:52 smithi151 bash[23756]: audit 2024-03-04T15:48:51.458774+0000 mon.a (mon.0) 613 : audit 1 from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-03-04T15:48:53.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:53 smithi117 bash[21480]: audit 2024-03-04T15:48:52.246658+0000 mon.a (mon.0) 614 : audit 1 from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-04T15:48:53.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:53 smithi117 bash[21480]: audit 2024-03-04T15:48:52.252548+0000 mon.b (mon.1) 25 : audit 1 from='osd.7 [v2:172.21.15.151:6824/3965450917,v1:172.21.15.151:6825/3965450917]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:48:53.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:53 smithi117 bash[21480]: cluster 2024-03-04T15:48:52.252856+0000 mon.a (mon.0) 615 : cluster 0 osdmap e53: 8 total, 7 up, 8 in 2024-03-04T15:48:53.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:53 smithi117 bash[21480]: audit 2024-03-04T15:48:52.254262+0000 mon.a (mon.0) 616 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T15:48:53.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:53 smithi117 bash[21480]: audit 2024-03-04T15:48:52.255072+0000 mon.a (mon.0) 617 : audit 1 from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:48:53.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:53 smithi117 bash[21480]: audit 2024-03-04T15:48:53.249200+0000 mon.a (mon.0) 618 : audit 1 from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-03-04T15:48:53.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:53 smithi117 bash[28843]: audit 2024-03-04T15:48:52.246658+0000 mon.a (mon.0) 614 : audit 1 from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-04T15:48:53.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:53 smithi117 bash[28843]: audit 2024-03-04T15:48:52.252548+0000 mon.b (mon.1) 25 : audit 1 from='osd.7 [v2:172.21.15.151:6824/3965450917,v1:172.21.15.151:6825/3965450917]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:48:53.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:53 smithi117 bash[28843]: cluster 2024-03-04T15:48:52.252856+0000 mon.a (mon.0) 615 : cluster 0 osdmap e53: 8 total, 7 up, 8 in 2024-03-04T15:48:53.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:53 smithi117 bash[28843]: audit 2024-03-04T15:48:52.254262+0000 mon.a (mon.0) 616 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T15:48:53.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:53 smithi117 bash[28843]: audit 2024-03-04T15:48:52.255072+0000 mon.a (mon.0) 617 : audit 1 from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:48:53.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:53 smithi117 bash[28843]: audit 2024-03-04T15:48:53.249200+0000 mon.a (mon.0) 618 : audit 1 from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-03-04T15:48:53.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:53 smithi151 bash[23756]: audit 2024-03-04T15:48:52.246658+0000 mon.a (mon.0) 614 : audit 1 from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-03-04T15:48:53.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:53 smithi151 bash[23756]: audit 2024-03-04T15:48:52.252548+0000 mon.b (mon.1) 25 : audit 1 from='osd.7 [v2:172.21.15.151:6824/3965450917,v1:172.21.15.151:6825/3965450917]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:48:53.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:53 smithi151 bash[23756]: cluster 2024-03-04T15:48:52.252856+0000 mon.a (mon.0) 615 : cluster 0 osdmap e53: 8 total, 7 up, 8 in 2024-03-04T15:48:53.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:53 smithi151 bash[23756]: audit 2024-03-04T15:48:52.254262+0000 mon.a (mon.0) 616 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T15:48:53.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:53 smithi151 bash[23756]: audit 2024-03-04T15:48:52.255072+0000 mon.a (mon.0) 617 : audit 1 from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]: dispatch 2024-03-04T15:48:53.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:53 smithi151 bash[23756]: audit 2024-03-04T15:48:53.249200+0000 mon.a (mon.0) 618 : audit 1 from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi151", "root=default"]}]': finished 2024-03-04T15:48:54.305 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:54.301+0000 7f4a877fe700 1 -- 172.21.15.151:0/1390650023 <== mgr.14152 v2:172.21.15.117:6800/2865911351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f4a98062220 con 0x7f4a8006d690 2024-03-04T15:48:54.306 INFO:teuthology.orchestra.run.smithi151.stdout:Created osd(s) 7 on host 'smithi151' 2024-03-04T15:48:54.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:54.305+0000 7f4a9ca39700 1 -- 172.21.15.151:0/1390650023 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f4a8006d690 msgr2=0x7f4a8006fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:48:54.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:54.305+0000 7f4a9ca39700 1 --2- 172.21.15.151:0/1390650023 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f4a8006d690 0x7f4a8006fb50 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4a98101dd0 tx=0x7f4a880076a0 comp rx=0 tx=0).stop 2024-03-04T15:48:54.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:54.305+0000 7f4a9ca39700 1 -- 172.21.15.151:0/1390650023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a98101f70 msgr2=0x7f4a9819c340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:48:54.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:54.305+0000 7f4a9ca39700 1 --2- 172.21.15.151:0/1390650023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a98101f70 0x7f4a9819c340 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f4a8c006010 tx=0x7f4a8c003950 comp rx=0 tx=0).stop 2024-03-04T15:48:54.308 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:54.305+0000 7f4a9ca39700 1 -- 172.21.15.151:0/1390650023 shutdown_connections 2024-03-04T15:48:54.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:54.305+0000 7f4a9ca39700 1 --2- 172.21.15.151:0/1390650023 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4a98102910 0x7f4a9819e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:48:54.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:54.305+0000 7f4a9ca39700 1 --2- 172.21.15.151:0/1390650023 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f4a8006d690 0x7f4a8006fb50 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:48:54.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:54.305+0000 7f4a9ca39700 1 --2- 172.21.15.151:0/1390650023 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4a98100d70 0x7f4a9806b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:48:54.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:54.305+0000 7f4a9ca39700 1 --2- 172.21.15.151:0/1390650023 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4a98101f70 0x7f4a9819c340 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:48:54.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:54.305+0000 7f4a9ca39700 1 -- 172.21.15.151:0/1390650023 >> 172.21.15.151:0/1390650023 conn(0x7f4a980fc330 msgr2=0x7f4a980fde30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:48:54.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:54.305+0000 7f4a9ca39700 1 -- 172.21.15.151:0/1390650023 shutdown_connections 2024-03-04T15:48:54.309 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:48:54.305+0000 7f4a9ca39700 1 -- 172.21.15.151:0/1390650023 wait complete. 2024-03-04T15:48:54.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[21480]: cluster 2024-03-04T15:48:52.827189+0000 mgr.y (mgr.14152) 316 : cluster 0 pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:54.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[21480]: cluster 2024-03-04T15:48:53.257489+0000 mon.a (mon.0) 619 : cluster 0 osdmap e54: 8 total, 7 up, 8 in 2024-03-04T15:48:54.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[21480]: audit 2024-03-04T15:48:53.259002+0000 mon.a (mon.0) 620 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T15:48:54.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[21480]: audit 2024-03-04T15:48:53.763575+0000 mon.a (mon.0) 621 : audit 1 from='osd.7 ' entity='osd.7' 2024-03-04T15:48:54.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[21480]: audit 2024-03-04T15:48:54.247694+0000 mon.a (mon.0) 622 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:48:54.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[21480]: audit 2024-03-04T15:48:54.255563+0000 mon.a (mon.0) 623 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:54.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[21480]: audit 2024-03-04T15:48:54.259973+0000 mon.a (mon.0) 624 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T15:48:54.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[21480]: cluster 2024-03-04T15:48:54.266718+0000 mon.a (mon.0) 625 : cluster 1 osd.7 [v2:172.21.15.151:6824/3965450917,v1:172.21.15.151:6825/3965450917] boot 2024-03-04T15:48:54.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[21480]: cluster 2024-03-04T15:48:54.266894+0000 mon.a (mon.0) 626 : cluster 0 osdmap e55: 8 total, 8 up, 8 in 2024-03-04T15:48:54.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[21480]: audit 2024-03-04T15:48:54.267802+0000 mon.a (mon.0) 627 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T15:48:54.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[21480]: audit 2024-03-04T15:48:54.277562+0000 mon.a (mon.0) 628 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:54.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[28843]: cluster 2024-03-04T15:48:52.827189+0000 mgr.y (mgr.14152) 316 : cluster 0 pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:54.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[28843]: cluster 2024-03-04T15:48:53.257489+0000 mon.a (mon.0) 619 : cluster 0 osdmap e54: 8 total, 7 up, 8 in 2024-03-04T15:48:54.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[28843]: audit 2024-03-04T15:48:53.259002+0000 mon.a (mon.0) 620 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T15:48:54.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[28843]: audit 2024-03-04T15:48:53.763575+0000 mon.a (mon.0) 621 : audit 1 from='osd.7 ' entity='osd.7' 2024-03-04T15:48:54.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[28843]: audit 2024-03-04T15:48:54.247694+0000 mon.a (mon.0) 622 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:48:54.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[28843]: audit 2024-03-04T15:48:54.255563+0000 mon.a (mon.0) 623 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:54.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[28843]: audit 2024-03-04T15:48:54.259973+0000 mon.a (mon.0) 624 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T15:48:54.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[28843]: cluster 2024-03-04T15:48:54.266718+0000 mon.a (mon.0) 625 : cluster 1 osd.7 [v2:172.21.15.151:6824/3965450917,v1:172.21.15.151:6825/3965450917] boot 2024-03-04T15:48:54.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[28843]: cluster 2024-03-04T15:48:54.266894+0000 mon.a (mon.0) 626 : cluster 0 osdmap e55: 8 total, 8 up, 8 in 2024-03-04T15:48:54.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[28843]: audit 2024-03-04T15:48:54.267802+0000 mon.a (mon.0) 627 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T15:48:54.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:54 smithi117 bash[28843]: audit 2024-03-04T15:48:54.277562+0000 mon.a (mon.0) 628 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:54.612 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:54 smithi151 bash[23756]: cluster 2024-03-04T15:48:52.827189+0000 mgr.y (mgr.14152) 316 : cluster 0 pgmap v298: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:54.612 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:54 smithi151 bash[23756]: cluster 2024-03-04T15:48:53.257489+0000 mon.a (mon.0) 619 : cluster 0 osdmap e54: 8 total, 7 up, 8 in 2024-03-04T15:48:54.612 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:54 smithi151 bash[23756]: audit 2024-03-04T15:48:53.259002+0000 mon.a (mon.0) 620 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T15:48:54.612 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:54 smithi151 bash[23756]: audit 2024-03-04T15:48:53.763575+0000 mon.a (mon.0) 621 : audit 1 from='osd.7 ' entity='osd.7' 2024-03-04T15:48:54.613 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:54 smithi151 bash[23756]: audit 2024-03-04T15:48:54.247694+0000 mon.a (mon.0) 622 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:48:54.613 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:54 smithi151 bash[23756]: audit 2024-03-04T15:48:54.255563+0000 mon.a (mon.0) 623 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:54.613 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:54 smithi151 bash[23756]: audit 2024-03-04T15:48:54.259973+0000 mon.a (mon.0) 624 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T15:48:54.613 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:54 smithi151 bash[23756]: cluster 2024-03-04T15:48:54.266718+0000 mon.a (mon.0) 625 : cluster 1 osd.7 [v2:172.21.15.151:6824/3965450917,v1:172.21.15.151:6825/3965450917] boot 2024-03-04T15:48:54.613 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:54 smithi151 bash[23756]: cluster 2024-03-04T15:48:54.266894+0000 mon.a (mon.0) 626 : cluster 0 osdmap e55: 8 total, 8 up, 8 in 2024-03-04T15:48:54.613 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:54 smithi151 bash[23756]: audit 2024-03-04T15:48:54.267802+0000 mon.a (mon.0) 627 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T15:48:54.613 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:54 smithi151 bash[23756]: audit 2024-03-04T15:48:54.277562+0000 mon.a (mon.0) 628 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:48:55.138 DEBUG:teuthology.orchestra.run.smithi151:osd.7> sudo journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.7.service 2024-03-04T15:48:55.141 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-03-04T15:48:55.141 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph osd stat -f json 2024-03-04T15:48:55.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:55 smithi151 bash[23756]: cluster 2024-03-04T15:48:52.475841+0000 osd.7 (osd.7) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:48:55.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:55 smithi151 bash[23756]: cluster 2024-03-04T15:48:52.475962+0000 osd.7 (osd.7) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:48:55.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:55 smithi151 bash[23756]: cluster 2024-03-04T15:48:55.279621+0000 mon.a (mon.0) 629 : cluster 0 osdmap e56: 8 total, 8 up, 8 in 2024-03-04T15:48:55.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:55 smithi117 bash[21480]: cluster 2024-03-04T15:48:52.475841+0000 osd.7 (osd.7) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:48:55.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:55 smithi117 bash[21480]: cluster 2024-03-04T15:48:52.475962+0000 osd.7 (osd.7) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:48:55.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:55 smithi117 bash[21480]: cluster 2024-03-04T15:48:55.279621+0000 mon.a (mon.0) 629 : cluster 0 osdmap e56: 8 total, 8 up, 8 in 2024-03-04T15:48:55.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:55 smithi117 bash[28843]: cluster 2024-03-04T15:48:52.475841+0000 osd.7 (osd.7) 1 : cluster 0 purged_snaps scrub starts 2024-03-04T15:48:55.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:55 smithi117 bash[28843]: cluster 2024-03-04T15:48:52.475962+0000 osd.7 (osd.7) 2 : cluster 0 purged_snaps scrub ok 2024-03-04T15:48:55.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:55 smithi117 bash[28843]: cluster 2024-03-04T15:48:55.279621+0000 mon.a (mon.0) 629 : cluster 0 osdmap e56: 8 total, 8 up, 8 in 2024-03-04T15:48:56.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:56 smithi117 bash[21480]: cluster 2024-03-04T15:48:54.827749+0000 mgr.y (mgr.14152) 317 : cluster 0 pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:56.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:56 smithi117 bash[21480]: cluster 2024-03-04T15:48:56.282080+0000 mon.a (mon.0) 630 : cluster 0 osdmap e57: 8 total, 8 up, 8 in 2024-03-04T15:48:56.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:56 smithi117 bash[28843]: cluster 2024-03-04T15:48:54.827749+0000 mgr.y (mgr.14152) 317 : cluster 0 pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:56.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:56 smithi117 bash[28843]: cluster 2024-03-04T15:48:56.282080+0000 mon.a (mon.0) 630 : cluster 0 osdmap e57: 8 total, 8 up, 8 in 2024-03-04T15:48:56.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:56 smithi151 bash[23756]: cluster 2024-03-04T15:48:54.827749+0000 mgr.y (mgr.14152) 317 : cluster 0 pgmap v301: 1 pgs: 1 active+clean; 577 KiB data, 214 MiB used, 626 GiB / 626 GiB avail 2024-03-04T15:48:56.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:56 smithi151 bash[23756]: cluster 2024-03-04T15:48:56.282080+0000 mon.a (mon.0) 630 : cluster 0 osdmap e57: 8 total, 8 up, 8 in 2024-03-04T15:48:58.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:57 smithi117 bash[21480]: cluster 2024-03-04T15:48:56.779334+0000 mon.a (mon.0) 631 : cluster 0 osdmap e58: 8 total, 8 up, 8 in 2024-03-04T15:48:58.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:57 smithi117 bash[21480]: cluster 2024-03-04T15:48:56.828282+0000 mgr.y (mgr.14152) 318 : cluster 0 pgmap v305: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:48:58.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:57 smithi117 bash[28843]: cluster 2024-03-04T15:48:56.779334+0000 mon.a (mon.0) 631 : cluster 0 osdmap e58: 8 total, 8 up, 8 in 2024-03-04T15:48:58.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:57 smithi117 bash[28843]: cluster 2024-03-04T15:48:56.828282+0000 mgr.y (mgr.14152) 318 : cluster 0 pgmap v305: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:48:58.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:57 smithi151 bash[23756]: cluster 2024-03-04T15:48:56.779334+0000 mon.a (mon.0) 631 : cluster 0 osdmap e58: 8 total, 8 up, 8 in 2024-03-04T15:48:58.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:57 smithi151 bash[23756]: cluster 2024-03-04T15:48:56.828282+0000 mgr.y (mgr.14152) 318 : cluster 0 pgmap v305: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:48:59.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:58 smithi117 bash[21480]: cluster 2024-03-04T15:48:57.788828+0000 mon.a (mon.0) 632 : cluster 0 osdmap e59: 8 total, 8 up, 8 in 2024-03-04T15:48:59.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:58 smithi117 bash[28843]: cluster 2024-03-04T15:48:57.788828+0000 mon.a (mon.0) 632 : cluster 0 osdmap e59: 8 total, 8 up, 8 in 2024-03-04T15:48:59.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:58 smithi151 bash[23756]: cluster 2024-03-04T15:48:57.788828+0000 mon.a (mon.0) 632 : cluster 0 osdmap e59: 8 total, 8 up, 8 in 2024-03-04T15:48:59.903 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:49:00.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:48:59 smithi117 bash[21480]: cluster 2024-03-04T15:48:58.828881+0000 mgr.y (mgr.14152) 319 : cluster 0 pgmap v307: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:00.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:48:59 smithi117 bash[28843]: cluster 2024-03-04T15:48:58.828881+0000 mgr.y (mgr.14152) 319 : cluster 0 pgmap v307: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:00.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:48:59 smithi151 bash[23756]: cluster 2024-03-04T15:48:58.828881+0000 mgr.y (mgr.14152) 319 : cluster 0 pgmap v307: 1 pgs: 1 remapped+peering; 577 KiB data, 245 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:02.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:01 smithi151 bash[23756]: cluster 2024-03-04T15:49:00.829451+0000 mgr.y (mgr.14152) 320 : cluster 0 pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-03-04T15:49:02.213 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:01 smithi117 bash[21480]: cluster 2024-03-04T15:49:00.829451+0000 mgr.y (mgr.14152) 320 : cluster 0 pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-03-04T15:49:02.214 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:01 smithi117 bash[28843]: cluster 2024-03-04T15:49:00.829451+0000 mgr.y (mgr.14152) 320 : cluster 0 pgmap v308: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 104 KiB/s, 0 objects/s recovering 2024-03-04T15:49:02.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f91ab0700 1 -- 172.21.15.117:0/3063780105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f8c105c30 msgr2=0x7f2f8c10c4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:02.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f91ab0700 1 --2- 172.21.15.117:0/3063780105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f8c105c30 0x7f2f8c10c4e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2f8400b0a0 tx=0x7f2f8400b3b0 comp rx=0 tx=0).stop 2024-03-04T15:49:02.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f91ab0700 1 -- 172.21.15.117:0/3063780105 shutdown_connections 2024-03-04T15:49:02.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f91ab0700 1 --2- 172.21.15.117:0/3063780105 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f8c105c30 0x7f2f8c10c4e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:02.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f91ab0700 1 --2- 172.21.15.117:0/3063780105 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f8c104090 0x7f2f8c1044b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:02.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f91ab0700 1 --2- 172.21.15.117:0/3063780105 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f8c105290 0x7f2f8c1056f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:02.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f91ab0700 1 -- 172.21.15.117:0/3063780105 >> 172.21.15.117:0/3063780105 conn(0x7f2f8c0ff630 msgr2=0x7f2f8c101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:49:02.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f91ab0700 1 -- 172.21.15.117:0/3063780105 shutdown_connections 2024-03-04T15:49:02.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f91ab0700 1 -- 172.21.15.117:0/3063780105 wait complete. 2024-03-04T15:49:02.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f91ab0700 1 Processor -- start 2024-03-04T15:49:02.394 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f91ab0700 1 -- start start 2024-03-04T15:49:02.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f91ab0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f8c104090 0x7f2f8c1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:02.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f91ab0700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f8c105290 0x7f2f8c199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:02.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f91ab0700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f8c105c30 0x7f2f8c1a0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:02.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f91ab0700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c10f1c0 con 0x7f2f8c104090 2024-03-04T15:49:02.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f91ab0700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c10f040 con 0x7f2f8c105c30 2024-03-04T15:49:02.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f91ab0700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f8c10f340 con 0x7f2f8c105290 2024-03-04T15:49:02.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f8affd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f8c105290 0x7f2f8c199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:02.395 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f8b7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f8c104090 0x7f2f8c1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:02.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f8affd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f8c105290 0x7f2f8c199ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:57494/0 (socket says 172.21.15.117:57494) 2024-03-04T15:49:02.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f8affd700 1 -- 172.21.15.117:0/962387920 learned_addr learned my addr 172.21.15.117:0/962387920 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:49:02.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f8bfff700 1 --2- 172.21.15.117:0/962387920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f8c105c30 0x7f2f8c1a0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:02.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f8affd700 1 -- 172.21.15.117:0/962387920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f8c105c30 msgr2=0x7f2f8c1a0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:02.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f8affd700 1 --2- 172.21.15.117:0/962387920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f8c105c30 0x7f2f8c1a0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:02.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f8affd700 1 -- 172.21.15.117:0/962387920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f8c104090 msgr2=0x7f2f8c1997a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:02.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f8affd700 1 --2- 172.21.15.117:0/962387920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f8c104090 0x7f2f8c1997a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:02.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f8affd700 1 -- 172.21.15.117:0/962387920 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f8c1a1400 con 0x7f2f8c105290 2024-03-04T15:49:02.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f8bfff700 1 --2- 172.21.15.117:0/962387920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f8c105c30 0x7f2f8c1a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T15:49:02.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f8b7fe700 1 --2- 172.21.15.117:0/962387920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f8c104090 0x7f2f8c1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:49:02.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f8affd700 1 --2- 172.21.15.117:0/962387920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f8c105290 0x7f2f8c199ce0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400e9d0 tx=0x7f2f7400ece0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:49:02.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f88ff9700 1 -- 172.21.15.117:0/962387920 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f7400cbd0 con 0x7f2f8c105290 2024-03-04T15:49:02.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f88ff9700 1 -- 172.21.15.117:0/962387920 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f2f7400cd70 con 0x7f2f8c105290 2024-03-04T15:49:02.396 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f88ff9700 1 -- 172.21.15.117:0/962387920 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f74018680 con 0x7f2f8c105290 2024-03-04T15:49:02.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f91ab0700 1 -- 172.21.15.117:0/962387920 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f8c1a16f0 con 0x7f2f8c105290 2024-03-04T15:49:02.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.389+0000 7f2f91ab0700 1 -- 172.21.15.117:0/962387920 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f8c1a1be0 con 0x7f2f8c105290 2024-03-04T15:49:02.397 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.393+0000 7f2f91ab0700 1 -- 172.21.15.117:0/962387920 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f8c04faa0 con 0x7f2f8c105290 2024-03-04T15:49:02.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.393+0000 7f2f88ff9700 1 -- 172.21.15.117:0/962387920 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 15) v1 ==== 91397+0+0 (secure 0 0 0) 0x7f2f74010430 con 0x7f2f8c105290 2024-03-04T15:49:02.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.393+0000 7f2f88ff9700 1 --2- 172.21.15.117:0/962387920 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f2f7806d690 0x7f2f7806fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:02.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.393+0000 7f2f88ff9700 1 -- 172.21.15.117:0/962387920 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5287+0+0 (secure 0 0 0) 0x7f2f7408d2a0 con 0x7f2f8c105290 2024-03-04T15:49:02.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.393+0000 7f2f8b7fe700 1 --2- 172.21.15.117:0/962387920 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f2f7806d690 0x7f2f7806fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:02.398 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.393+0000 7f2f8b7fe700 1 --2- 172.21.15.117:0/962387920 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f2f7806d690 0x7f2f7806fb50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c005950 tx=0x7f2f7c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:49:02.401 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.397+0000 7f2f88ff9700 1 -- 172.21.15.117:0/962387920 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2f7405b3c0 con 0x7f2f8c105290 2024-03-04T15:49:02.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.605+0000 7f2f91ab0700 1 -- 172.21.15.117:0/962387920 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f2f8c02d830 con 0x7f2f8c105290 2024-03-04T15:49:02.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.605+0000 7f2f88ff9700 1 -- 172.21.15.117:0/962387920 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v59) v1 ==== 74+0+130 (secure 0 0 0) 0x7f2f74010e20 con 0x7f2f8c105290 2024-03-04T15:49:02.610 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-04T15:49:02.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.609+0000 7f2f91ab0700 1 -- 172.21.15.117:0/962387920 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f2f7806d690 msgr2=0x7f2f7806fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:02.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.609+0000 7f2f91ab0700 1 --2- 172.21.15.117:0/962387920 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f2f7806d690 0x7f2f7806fb50 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f2f7c005950 tx=0x7f2f7c00a300 comp rx=0 tx=0).stop 2024-03-04T15:49:02.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.609+0000 7f2f91ab0700 1 -- 172.21.15.117:0/962387920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f8c105290 msgr2=0x7f2f8c199ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:02.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.609+0000 7f2f91ab0700 1 --2- 172.21.15.117:0/962387920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f8c105290 0x7f2f8c199ce0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2f7400e9d0 tx=0x7f2f7400ece0 comp rx=0 tx=0).stop 2024-03-04T15:49:02.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.609+0000 7f2f91ab0700 1 -- 172.21.15.117:0/962387920 shutdown_connections 2024-03-04T15:49:02.613 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.609+0000 7f2f91ab0700 1 --2- 172.21.15.117:0/962387920 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2f8c104090 0x7f2f8c1997a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:02.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.609+0000 7f2f91ab0700 1 --2- 172.21.15.117:0/962387920 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f2f7806d690 0x7f2f7806fb50 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:02.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.609+0000 7f2f91ab0700 1 --2- 172.21.15.117:0/962387920 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2f8c105c30 0x7f2f8c1a0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:02.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.609+0000 7f2f91ab0700 1 --2- 172.21.15.117:0/962387920 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2f8c105290 0x7f2f8c199ce0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:02.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.609+0000 7f2f91ab0700 1 -- 172.21.15.117:0/962387920 >> 172.21.15.117:0/962387920 conn(0x7f2f8c0ff630 msgr2=0x7f2f8c108590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:49:02.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.609+0000 7f2f91ab0700 1 -- 172.21.15.117:0/962387920 shutdown_connections 2024-03-04T15:49:02.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:02.609+0000 7f2f91ab0700 1 -- 172.21.15.117:0/962387920 wait complete. 2024-03-04T15:49:03.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:02 smithi151 bash[23756]: audit 2024-03-04T15:49:02.611486+0000 mon.c (mon.2) 12 : audit 0 from='client.? 172.21.15.117:0/962387920' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-04T15:49:03.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:02 smithi117 bash[21480]: audit 2024-03-04T15:49:02.611486+0000 mon.c (mon.2) 12 : audit 0 from='client.? 172.21.15.117:0/962387920' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-04T15:49:03.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:02 smithi117 bash[28843]: audit 2024-03-04T15:49:02.611486+0000 mon.c (mon.2) 12 : audit 0 from='client.? 172.21.15.117:0/962387920' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-03-04T15:49:03.324 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":59,"num_osds":8,"num_up_osds":8,"osd_up_since":1709567334,"num_in_osds":8,"osd_in_since":1709567307,"num_remapped_pgs":0} 2024-03-04T15:49:03.325 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-04T15:49:04.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:03 smithi151 bash[23756]: cluster 2024-03-04T15:49:02.829971+0000 mgr.y (mgr.14152) 321 : cluster 0 pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-04T15:49:04.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:03 smithi117 bash[21480]: cluster 2024-03-04T15:49:02.829971+0000 mgr.y (mgr.14152) 321 : cluster 0 pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-04T15:49:04.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:03 smithi117 bash[28843]: cluster 2024-03-04T15:49:02.829971+0000 mgr.y (mgr.14152) 321 : cluster 0 pgmap v309: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 88 KiB/s, 0 objects/s recovering 2024-03-04T15:49:05.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:05 smithi151 bash[23756]: cephadm 2024-03-04T15:49:04.089683+0000 mgr.y (mgr.14152) 322 : cephadm 1 Detected new or changed devices on smithi151 2024-03-04T15:49:05.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:05 smithi151 bash[23756]: audit 2024-03-04T15:49:04.100212+0000 mon.a (mon.0) 633 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:05.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:05 smithi151 bash[23756]: audit 2024-03-04T15:49:04.110444+0000 mon.a (mon.0) 634 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:05.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:05 smithi151 bash[23756]: audit 2024-03-04T15:49:04.113251+0000 mon.a (mon.0) 635 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:49:05.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:05 smithi151 bash[23756]: audit 2024-03-04T15:49:04.115336+0000 mon.a (mon.0) 636 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:49:05.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:05 smithi151 bash[23756]: audit 2024-03-04T15:49:04.117494+0000 mon.a (mon.0) 637 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:49:05.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:05 smithi151 bash[23756]: audit 2024-03-04T15:49:04.119548+0000 mon.a (mon.0) 638 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:49:05.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:05 smithi151 bash[23756]: cephadm 2024-03-04T15:49:04.121063+0000 mgr.y (mgr.14152) 323 : cephadm 1 Adjusting osd_memory_target on smithi151 to 4316M 2024-03-04T15:49:05.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:05 smithi151 bash[23756]: audit 2024-03-04T15:49:04.127812+0000 mon.a (mon.0) 639 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:05.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:05 smithi151 bash[23756]: audit 2024-03-04T15:49:04.132134+0000 mon.a (mon.0) 640 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:05.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:05 smithi151 bash[23756]: audit 2024-03-04T15:49:04.133974+0000 mon.a (mon.0) 641 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:49:05.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:05 smithi151 bash[23756]: audit 2024-03-04T15:49:04.143743+0000 mon.a (mon.0) 642 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:05.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[28843]: cephadm 2024-03-04T15:49:04.089683+0000 mgr.y (mgr.14152) 322 : cephadm 1 Detected new or changed devices on smithi151 2024-03-04T15:49:05.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[28843]: audit 2024-03-04T15:49:04.100212+0000 mon.a (mon.0) 633 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:05.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[28843]: audit 2024-03-04T15:49:04.110444+0000 mon.a (mon.0) 634 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:05.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[28843]: audit 2024-03-04T15:49:04.113251+0000 mon.a (mon.0) 635 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:49:05.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[28843]: audit 2024-03-04T15:49:04.115336+0000 mon.a (mon.0) 636 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:49:05.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[28843]: audit 2024-03-04T15:49:04.117494+0000 mon.a (mon.0) 637 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:49:05.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[28843]: audit 2024-03-04T15:49:04.119548+0000 mon.a (mon.0) 638 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:49:05.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[28843]: cephadm 2024-03-04T15:49:04.121063+0000 mgr.y (mgr.14152) 323 : cephadm 1 Adjusting osd_memory_target on smithi151 to 4316M 2024-03-04T15:49:05.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[28843]: audit 2024-03-04T15:49:04.127812+0000 mon.a (mon.0) 639 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:05.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[28843]: audit 2024-03-04T15:49:04.132134+0000 mon.a (mon.0) 640 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:05.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[28843]: audit 2024-03-04T15:49:04.133974+0000 mon.a (mon.0) 641 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:49:05.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[28843]: audit 2024-03-04T15:49:04.143743+0000 mon.a (mon.0) 642 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:05.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[21480]: cephadm 2024-03-04T15:49:04.089683+0000 mgr.y (mgr.14152) 322 : cephadm 1 Detected new or changed devices on smithi151 2024-03-04T15:49:05.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[21480]: audit 2024-03-04T15:49:04.100212+0000 mon.a (mon.0) 633 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:05.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[21480]: audit 2024-03-04T15:49:04.110444+0000 mon.a (mon.0) 634 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:05.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[21480]: audit 2024-03-04T15:49:04.113251+0000 mon.a (mon.0) 635 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:49:05.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[21480]: audit 2024-03-04T15:49:04.115336+0000 mon.a (mon.0) 636 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:49:05.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[21480]: audit 2024-03-04T15:49:04.117494+0000 mon.a (mon.0) 637 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:49:05.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[21480]: audit 2024-03-04T15:49:04.119548+0000 mon.a (mon.0) 638 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:49:05.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[21480]: cephadm 2024-03-04T15:49:04.121063+0000 mgr.y (mgr.14152) 323 : cephadm 1 Adjusting osd_memory_target on smithi151 to 4316M 2024-03-04T15:49:05.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[21480]: audit 2024-03-04T15:49:04.127812+0000 mon.a (mon.0) 639 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:05.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[21480]: audit 2024-03-04T15:49:04.132134+0000 mon.a (mon.0) 640 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:05.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[21480]: audit 2024-03-04T15:49:04.133974+0000 mon.a (mon.0) 641 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:49:05.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:05 smithi117 bash[21480]: audit 2024-03-04T15:49:04.143743+0000 mon.a (mon.0) 642 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:06.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:06 smithi151 bash[23756]: cluster 2024-03-04T15:49:04.830501+0000 mgr.y (mgr.14152) 324 : cluster 0 pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T15:49:06.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:06 smithi117 bash[21480]: cluster 2024-03-04T15:49:04.830501+0000 mgr.y (mgr.14152) 324 : cluster 0 pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T15:49:06.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:06 smithi117 bash[28843]: cluster 2024-03-04T15:49:04.830501+0000 mgr.y (mgr.14152) 324 : cluster 0 pgmap v310: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-03-04T15:49:08.086 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:49:08.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:08 smithi151 bash[23756]: cluster 2024-03-04T15:49:06.831091+0000 mgr.y (mgr.14152) 325 : cluster 0 pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-04T15:49:08.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:08 smithi117 bash[21480]: cluster 2024-03-04T15:49:06.831091+0000 mgr.y (mgr.14152) 325 : cluster 0 pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-04T15:49:08.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:08 smithi117 bash[28843]: cluster 2024-03-04T15:49:06.831091+0000 mgr.y (mgr.14152) 325 : cluster 0 pgmap v311: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-03-04T15:49:10.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:10 smithi151 bash[23756]: cluster 2024-03-04T15:49:08.831676+0000 mgr.y (mgr.14152) 326 : cluster 0 pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-04T15:49:10.477 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:10 smithi117 bash[21480]: cluster 2024-03-04T15:49:08.831676+0000 mgr.y (mgr.14152) 326 : cluster 0 pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-04T15:49:10.477 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:10 smithi117 bash[28843]: cluster 2024-03-04T15:49:08.831676+0000 mgr.y (mgr.14152) 326 : cluster 0 pgmap v312: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 52 KiB/s, 0 objects/s recovering 2024-03-04T15:49:10.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe5405f9700 1 -- 172.21.15.117:0/3403295394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe53806a7f0 msgr2=0x7fe53806abd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:10.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe5405f9700 1 --2- 172.21.15.117:0/3403295394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe53806a7f0 0x7fe53806abd0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fe528009990 tx=0x7fe528009ca0 comp rx=0 tx=0).stop 2024-03-04T15:49:10.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe5405f9700 1 -- 172.21.15.117:0/3403295394 shutdown_connections 2024-03-04T15:49:10.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe5405f9700 1 --2- 172.21.15.117:0/3403295394 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe53806a7f0 0x7fe53806abd0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:10.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe5405f9700 1 --2- 172.21.15.117:0/3403295394 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5381044c0 0x7fe53810ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:10.658 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe5405f9700 1 --2- 172.21.15.117:0/3403295394 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe538103b00 0x7fe538103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:10.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe5405f9700 1 -- 172.21.15.117:0/3403295394 >> 172.21.15.117:0/3403295394 conn(0x7fe538076d80 msgr2=0x7fe538077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:49:10.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe5405f9700 1 -- 172.21.15.117:0/3403295394 shutdown_connections 2024-03-04T15:49:10.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe5405f9700 1 -- 172.21.15.117:0/3403295394 wait complete. 2024-03-04T15:49:10.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe5405f9700 1 Processor -- start 2024-03-04T15:49:10.659 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe5405f9700 1 -- start start 2024-03-04T15:49:10.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe5405f9700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe53806a7f0 0x7fe538100eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:10.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe5405f9700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe538103b00 0x7fe5381013f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:10.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe5405f9700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5381044c0 0x7fe538101930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:10.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe5405f9700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe538111710 con 0x7fe538103b00 2024-03-04T15:49:10.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe5405f9700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe538111590 con 0x7fe53806a7f0 2024-03-04T15:49:10.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe5405f9700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe538111890 con 0x7fe5381044c0 2024-03-04T15:49:10.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe53eb96700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5381044c0 0x7fe538101930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:10.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe53db94700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe538103b00 0x7fe5381013f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:10.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe53eb96700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5381044c0 0x7fe538101930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:57826/0 (socket says 172.21.15.117:57826) 2024-03-04T15:49:10.661 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe53eb96700 1 -- 172.21.15.117:0/899807027 learned_addr learned my addr 172.21.15.117:0/899807027 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:49:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe53db94700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe538103b00 0x7fe5381013f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:53000/0 (socket says 172.21.15.117:53000) 2024-03-04T15:49:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe53e395700 1 --2- 172.21.15.117:0/899807027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe53806a7f0 0x7fe538100eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe53eb96700 1 -- 172.21.15.117:0/899807027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe53806a7f0 msgr2=0x7fe538100eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe53eb96700 1 --2- 172.21.15.117:0/899807027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe53806a7f0 0x7fe538100eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe53eb96700 1 -- 172.21.15.117:0/899807027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe538103b00 msgr2=0x7fe5381013f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe53eb96700 1 --2- 172.21.15.117:0/899807027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe538103b00 0x7fe5381013f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe53eb96700 1 -- 172.21.15.117:0/899807027 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5381a6300 con 0x7fe5381044c0 2024-03-04T15:49:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe53db94700 1 --2- 172.21.15.117:0/899807027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe538103b00 0x7fe5381013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:49:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe53e395700 1 --2- 172.21.15.117:0/899807027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe53806a7f0 0x7fe538100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T15:49:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe53eb96700 1 --2- 172.21.15.117:0/899807027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5381044c0 0x7fe538101930 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe53000c370 tx=0x7fe53000c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:49:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe52f7fe700 1 -- 172.21.15.117:0/899807027 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53000ce20 con 0x7fe5381044c0 2024-03-04T15:49:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe5405f9700 1 -- 172.21.15.117:0/899807027 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5381a6590 con 0x7fe5381044c0 2024-03-04T15:49:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe52f7fe700 1 -- 172.21.15.117:0/899807027 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe5300090d0 con 0x7fe5381044c0 2024-03-04T15:49:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe52f7fe700 1 -- 172.21.15.117:0/899807027 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5300046c0 con 0x7fe5381044c0 2024-03-04T15:49:10.662 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.653+0000 7fe5405f9700 1 -- 172.21.15.117:0/899807027 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5381a6ad0 con 0x7fe5381044c0 2024-03-04T15:49:10.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.657+0000 7fe5405f9700 1 -- 172.21.15.117:0/899807027 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe53804faa0 con 0x7fe5381044c0 2024-03-04T15:49:10.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.657+0000 7fe52f7fe700 1 -- 172.21.15.117:0/899807027 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 15) v1 ==== 91397+0+0 (secure 0 0 0) 0x7fe53001a020 con 0x7fe5381044c0 2024-03-04T15:49:10.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.657+0000 7fe52f7fe700 1 --2- 172.21.15.117:0/899807027 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fe52406d690 0x7fe52406fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:10.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.657+0000 7fe52f7fe700 1 -- 172.21.15.117:0/899807027 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5287+0+0 (secure 0 0 0) 0x7fe53008c980 con 0x7fe5381044c0 2024-03-04T15:49:10.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.657+0000 7fe53e395700 1 --2- 172.21.15.117:0/899807027 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fe52406d690 0x7fe52406fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:10.663 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.657+0000 7fe53e395700 1 --2- 172.21.15.117:0/899807027 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fe52406d690 0x7fe52406fb50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe528006010 tx=0x7fe52801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:49:10.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.661+0000 7fe52f7fe700 1 -- 172.21.15.117:0/899807027 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe530010040 con 0x7fe5381044c0 2024-03-04T15:49:10.876 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.869+0000 7fe5405f9700 1 -- 172.21.15.117:0/899807027 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe5381a6d30 con 0x7fe5381044c0 2024-03-04T15:49:10.877 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.873+0000 7fe52f7fe700 1 -- 172.21.15.117:0/899807027 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v59) v1 ==== 74+0+13621 (secure 0 0 0) 0x7fe53005aaa0 con 0x7fe5381044c0 2024-03-04T15:49:10.877 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-04T15:49:10.878 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":59,"fsid":"e40b5504-da3c-11ee-95c6-87774f69a715","created":"2024-03-04T15:39:15.691654+0000","modified":"2024-03-04T15:48:57.774386+0000","last_up_change":"2024-03-04T15:48:54.249045+0000","last_in_change":"2024-03-04T15:48:27.158859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T15:44:31.324972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf1f40a0-bf1a-4681-84de-fbf6fee19de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1332942120},{"type":"v1","addr":"172.21.15.117:6803","nonce":1332942120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1332942120},{"type":"v1","addr":"172.21.15.117:6805","nonce":1332942120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1332942120},{"type":"v1","addr":"172.21.15.117:6809","nonce":1332942120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1332942120},{"type":"v1","addr":"172.21.15.117:6807","nonce":1332942120}]},"public_addr":"172.21.15.117:6803/1332942120","cluster_addr":"172.21.15.117:6805/1332942120","heartbeat_back_addr":"172.21.15.117:6809/1332942120","heartbeat_front_addr":"172.21.15.117:6807/1332942120","state":["exists","up"]},{"osd":1,"uuid":"9e7ba793-aa0e-4435-bfd6-1bf6d28da6ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":34,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":444026972},{"type":"v1","addr":"172.21.15.117:6811","nonce":444026972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":444026972},{"type":"v1","addr":"172.21.15.117:6813","nonce":444026972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":444026972},{"type":"v1","addr":"172.21.15.117:6817","nonce":444026972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":444026972},{"type":"v1","addr":"172.21.15.117:6815","nonce":444026972}]},"public_addr":"172.21.15.117:6811/444026972","cluster_addr":"172.21.15.117:6813/444026972","heartbeat_back_addr":"172.21.15.117:6817/444026972","heartbeat_front_addr":"172.21.15.117:6815/444026972","state":["exists","up"]},{"osd":2,"uuid":"49710dbf-78b9-4983-bc61-2a5d0527e417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4102123427},{"type":"v1","addr":"172.21.15.117:6819","nonce":4102123427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":4102123427},{"type":"v1","addr":"172.21.15.117:6821","nonce":4102123427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":4102123427},{"type":"v1","addr":"172.21.15.117:6825","nonce":4102123427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4102123427},{"type":"v1","addr":"172.21.15.117:6823","nonce":4102123427}]},"public_addr":"172.21.15.117:6819/4102123427","cluster_addr":"172.21.15.117:6821/4102123427","heartbeat_back_addr":"172.21.15.117:6825/4102123427","heartbeat_front_addr":"172.21.15.117:6823/4102123427","state":["exists","up"]},{"osd":3,"uuid":"1e1fa658-d1a9-4129-ac7e-18f317b3b318","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3320151079},{"type":"v1","addr":"172.21.15.117:6827","nonce":3320151079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3320151079},{"type":"v1","addr":"172.21.15.117:6829","nonce":3320151079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3320151079},{"type":"v1","addr":"172.21.15.117:6833","nonce":3320151079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3320151079},{"type":"v1","addr":"172.21.15.117:6831","nonce":3320151079}]},"public_addr":"172.21.15.117:6827/3320151079","cluster_addr":"172.21.15.117:6829/3320151079","heartbeat_back_addr":"172.21.15.117:6833/3320151079","heartbeat_front_addr":"172.21.15.117:6831/3320151079","state":["exists","up"]},{"osd":4,"uuid":"35746d23-4625-45a9-81cd-c05f7ea5a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":1293012079},{"type":"v1","addr":"172.21.15.151:6801","nonce":1293012079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":1293012079},{"type":"v1","addr":"172.21.15.151:6803","nonce":1293012079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":1293012079},{"type":"v1","addr":"172.21.15.151:6807","nonce":1293012079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":1293012079},{"type":"v1","addr":"172.21.15.151:6805","nonce":1293012079}]},"public_addr":"172.21.15.151:6801/1293012079","cluster_addr":"172.21.15.151:6803/1293012079","heartbeat_back_addr":"172.21.15.151:6807/1293012079","heartbeat_front_addr":"172.21.15.151:6805/1293012079","state":["exists","up"]},{"osd":5,"uuid":"9660e52c-72e0-4f9c-8cf5-bf5868080e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":42,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3277640682},{"type":"v1","addr":"172.21.15.151:6809","nonce":3277640682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3277640682},{"type":"v1","addr":"172.21.15.151:6811","nonce":3277640682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3277640682},{"type":"v1","addr":"172.21.15.151:6815","nonce":3277640682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3277640682},{"type":"v1","addr":"172.21.15.151:6813","nonce":3277640682}]},"public_addr":"172.21.15.151:6809/3277640682","cluster_addr":"172.21.15.151:6811/3277640682","heartbeat_back_addr":"172.21.15.151:6815/3277640682","heartbeat_front_addr":"172.21.15.151:6813/3277640682","state":["exists","up"]},{"osd":6,"uuid":"706cd130-5cc0-41f3-9e78-85922c29a01c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":3501440615},{"type":"v1","addr":"172.21.15.151:6817","nonce":3501440615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":3501440615},{"type":"v1","addr":"172.21.15.151:6819","nonce":3501440615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":3501440615},{"type":"v1","addr":"172.21.15.151:6823","nonce":3501440615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":3501440615},{"type":"v1","addr":"172.21.15.151:6821","nonce":3501440615}]},"public_addr":"172.21.15.151:6817/3501440615","cluster_addr":"172.21.15.151:6819/3501440615","heartbeat_back_addr":"172.21.15.151:6823/3501440615","heartbeat_front_addr":"172.21.15.151:6821/3501440615","state":["exists","up"]},{"osd":7,"uuid":"0f49ea27-0d78-4e68-8d06-b3c565e37a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":58,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3965450917},{"type":"v1","addr":"172.21.15.151:6825","nonce":3965450917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3965450917},{"type":"v1","addr":"172.21.15.151:6827","nonce":3965450917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3965450917},{"type":"v1","addr":"172.21.15.151:6831","nonce":3965450917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3965450917},{"type":"v1","addr":"172.21.15.151:6829","nonce":3965450917}]},"public_addr":"172.21.15.151:6825/3965450917","cluster_addr":"172.21.15.151:6827/3965450917","heartbeat_back_addr":"172.21.15.151:6831/3965450917","heartbeat_front_addr":"172.21.15.151:6829/3965450917","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:42:47.269066+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:43:37.699556+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:44:28.025013+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:45:20.549377+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:46:11.142621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:47:04.620036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:47:58.871589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:48:52.475968+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/546245806":"2024-03-05T15:40:22.691924+0000","172.21.15.117:6801/1454790014":"2024-03-05T15:40:22.691924+0000","172.21.15.117:0/163360369":"2024-03-05T15:40:22.691924+0000","172.21.15.117:0/923237697":"2024-03-05T15:40:22.691924+0000","172.21.15.117:6800/1454790014":"2024-03-05T15:40:22.691924+0000","172.21.15.117:0/2167394439":"2024-03-05T15:39:47.643279+0000","172.21.15.117:6801/427955642":"2024-03-05T15:39:47.643279+0000","172.21.15.117:6800/427955642":"2024-03-05T15:39:47.643279+0000","172.21.15.117:0/2488053720":"2024-03-05T15:39:47.643279+0000","172.21.15.117:0/4187340781":"2024-03-05T15:39:47.643279+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T15:49:10.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.873+0000 7fe5405f9700 1 -- 172.21.15.117:0/899807027 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fe52406d690 msgr2=0x7fe52406fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:10.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.873+0000 7fe5405f9700 1 --2- 172.21.15.117:0/899807027 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fe52406d690 0x7fe52406fb50 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fe528006010 tx=0x7fe52801a040 comp rx=0 tx=0).stop 2024-03-04T15:49:10.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.873+0000 7fe5405f9700 1 -- 172.21.15.117:0/899807027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5381044c0 msgr2=0x7fe538101930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:10.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.873+0000 7fe5405f9700 1 --2- 172.21.15.117:0/899807027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5381044c0 0x7fe538101930 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe53000c370 tx=0x7fe53000c680 comp rx=0 tx=0).stop 2024-03-04T15:49:10.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.873+0000 7fe5405f9700 1 -- 172.21.15.117:0/899807027 shutdown_connections 2024-03-04T15:49:10.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.873+0000 7fe5405f9700 1 --2- 172.21.15.117:0/899807027 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7fe52406d690 0x7fe52406fb50 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:10.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.873+0000 7fe5405f9700 1 --2- 172.21.15.117:0/899807027 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe5381044c0 0x7fe538101930 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:10.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.873+0000 7fe5405f9700 1 --2- 172.21.15.117:0/899807027 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe53806a7f0 0x7fe538100eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:10.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.873+0000 7fe5405f9700 1 --2- 172.21.15.117:0/899807027 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe538103b00 0x7fe5381013f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:10.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.873+0000 7fe5405f9700 1 -- 172.21.15.117:0/899807027 >> 172.21.15.117:0/899807027 conn(0x7fe538076d80 msgr2=0x7fe538106c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:49:10.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.877+0000 7fe5405f9700 1 -- 172.21.15.117:0/899807027 shutdown_connections 2024-03-04T15:49:10.880 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:10.877+0000 7fe5405f9700 1 -- 172.21.15.117:0/899807027 wait complete. 2024-03-04T15:49:11.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:11 smithi151 bash[23756]: audit 2024-03-04T15:49:10.877575+0000 mon.c (mon.2) 13 : audit 0 from='client.? 172.21.15.117:0/899807027' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T15:49:11.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:11 smithi117 bash[21480]: audit 2024-03-04T15:49:10.877575+0000 mon.c (mon.2) 13 : audit 0 from='client.? 172.21.15.117:0/899807027' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T15:49:11.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:11 smithi117 bash[28843]: audit 2024-03-04T15:49:10.877575+0000 mon.c (mon.2) 13 : audit 0 from='client.? 172.21.15.117:0/899807027' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T15:49:11.578 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-03-04T15:44:31.324972+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 7.889999866485596, 'score_stable': 7.889999866485596, 'optimal_score': 0.3799999952316284, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-03-04T15:49:11.578 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph osd pool get .mgr pg_num 2024-03-04T15:49:12.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:12 smithi151 bash[23756]: cluster 2024-03-04T15:49:10.832267+0000 mgr.y (mgr.14152) 327 : cluster 0 pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-04T15:49:12.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:12 smithi117 bash[21480]: cluster 2024-03-04T15:49:10.832267+0000 mgr.y (mgr.14152) 327 : cluster 0 pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-04T15:49:12.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:12 smithi117 bash[28843]: cluster 2024-03-04T15:49:10.832267+0000 mgr.y (mgr.14152) 327 : cluster 0 pgmap v313: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-03-04T15:49:14.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:14 smithi151 bash[23756]: cluster 2024-03-04T15:49:12.832877+0000 mgr.y (mgr.14152) 328 : cluster 0 pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:14.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:14 smithi117 bash[21480]: cluster 2024-03-04T15:49:12.832877+0000 mgr.y (mgr.14152) 328 : cluster 0 pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:14.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:14 smithi117 bash[28843]: cluster 2024-03-04T15:49:12.832877+0000 mgr.y (mgr.14152) 328 : cluster 0 pgmap v314: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:16.331 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:49:16.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:16 smithi151 bash[23756]: cluster 2024-03-04T15:49:14.833481+0000 mgr.y (mgr.14152) 329 : cluster 0 pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:16.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:16 smithi117 bash[21480]: cluster 2024-03-04T15:49:14.833481+0000 mgr.y (mgr.14152) 329 : cluster 0 pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:16.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:16 smithi117 bash[28843]: cluster 2024-03-04T15:49:14.833481+0000 mgr.y (mgr.14152) 329 : cluster 0 pgmap v315: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:18.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:18 smithi151 bash[23756]: cluster 2024-03-04T15:49:16.834028+0000 mgr.y (mgr.14152) 330 : cluster 0 pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:18.542 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:18 smithi117 bash[21480]: cluster 2024-03-04T15:49:16.834028+0000 mgr.y (mgr.14152) 330 : cluster 0 pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:18.542 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:18 smithi117 bash[28843]: cluster 2024-03-04T15:49:16.834028+0000 mgr.y (mgr.14152) 330 : cluster 0 pgmap v316: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:18.701 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cb1e30700 1 -- 172.21.15.117:0/3294596302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2cac06b0b0 msgr2=0x7f2cac106a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:18.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cb1e30700 1 --2- 172.21.15.117:0/3294596302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2cac06b0b0 0x7f2cac106a90 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009990 tx=0x7f2c9c009ca0 comp rx=0 tx=0).stop 2024-03-04T15:49:18.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cb1e30700 1 -- 172.21.15.117:0/3294596302 shutdown_connections 2024-03-04T15:49:18.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cb1e30700 1 --2- 172.21.15.117:0/3294596302 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cac107170 0x7f2cac10ed40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:18.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cb1e30700 1 --2- 172.21.15.117:0/3294596302 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cac06a700 0x7f2cac06aae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:18.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cb1e30700 1 --2- 172.21.15.117:0/3294596302 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2cac06b0b0 0x7f2cac106a90 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:18.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cb1e30700 1 -- 172.21.15.117:0/3294596302 >> 172.21.15.117:0/3294596302 conn(0x7f2cac077d30 msgr2=0x7f2cac078140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:49:18.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cb1e30700 1 -- 172.21.15.117:0/3294596302 shutdown_connections 2024-03-04T15:49:18.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cb1e30700 1 -- 172.21.15.117:0/3294596302 wait complete. 2024-03-04T15:49:18.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cb1e30700 1 Processor -- start 2024-03-04T15:49:18.702 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cb1e30700 1 -- start start 2024-03-04T15:49:18.703 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cb1e30700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cac06a700 0x7f2cac19d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:18.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cb1e30700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cac06b0b0 0x7f2cac19dcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:18.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cb1e30700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2cac107170 0x7f2cac197940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:18.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cb1e30700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac111660 con 0x7f2cac06a700 2024-03-04T15:49:18.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cb1e30700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac1114e0 con 0x7f2cac06b0b0 2024-03-04T15:49:18.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cb1e30700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2cac1117e0 con 0x7f2cac107170 2024-03-04T15:49:18.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cab7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cac06a700 0x7f2cac19d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:18.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cab7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cac06a700 0x7f2cac19d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38104/0 (socket says 172.21.15.117:38104) 2024-03-04T15:49:18.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cab7fe700 1 -- 172.21.15.117:0/2021201222 learned_addr learned my addr 172.21.15.117:0/2021201222 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:49:18.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cabfff700 1 --2- 172.21.15.117:0/2021201222 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2cac107170 0x7f2cac197940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:18.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2caaffd700 1 --2- 172.21.15.117:0/2021201222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cac06b0b0 0x7f2cac19dcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:18.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cab7fe700 1 -- 172.21.15.117:0/2021201222 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2cac107170 msgr2=0x7f2cac197940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:18.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cab7fe700 1 --2- 172.21.15.117:0/2021201222 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2cac107170 0x7f2cac197940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:18.704 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cab7fe700 1 -- 172.21.15.117:0/2021201222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cac06b0b0 msgr2=0x7f2cac19dcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:18.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cab7fe700 1 --2- 172.21.15.117:0/2021201222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cac06b0b0 0x7f2cac19dcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:18.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cab7fe700 1 -- 172.21.15.117:0/2021201222 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2cac1980a0 con 0x7f2cac06a700 2024-03-04T15:49:18.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cabfff700 1 --2- 172.21.15.117:0/2021201222 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2cac107170 0x7f2cac197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:49:18.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cab7fe700 1 --2- 172.21.15.117:0/2021201222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cac06a700 0x7f2cac19d780 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2c940060b0 tx=0x7f2c94007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:49:18.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2ca8ff9700 1 -- 172.21.15.117:0/2021201222 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9401a070 con 0x7f2cac06a700 2024-03-04T15:49:18.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2ca8ff9700 1 -- 172.21.15.117:0/2021201222 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f2c9400dd60 con 0x7f2cac06a700 2024-03-04T15:49:18.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2ca8ff9700 1 -- 172.21.15.117:0/2021201222 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c94014600 con 0x7f2cac06a700 2024-03-04T15:49:18.705 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.697+0000 7f2cb1e30700 1 -- 172.21.15.117:0/2021201222 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2cac198390 con 0x7f2cac06a700 2024-03-04T15:49:18.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.701+0000 7f2cb1e30700 1 -- 172.21.15.117:0/2021201222 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2cac100cc0 con 0x7f2cac06a700 2024-03-04T15:49:18.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.701+0000 7f2cb1e30700 1 -- 172.21.15.117:0/2021201222 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2cac04faa0 con 0x7f2cac06a700 2024-03-04T15:49:18.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.701+0000 7f2ca8ff9700 1 -- 172.21.15.117:0/2021201222 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 15) v1 ==== 91397+0+0 (secure 0 0 0) 0x7f2c94004ad0 con 0x7f2cac06a700 2024-03-04T15:49:18.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.701+0000 7f2ca8ff9700 1 --2- 172.21.15.117:0/2021201222 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f2c8c06d690 0x7f2c8c06fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:18.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.701+0000 7f2ca8ff9700 1 -- 172.21.15.117:0/2021201222 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5287+0+0 (secure 0 0 0) 0x7f2c9408ccc0 con 0x7f2cac06a700 2024-03-04T15:49:18.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.701+0000 7f2caaffd700 1 --2- 172.21.15.117:0/2021201222 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f2c8c06d690 0x7f2c8c06fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:18.706 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.701+0000 7f2caaffd700 1 --2- 172.21.15.117:0/2021201222 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f2c8c06d690 0x7f2c8c06fb50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c006010 tx=0x7f2c9c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:49:18.710 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.705+0000 7f2ca8ff9700 1 -- 172.21.15.117:0/2021201222 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2c94010070 con 0x7f2cac06a700 2024-03-04T15:49:18.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.913+0000 7f2cb1e30700 1 -- 172.21.15.117:0/2021201222 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2cac198e70 con 0x7f2cac06a700 2024-03-04T15:49:18.917 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.913+0000 7f2ca8ff9700 1 -- 172.21.15.117:0/2021201222 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v59) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2c9405ae60 con 0x7f2cac06a700 2024-03-04T15:49:18.918 INFO:teuthology.orchestra.run.smithi117.stdout:pg_num: 1 2024-03-04T15:49:18.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.917+0000 7f2cb1e30700 1 -- 172.21.15.117:0/2021201222 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f2c8c06d690 msgr2=0x7f2c8c06fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:18.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.917+0000 7f2cb1e30700 1 --2- 172.21.15.117:0/2021201222 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f2c8c06d690 0x7f2c8c06fb50 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c006010 tx=0x7f2c9c01a040 comp rx=0 tx=0).stop 2024-03-04T15:49:18.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.917+0000 7f2cb1e30700 1 -- 172.21.15.117:0/2021201222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cac06a700 msgr2=0x7f2cac19d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:18.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.917+0000 7f2cb1e30700 1 --2- 172.21.15.117:0/2021201222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cac06a700 0x7f2cac19d780 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2c940060b0 tx=0x7f2c94007f90 comp rx=0 tx=0).stop 2024-03-04T15:49:18.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.917+0000 7f2cb1e30700 1 -- 172.21.15.117:0/2021201222 shutdown_connections 2024-03-04T15:49:18.920 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.917+0000 7f2cb1e30700 1 --2- 172.21.15.117:0/2021201222 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2cac06a700 0x7f2cac19d780 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:18.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.917+0000 7f2cb1e30700 1 --2- 172.21.15.117:0/2021201222 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f2c8c06d690 0x7f2c8c06fb50 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:18.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.917+0000 7f2cb1e30700 1 --2- 172.21.15.117:0/2021201222 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2cac107170 0x7f2cac197940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:18.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.917+0000 7f2cb1e30700 1 --2- 172.21.15.117:0/2021201222 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2cac06b0b0 0x7f2cac19dcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:18.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.917+0000 7f2cb1e30700 1 -- 172.21.15.117:0/2021201222 >> 172.21.15.117:0/2021201222 conn(0x7f2cac077d30 msgr2=0x7f2cac0fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:49:18.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.917+0000 7f2cb1e30700 1 -- 172.21.15.117:0/2021201222 shutdown_connections 2024-03-04T15:49:18.921 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:49:18.917+0000 7f2cb1e30700 1 -- 172.21.15.117:0/2021201222 wait complete. 2024-03-04T15:49:19.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:19 smithi117 bash[21480]: audit 2024-03-04T15:49:18.918716+0000 mon.a (mon.0) 643 : audit 0 from='client.? 172.21.15.117:0/2021201222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T15:49:19.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:19 smithi117 bash[28843]: audit 2024-03-04T15:49:18.918716+0000 mon.a (mon.0) 643 : audit 0 from='client.? 172.21.15.117:0/2021201222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T15:49:19.605 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi117 2024-03-04T15:49:19.605 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph orch apply rgw foo.a --placement '1;smithi117=foo.a' 2024-03-04T15:49:19.616 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:19 smithi151 bash[23756]: audit 2024-03-04T15:49:18.918716+0000 mon.a (mon.0) 643 : audit 0 from='client.? 172.21.15.117:0/2021201222' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-03-04T15:49:20.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:20 smithi117 bash[21480]: cluster 2024-03-04T15:49:18.834560+0000 mgr.y (mgr.14152) 331 : cluster 0 pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:20.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:20 smithi117 bash[28843]: cluster 2024-03-04T15:49:18.834560+0000 mgr.y (mgr.14152) 331 : cluster 0 pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:20.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:20 smithi151 bash[23756]: cluster 2024-03-04T15:49:18.834560+0000 mgr.y (mgr.14152) 331 : cluster 0 pgmap v317: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:22.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:22 smithi117 bash[21480]: cluster 2024-03-04T15:49:20.835147+0000 mgr.y (mgr.14152) 332 : cluster 0 pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:22.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:22 smithi117 bash[28843]: cluster 2024-03-04T15:49:20.835147+0000 mgr.y (mgr.14152) 332 : cluster 0 pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:22.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:22 smithi151 bash[23756]: cluster 2024-03-04T15:49:20.835147+0000 mgr.y (mgr.14152) 332 : cluster 0 pgmap v318: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:24.350 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T15:49:24.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:24 smithi117 bash[21480]: cluster 2024-03-04T15:49:22.835723+0000 mgr.y (mgr.14152) 333 : cluster 0 pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:24.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:24 smithi117 bash[28843]: cluster 2024-03-04T15:49:22.835723+0000 mgr.y (mgr.14152) 333 : cluster 0 pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:24.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:24 smithi151 bash[23756]: cluster 2024-03-04T15:49:22.835723+0000 mgr.y (mgr.14152) 333 : cluster 0 pgmap v319: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:26.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:26 smithi117 bash[21480]: cluster 2024-03-04T15:49:24.836373+0000 mgr.y (mgr.14152) 334 : cluster 0 pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:26.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:26 smithi117 bash[28843]: cluster 2024-03-04T15:49:24.836373+0000 mgr.y (mgr.14152) 334 : cluster 0 pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:26.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:26 smithi151 bash[23756]: cluster 2024-03-04T15:49:24.836373+0000 mgr.y (mgr.14152) 334 : cluster 0 pgmap v320: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:27.439 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47daa2e700 1 -- 172.21.15.151:0/3119200981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47d4101e70 msgr2=0x7f47d4102290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47daa2e700 1 --2- 172.21.15.151:0/3119200981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47d4101e70 0x7f47d4102290 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f47c4009990 tx=0x7f47c4009ca0 comp rx=0 tx=0).stop 2024-03-04T15:49:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47daa2e700 1 -- 172.21.15.151:0/3119200981 shutdown_connections 2024-03-04T15:49:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47daa2e700 1 --2- 172.21.15.151:0/3119200981 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47d4103990 0x7f47d410a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47daa2e700 1 --2- 172.21.15.151:0/3119200981 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47d4101e70 0x7f47d4102290 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47daa2e700 1 --2- 172.21.15.151:0/3119200981 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47d4102fd0 0x7f47d4103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47daa2e700 1 -- 172.21.15.151:0/3119200981 >> 172.21.15.151:0/3119200981 conn(0x7f47d40fd410 msgr2=0x7f47d40ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:49:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47daa2e700 1 -- 172.21.15.151:0/3119200981 shutdown_connections 2024-03-04T15:49:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47daa2e700 1 -- 172.21.15.151:0/3119200981 wait complete. 2024-03-04T15:49:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47daa2e700 1 Processor -- start 2024-03-04T15:49:27.440 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47daa2e700 1 -- start start 2024-03-04T15:49:27.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47daa2e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47d4101e70 0x7f47d41976b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:27.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47daa2e700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47d4102fd0 0x7f47d4197bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:27.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47daa2e700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47d4103990 0x7f47d419ec70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:27.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47daa2e700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d410cf80 con 0x7f47d4102fd0 2024-03-04T15:49:27.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47daa2e700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47d410ce00 con 0x7f47d4103990 2024-03-04T15:49:27.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47daa2e700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47d410d100 con 0x7f47d4101e70 2024-03-04T15:49:27.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47d8fcb700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47d4103990 0x7f47d419ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:27.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47d8fcb700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47d4103990 0x7f47d419ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:46842/0 (socket says 172.21.15.151:46842) 2024-03-04T15:49:27.442 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47d8fcb700 1 -- 172.21.15.151:0/1564137049 learned_addr learned my addr 172.21.15.151:0/1564137049 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-04T15:49:27.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47cb7fe700 1 --2- 172.21.15.151:0/1564137049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47d4102fd0 0x7f47d4197bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:27.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47d3fff700 1 --2- 172.21.15.151:0/1564137049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47d4101e70 0x7f47d41976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:27.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47d8fcb700 1 -- 172.21.15.151:0/1564137049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47d4101e70 msgr2=0x7f47d41976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:27.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47d8fcb700 1 --2- 172.21.15.151:0/1564137049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47d4101e70 0x7f47d41976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:27.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47d8fcb700 1 -- 172.21.15.151:0/1564137049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47d4102fd0 msgr2=0x7f47d4197bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:27.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47d8fcb700 1 --2- 172.21.15.151:0/1564137049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47d4102fd0 0x7f47d4197bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:27.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47d8fcb700 1 -- 172.21.15.151:0/1564137049 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47d419f310 con 0x7f47d4103990 2024-03-04T15:49:27.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47d3fff700 1 --2- 172.21.15.151:0/1564137049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47d4101e70 0x7f47d41976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T15:49:27.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47d8fcb700 1 --2- 172.21.15.151:0/1564137049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47d4103990 0x7f47d419ec70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f47cc007ee0 tx=0x7f47cc00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:49:27.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47d1ffb700 1 -- 172.21.15.151:0/1564137049 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc00dbd0 con 0x7f47d4103990 2024-03-04T15:49:27.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47d1ffb700 1 -- 172.21.15.151:0/1564137049 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f47cc00dd70 con 0x7f47d4103990 2024-03-04T15:49:27.443 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47daa2e700 1 -- 172.21.15.151:0/1564137049 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47d419f600 con 0x7f47d4103990 2024-03-04T15:49:27.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47d1ffb700 1 -- 172.21.15.151:0/1564137049 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47cc005890 con 0x7f47d4103990 2024-03-04T15:49:27.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47daa2e700 1 -- 172.21.15.151:0/1564137049 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47d419fb70 con 0x7f47d4103990 2024-03-04T15:49:27.444 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.437+0000 7f47daa2e700 1 -- 172.21.15.151:0/1564137049 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47d404faa0 con 0x7f47d4103990 2024-03-04T15:49:27.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.441+0000 7f47d1ffb700 1 -- 172.21.15.151:0/1564137049 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 15) v1 ==== 91397+0+0 (secure 0 0 0) 0x7f47cc01a020 con 0x7f47d4103990 2024-03-04T15:49:27.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.441+0000 7f47d1ffb700 1 --2- 172.21.15.151:0/1564137049 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f47c006d690 0x7f47c006fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:27.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.441+0000 7f47d1ffb700 1 -- 172.21.15.151:0/1564137049 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5287+0+0 (secure 0 0 0) 0x7f47cc08c8f0 con 0x7f47d4103990 2024-03-04T15:49:27.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.441+0000 7f47d3fff700 1 --2- 172.21.15.151:0/1564137049 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f47c006d690 0x7f47c006fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:27.445 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.441+0000 7f47d3fff700 1 --2- 172.21.15.151:0/1564137049 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f47c006d690 0x7f47c006fb50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f47c400b5c0 tx=0x7f47c4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:49:27.448 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.445+0000 7f47d1ffb700 1 -- 172.21.15.151:0/1564137049 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f47cc010070 con 0x7f47d4103990 2024-03-04T15:49:27.664 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.661+0000 7f47daa2e700 1 -- 172.21.15.151:0/1564137049 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi117=foo.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f47d4062220 con 0x7f47c006d690 2024-03-04T15:49:27.679 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled rgw.foo.a update... 2024-03-04T15:49:27.679 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.677+0000 7f47d1ffb700 1 -- 172.21.15.151:0/1564137049 <== mgr.14152 v2:172.21.15.117:6800/2865911351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7f47d4062220 con 0x7f47c006d690 2024-03-04T15:49:27.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.677+0000 7f47daa2e700 1 -- 172.21.15.151:0/1564137049 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f47c006d690 msgr2=0x7f47c006fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:27.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.677+0000 7f47daa2e700 1 --2- 172.21.15.151:0/1564137049 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f47c006d690 0x7f47c006fb50 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f47c400b5c0 tx=0x7f47c4005fb0 comp rx=0 tx=0).stop 2024-03-04T15:49:27.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.677+0000 7f47daa2e700 1 -- 172.21.15.151:0/1564137049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47d4103990 msgr2=0x7f47d419ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:27.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.677+0000 7f47daa2e700 1 --2- 172.21.15.151:0/1564137049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47d4103990 0x7f47d419ec70 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f47cc007ee0 tx=0x7f47cc00d3b0 comp rx=0 tx=0).stop 2024-03-04T15:49:27.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.677+0000 7f47daa2e700 1 -- 172.21.15.151:0/1564137049 shutdown_connections 2024-03-04T15:49:27.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.677+0000 7f47daa2e700 1 --2- 172.21.15.151:0/1564137049 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f47c006d690 0x7f47c006fb50 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:27.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.677+0000 7f47daa2e700 1 --2- 172.21.15.151:0/1564137049 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f47d4103990 0x7f47d419ec70 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:27.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.677+0000 7f47daa2e700 1 --2- 172.21.15.151:0/1564137049 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f47d4101e70 0x7f47d41976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:27.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.677+0000 7f47daa2e700 1 --2- 172.21.15.151:0/1564137049 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f47d4102fd0 0x7f47d4197bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:27.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.677+0000 7f47daa2e700 1 -- 172.21.15.151:0/1564137049 >> 172.21.15.151:0/1564137049 conn(0x7f47d40fd410 msgr2=0x7f47d40fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:49:27.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.677+0000 7f47daa2e700 1 -- 172.21.15.151:0/1564137049 shutdown_connections 2024-03-04T15:49:27.682 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:27.677+0000 7f47daa2e700 1 -- 172.21.15.151:0/1564137049 wait complete. 2024-03-04T15:49:28.453 DEBUG:teuthology.orchestra.run.smithi117:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@rgw.foo.a.service 2024-03-04T15:49:28.455 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi151 2024-03-04T15:49:28.456 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph osd pool create datapool 3 3 replicated 2024-03-04T15:49:28.766 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[21480]: cluster 2024-03-04T15:49:26.837124+0000 mgr.y (mgr.14152) 335 : cluster 0 pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:28.767 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[21480]: audit 2024-03-04T15:49:27.667940+0000 mgr.y (mgr.14152) 336 : audit 0 from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi117=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:49:28.767 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[21480]: cephadm 2024-03-04T15:49:27.671942+0000 mgr.y (mgr.14152) 337 : cephadm 1 Saving service rgw.foo.a spec with placement smithi117=foo.a;count:1 2024-03-04T15:49:28.767 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[21480]: audit 2024-03-04T15:49:27.679412+0000 mon.a (mon.0) 644 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:28.767 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[21480]: audit 2024-03-04T15:49:27.689948+0000 mon.a (mon.0) 645 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:49:28.767 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[21480]: audit 2024-03-04T15:49:28.320484+0000 mon.a (mon.0) 646 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:28.767 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[21480]: audit 2024-03-04T15:49:28.322471+0000 mon.a (mon.0) 647 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:49:28.767 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[21480]: audit 2024-03-04T15:49:28.333194+0000 mon.a (mon.0) 648 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:28.767 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[21480]: audit 2024-03-04T15:49:28.339366+0000 mon.a (mon.0) 649 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-04T15:49:28.767 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[21480]: audit 2024-03-04T15:49:28.342533+0000 mon.a (mon.0) 650 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-04T15:49:28.767 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[21480]: audit 2024-03-04T15:49:28.353875+0000 mon.a (mon.0) 651 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:28.767 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[21480]: audit 2024-03-04T15:49:28.357998+0000 mon.a (mon.0) 652 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:28.768 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[28843]: cluster 2024-03-04T15:49:26.837124+0000 mgr.y (mgr.14152) 335 : cluster 0 pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:28.768 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[28843]: audit 2024-03-04T15:49:27.667940+0000 mgr.y (mgr.14152) 336 : audit 0 from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi117=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:49:28.768 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[28843]: cephadm 2024-03-04T15:49:27.671942+0000 mgr.y (mgr.14152) 337 : cephadm 1 Saving service rgw.foo.a spec with placement smithi117=foo.a;count:1 2024-03-04T15:49:28.768 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[28843]: audit 2024-03-04T15:49:27.679412+0000 mon.a (mon.0) 644 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:28.768 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[28843]: audit 2024-03-04T15:49:27.689948+0000 mon.a (mon.0) 645 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:49:28.768 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[28843]: audit 2024-03-04T15:49:28.320484+0000 mon.a (mon.0) 646 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:28.768 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[28843]: audit 2024-03-04T15:49:28.322471+0000 mon.a (mon.0) 647 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:49:28.768 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[28843]: audit 2024-03-04T15:49:28.333194+0000 mon.a (mon.0) 648 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:28.768 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[28843]: audit 2024-03-04T15:49:28.339366+0000 mon.a (mon.0) 649 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-04T15:49:28.768 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[28843]: audit 2024-03-04T15:49:28.342533+0000 mon.a (mon.0) 650 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-04T15:49:28.768 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[28843]: audit 2024-03-04T15:49:28.353875+0000 mon.a (mon.0) 651 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:28.768 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:28 smithi117 bash[28843]: audit 2024-03-04T15:49:28.357998+0000 mon.a (mon.0) 652 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:28.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:28 smithi151 bash[23756]: cluster 2024-03-04T15:49:26.837124+0000 mgr.y (mgr.14152) 335 : cluster 0 pgmap v321: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:28.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:28 smithi151 bash[23756]: audit 2024-03-04T15:49:27.667940+0000 mgr.y (mgr.14152) 336 : audit 0 from='client.24322 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi117=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:49:28.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:28 smithi151 bash[23756]: cephadm 2024-03-04T15:49:27.671942+0000 mgr.y (mgr.14152) 337 : cephadm 1 Saving service rgw.foo.a spec with placement smithi117=foo.a;count:1 2024-03-04T15:49:28.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:28 smithi151 bash[23756]: audit 2024-03-04T15:49:27.679412+0000 mon.a (mon.0) 644 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:28.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:28 smithi151 bash[23756]: audit 2024-03-04T15:49:27.689948+0000 mon.a (mon.0) 645 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:49:28.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:28 smithi151 bash[23756]: audit 2024-03-04T15:49:28.320484+0000 mon.a (mon.0) 646 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:28.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:28 smithi151 bash[23756]: audit 2024-03-04T15:49:28.322471+0000 mon.a (mon.0) 647 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:49:28.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:28 smithi151 bash[23756]: audit 2024-03-04T15:49:28.333194+0000 mon.a (mon.0) 648 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:28.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:28 smithi151 bash[23756]: audit 2024-03-04T15:49:28.339366+0000 mon.a (mon.0) 649 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-03-04T15:49:28.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:28 smithi151 bash[23756]: audit 2024-03-04T15:49:28.342533+0000 mon.a (mon.0) 650 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-03-04T15:49:28.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:28 smithi151 bash[23756]: audit 2024-03-04T15:49:28.353875+0000 mon.a (mon.0) 651 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:28.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:28 smithi151 bash[23756]: audit 2024-03-04T15:49:28.357998+0000 mon.a (mon.0) 652 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:29.906 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:29 smithi117 bash[28843]: cephadm 2024-03-04T15:49:28.360154+0000 mgr.y (mgr.14152) 338 : cephadm 1 Deploying daemon rgw.foo.a on smithi117 2024-03-04T15:49:29.906 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:29 smithi117 bash[21480]: cephadm 2024-03-04T15:49:28.360154+0000 mgr.y (mgr.14152) 338 : cephadm 1 Deploying daemon rgw.foo.a on smithi117 2024-03-04T15:49:30.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:29 smithi151 bash[23756]: cephadm 2024-03-04T15:49:28.360154+0000 mgr.y (mgr.14152) 338 : cephadm 1 Deploying daemon rgw.foo.a on smithi117 2024-03-04T15:49:30.668 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:49:30.668 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:49:30.669 INFO:journalctl@ceph.osd.0.smithi117.stdout:Mar 04 15:49:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:49:30.669 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 04 15:49:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:49:30.669 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:49:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:49:30.670 INFO:journalctl@ceph.osd.1.smithi117.stdout:Mar 04 15:49:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:49:30.670 INFO:journalctl@ceph.osd.2.smithi117.stdout:Mar 04 15:49:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:49:30.670 INFO:journalctl@ceph.rgw.foo.a.smithi117.stdout:Mar 04 15:49:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:49:30.956 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:49:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:49:30.956 INFO:journalctl@ceph.osd.0.smithi117.stdout:Mar 04 15:49:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:49:30.957 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 04 15:49:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:49:30.957 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:30 smithi117 bash[21480]: cluster 2024-03-04T15:49:28.837624+0000 mgr.y (mgr.14152) 339 : cluster 0 pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:30.957 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:49:30.957 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:30 smithi117 bash[28843]: cluster 2024-03-04T15:49:28.837624+0000 mgr.y (mgr.14152) 339 : cluster 0 pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:30.958 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:49:30.958 INFO:journalctl@ceph.osd.1.smithi117.stdout:Mar 04 15:49:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:49:30.958 INFO:journalctl@ceph.osd.2.smithi117.stdout:Mar 04 15:49:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:49:30.959 INFO:journalctl@ceph.rgw.foo.a.smithi117.stdout:Mar 04 15:49:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:49:30.959 INFO:journalctl@ceph.rgw.foo.a.smithi117.stdout:Mar 04 15:49:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:49:30.959 INFO:journalctl@ceph.rgw.foo.a.smithi117.stdout:Mar 04 15:49:30 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:49:30.959 INFO:journalctl@ceph.rgw.foo.a.smithi117.stdout:Mar 04 15:49:30 smithi117 systemd[1]: Started Ceph rgw.foo.a for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T15:49:31.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:30 smithi151 bash[23756]: cluster 2024-03-04T15:49:28.837624+0000 mgr.y (mgr.14152) 339 : cluster 0 pgmap v322: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:32.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:32 smithi151 bash[23756]: cluster 2024-03-04T15:49:30.837952+0000 mgr.y (mgr.14152) 340 : cluster 0 pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:32 smithi151 bash[23756]: audit 2024-03-04T15:49:30.953477+0000 mon.a (mon.0) 653 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:32 smithi151 bash[23756]: audit 2024-03-04T15:49:30.960997+0000 mon.a (mon.0) 654 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:32 smithi151 bash[23756]: audit 2024-03-04T15:49:30.982987+0000 mon.a (mon.0) 655 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:32 smithi151 bash[23756]: cephadm 2024-03-04T15:49:30.983730+0000 mgr.y (mgr.14152) 341 : cephadm 1 Saving service rgw.foo.a spec with placement smithi117=foo.a;count:1 2024-03-04T15:49:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:32 smithi151 bash[23756]: audit 2024-03-04T15:49:30.989169+0000 mon.a (mon.0) 656 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:32 smithi151 bash[23756]: audit 2024-03-04T15:49:30.994627+0000 mon.a (mon.0) 657 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:32 smithi151 bash[23756]: audit 2024-03-04T15:49:31.043356+0000 mon.a (mon.0) 658 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:49:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:32 smithi117 bash[21480]: cluster 2024-03-04T15:49:30.837952+0000 mgr.y (mgr.14152) 340 : cluster 0 pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:32 smithi117 bash[21480]: audit 2024-03-04T15:49:30.953477+0000 mon.a (mon.0) 653 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:32 smithi117 bash[21480]: audit 2024-03-04T15:49:30.960997+0000 mon.a (mon.0) 654 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:32.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:32 smithi117 bash[21480]: audit 2024-03-04T15:49:30.982987+0000 mon.a (mon.0) 655 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:32.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:32 smithi117 bash[21480]: cephadm 2024-03-04T15:49:30.983730+0000 mgr.y (mgr.14152) 341 : cephadm 1 Saving service rgw.foo.a spec with placement smithi117=foo.a;count:1 2024-03-04T15:49:32.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:32 smithi117 bash[21480]: audit 2024-03-04T15:49:30.989169+0000 mon.a (mon.0) 656 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:32.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:32 smithi117 bash[21480]: audit 2024-03-04T15:49:30.994627+0000 mon.a (mon.0) 657 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:32.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:32 smithi117 bash[21480]: audit 2024-03-04T15:49:31.043356+0000 mon.a (mon.0) 658 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:49:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:32 smithi117 bash[28843]: cluster 2024-03-04T15:49:30.837952+0000 mgr.y (mgr.14152) 340 : cluster 0 pgmap v323: 1 pgs: 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:32 smithi117 bash[28843]: audit 2024-03-04T15:49:30.953477+0000 mon.a (mon.0) 653 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:32 smithi117 bash[28843]: audit 2024-03-04T15:49:30.960997+0000 mon.a (mon.0) 654 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:32 smithi117 bash[28843]: audit 2024-03-04T15:49:30.982987+0000 mon.a (mon.0) 655 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:32 smithi117 bash[28843]: cephadm 2024-03-04T15:49:30.983730+0000 mgr.y (mgr.14152) 341 : cephadm 1 Saving service rgw.foo.a spec with placement smithi117=foo.a;count:1 2024-03-04T15:49:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:32 smithi117 bash[28843]: audit 2024-03-04T15:49:30.989169+0000 mon.a (mon.0) 656 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:32 smithi117 bash[28843]: audit 2024-03-04T15:49:30.994627+0000 mon.a (mon.0) 657 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:32.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:32 smithi117 bash[28843]: audit 2024-03-04T15:49:31.043356+0000 mon.a (mon.0) 658 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:49:33.238 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T15:49:33.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:33 smithi151 bash[23756]: audit 2024-03-04T15:49:32.364427+0000 mon.a (mon.0) 659 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:33.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:33 smithi151 bash[23756]: cluster 2024-03-04T15:49:32.701138+0000 mon.a (mon.0) 660 : cluster 0 osdmap e60: 8 total, 8 up, 8 in 2024-03-04T15:49:33.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:33 smithi151 bash[23756]: audit 2024-03-04T15:49:32.703611+0000 mon.a (mon.0) 661 : audit 1 from='client.? 172.21.15.117:0/3506201859' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-04T15:49:33.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:33 smithi117 bash[28843]: audit 2024-03-04T15:49:32.364427+0000 mon.a (mon.0) 659 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:33.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:33 smithi117 bash[28843]: cluster 2024-03-04T15:49:32.701138+0000 mon.a (mon.0) 660 : cluster 0 osdmap e60: 8 total, 8 up, 8 in 2024-03-04T15:49:33.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:33 smithi117 bash[28843]: audit 2024-03-04T15:49:32.703611+0000 mon.a (mon.0) 661 : audit 1 from='client.? 172.21.15.117:0/3506201859' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-04T15:49:33.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:33 smithi117 bash[21480]: audit 2024-03-04T15:49:32.364427+0000 mon.a (mon.0) 659 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:33.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:33 smithi117 bash[21480]: cluster 2024-03-04T15:49:32.701138+0000 mon.a (mon.0) 660 : cluster 0 osdmap e60: 8 total, 8 up, 8 in 2024-03-04T15:49:33.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:33 smithi117 bash[21480]: audit 2024-03-04T15:49:32.703611+0000 mon.a (mon.0) 661 : audit 1 from='client.? 172.21.15.117:0/3506201859' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-03-04T15:49:34.665 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:34 smithi151 bash[23756]: cluster 2024-03-04T15:49:32.838539+0000 mgr.y (mgr.14152) 342 : cluster 0 pgmap v325: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:34.665 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:34 smithi151 bash[23756]: cluster 2024-03-04T15:49:33.692553+0000 mon.a (mon.0) 662 : cluster 3 Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:49:34.665 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:34 smithi151 bash[23756]: audit 2024-03-04T15:49:33.703128+0000 mon.a (mon.0) 663 : audit 1 from='client.? 172.21.15.117:0/3506201859' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-03-04T15:49:34.665 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:34 smithi151 bash[23756]: cluster 2024-03-04T15:49:33.712185+0000 mon.a (mon.0) 664 : cluster 0 osdmap e61: 8 total, 8 up, 8 in 2024-03-04T15:49:34.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:34 smithi117 bash[21480]: cluster 2024-03-04T15:49:32.838539+0000 mgr.y (mgr.14152) 342 : cluster 0 pgmap v325: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:34.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:34 smithi117 bash[21480]: cluster 2024-03-04T15:49:33.692553+0000 mon.a (mon.0) 662 : cluster 3 Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:49:34.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:34 smithi117 bash[21480]: audit 2024-03-04T15:49:33.703128+0000 mon.a (mon.0) 663 : audit 1 from='client.? 172.21.15.117:0/3506201859' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-03-04T15:49:34.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:34 smithi117 bash[21480]: cluster 2024-03-04T15:49:33.712185+0000 mon.a (mon.0) 664 : cluster 0 osdmap e61: 8 total, 8 up, 8 in 2024-03-04T15:49:34.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:34 smithi117 bash[28843]: cluster 2024-03-04T15:49:32.838539+0000 mgr.y (mgr.14152) 342 : cluster 0 pgmap v325: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:34.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:34 smithi117 bash[28843]: cluster 2024-03-04T15:49:33.692553+0000 mon.a (mon.0) 662 : cluster 3 Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:49:34.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:34 smithi117 bash[28843]: audit 2024-03-04T15:49:33.703128+0000 mon.a (mon.0) 663 : audit 1 from='client.? 172.21.15.117:0/3506201859' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-03-04T15:49:34.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:34 smithi117 bash[28843]: cluster 2024-03-04T15:49:33.712185+0000 mon.a (mon.0) 664 : cluster 0 osdmap e61: 8 total, 8 up, 8 in 2024-03-04T15:49:36.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:35 smithi117 bash[21480]: cluster 2024-03-04T15:49:34.712090+0000 mon.a (mon.0) 665 : cluster 0 osdmap e62: 8 total, 8 up, 8 in 2024-03-04T15:49:36.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:35 smithi117 bash[21480]: audit 2024-03-04T15:49:34.715327+0000 mon.c (mon.2) 14 : audit 1 from='client.? 172.21.15.117:0/3875759579' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-04T15:49:36.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:35 smithi117 bash[21480]: audit 2024-03-04T15:49:34.716056+0000 mon.a (mon.0) 666 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-04T15:49:36.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:35 smithi117 bash[28843]: cluster 2024-03-04T15:49:34.712090+0000 mon.a (mon.0) 665 : cluster 0 osdmap e62: 8 total, 8 up, 8 in 2024-03-04T15:49:36.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:35 smithi117 bash[28843]: audit 2024-03-04T15:49:34.715327+0000 mon.c (mon.2) 14 : audit 1 from='client.? 172.21.15.117:0/3875759579' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-04T15:49:36.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:35 smithi117 bash[28843]: audit 2024-03-04T15:49:34.716056+0000 mon.a (mon.0) 666 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-04T15:49:36.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:35 smithi151 bash[23756]: cluster 2024-03-04T15:49:34.712090+0000 mon.a (mon.0) 665 : cluster 0 osdmap e62: 8 total, 8 up, 8 in 2024-03-04T15:49:36.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:35 smithi151 bash[23756]: audit 2024-03-04T15:49:34.715327+0000 mon.c (mon.2) 14 : audit 1 from='client.? 172.21.15.117:0/3875759579' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-04T15:49:36.168 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:35 smithi151 bash[23756]: audit 2024-03-04T15:49:34.716056+0000 mon.a (mon.0) 666 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-03-04T15:49:36.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2ecc762700 1 -- 172.21.15.151:0/3949666909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ec4101e60 msgr2=0x7f2ec4102280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:36.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2ecc762700 1 --2- 172.21.15.151:0/3949666909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ec4101e60 0x7f2ec4102280 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009990 tx=0x7f2ec0009ca0 comp rx=0 tx=0).stop 2024-03-04T15:49:36.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2ecc762700 1 -- 172.21.15.151:0/3949666909 shutdown_connections 2024-03-04T15:49:36.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2ecc762700 1 --2- 172.21.15.151:0/3949666909 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ec4103980 0x7f2ec410a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:36.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2ecc762700 1 --2- 172.21.15.151:0/3949666909 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ec4101e60 0x7f2ec4102280 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:36.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2ecc762700 1 --2- 172.21.15.151:0/3949666909 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ec4102fc0 0x7f2ec4103440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:36.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2ecc762700 1 -- 172.21.15.151:0/3949666909 >> 172.21.15.151:0/3949666909 conn(0x7f2ec40fd400 msgr2=0x7f2ec40ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:49:36.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2ecc762700 1 -- 172.21.15.151:0/3949666909 shutdown_connections 2024-03-04T15:49:36.325 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2ecc762700 1 -- 172.21.15.151:0/3949666909 wait complete. 2024-03-04T15:49:36.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2ecc762700 1 Processor -- start 2024-03-04T15:49:36.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2ecc762700 1 -- start start 2024-03-04T15:49:36.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2ecc762700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ec4101e60 0x7f2ec41975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:36.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2ecc762700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ec4102fc0 0x7f2ec4197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:36.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2ecc762700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ec4103980 0x7f2ec419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:36.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2ecc762700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec410ced0 con 0x7f2ec4102fc0 2024-03-04T15:49:36.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2ecc762700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec410cd50 con 0x7f2ec4103980 2024-03-04T15:49:36.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2ecc762700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec410d050 con 0x7f2ec4101e60 2024-03-04T15:49:36.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2ecacff700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ec4103980 0x7f2ec419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:36.327 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2eca4fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ec4101e60 0x7f2ec41975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:36.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2ec9cfd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ec4102fc0 0x7f2ec4197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:36.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2eca4fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ec4101e60 0x7f2ec41975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.151:48680/0 (socket says 172.21.15.151:48680) 2024-03-04T15:49:36.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.321+0000 7f2eca4fe700 1 -- 172.21.15.151:0/104730051 learned_addr learned my addr 172.21.15.151:0/104730051 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-04T15:49:36.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.325+0000 7f2ec9cfd700 1 -- 172.21.15.151:0/104730051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ec4101e60 msgr2=0x7f2ec41975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:36.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.325+0000 7f2ec9cfd700 1 --2- 172.21.15.151:0/104730051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ec4101e60 0x7f2ec41975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:36.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.325+0000 7f2ec9cfd700 1 -- 172.21.15.151:0/104730051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ec4103980 msgr2=0x7f2ec419eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:36.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.325+0000 7f2ec9cfd700 1 --2- 172.21.15.151:0/104730051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ec4103980 0x7f2ec419eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:36.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.325+0000 7f2ec9cfd700 1 -- 172.21.15.151:0/104730051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec4074010 con 0x7f2ec4102fc0 2024-03-04T15:49:36.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.325+0000 7f2ecacff700 1 --2- 172.21.15.151:0/104730051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ec4103980 0x7f2ec419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:49:36.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.325+0000 7f2eca4fe700 1 --2- 172.21.15.151:0/104730051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ec4101e60 0x7f2ec41975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:49:36.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.325+0000 7f2ec9cfd700 1 --2- 172.21.15.151:0/104730051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ec4102fc0 0x7f2ec4197b10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400ea20 tx=0x7f2eb400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:49:36.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.325+0000 7f2ebb7fe700 1 -- 172.21.15.151:0/104730051 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb400cbd0 con 0x7f2ec4102fc0 2024-03-04T15:49:36.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.325+0000 7f2ecc762700 1 -- 172.21.15.151:0/104730051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec4074300 con 0x7f2ec4102fc0 2024-03-04T15:49:36.328 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.325+0000 7f2ebb7fe700 1 -- 172.21.15.151:0/104730051 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f2eb400ee60 con 0x7f2ec4102fc0 2024-03-04T15:49:36.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.325+0000 7f2ebb7fe700 1 -- 172.21.15.151:0/104730051 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb4018420 con 0x7f2ec4102fc0 2024-03-04T15:49:36.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.325+0000 7f2ecc762700 1 -- 172.21.15.151:0/104730051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec4074890 con 0x7f2ec4102fc0 2024-03-04T15:49:36.329 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.325+0000 7f2ecc762700 1 -- 172.21.15.151:0/104730051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec404faa0 con 0x7f2ec4102fc0 2024-03-04T15:49:36.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.325+0000 7f2ebb7fe700 1 -- 172.21.15.151:0/104730051 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 15) v1 ==== 91397+0+0 (secure 0 0 0) 0x7f2eb4010430 con 0x7f2ec4102fc0 2024-03-04T15:49:36.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.325+0000 7f2ebb7fe700 1 --2- 172.21.15.151:0/104730051 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f2eb006d690 0x7f2eb006fb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:36.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.325+0000 7f2ebb7fe700 1 -- 172.21.15.151:0/104730051 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(63..63 src has 1..63) v4 ==== 6011+0+0 (secure 0 0 0) 0x7f2eb408d8f0 con 0x7f2ec4102fc0 2024-03-04T15:49:36.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.325+0000 7f2eca4fe700 1 --2- 172.21.15.151:0/104730051 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f2eb006d690 0x7f2eb006fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:36.330 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.325+0000 7f2eca4fe700 1 --2- 172.21.15.151:0/104730051 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f2eb006d690 0x7f2eb006fb50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009750 tx=0x7f2ec0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:49:36.334 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.333+0000 7f2ebb7fe700 1 -- 172.21.15.151:0/104730051 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f2eb405b720 con 0x7f2ec4102fc0 2024-03-04T15:49:36.548 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.545+0000 7f2ecc762700 1 -- 172.21.15.151:0/104730051 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) v1 -- 0x7f2ec4075e30 con 0x7f2ec4102fc0 2024-03-04T15:49:36.971 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.969+0000 7f2ebb7fe700 1 -- 172.21.15.151:0/104730051 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v64) v1 ==== 160+0+0 (secure 0 0 0) 0x7f2eb405b020 con 0x7f2ec4102fc0 2024-03-04T15:49:36.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.981+0000 7f2ecc762700 1 -- 172.21.15.151:0/104730051 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f2eb006d690 msgr2=0x7f2eb006fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:36.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.981+0000 7f2ecc762700 1 --2- 172.21.15.151:0/104730051 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f2eb006d690 0x7f2eb006fb50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009750 tx=0x7f2ec0005960 comp rx=0 tx=0).stop 2024-03-04T15:49:36.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.981+0000 7f2ecc762700 1 -- 172.21.15.151:0/104730051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ec4102fc0 msgr2=0x7f2ec4197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:36.983 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.981+0000 7f2ecc762700 1 --2- 172.21.15.151:0/104730051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ec4102fc0 0x7f2ec4197b10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2eb400ea20 tx=0x7f2eb400ed30 comp rx=0 tx=0).stop 2024-03-04T15:49:36.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.985+0000 7f2ecc762700 1 -- 172.21.15.151:0/104730051 shutdown_connections 2024-03-04T15:49:36.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.985+0000 7f2ecc762700 1 --2- 172.21.15.151:0/104730051 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f2eb006d690 0x7f2eb006fb50 secure :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f2ec0009750 tx=0x7f2ec0005960 comp rx=0 tx=0).stop 2024-03-04T15:49:36.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.985+0000 7f2ecc762700 1 --2- 172.21.15.151:0/104730051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f2ec4103980 0x7f2ec419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:36.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.985+0000 7f2ecc762700 1 --2- 172.21.15.151:0/104730051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f2ec4101e60 0x7f2ec41975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:36.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.985+0000 7f2ecc762700 1 --2- 172.21.15.151:0/104730051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f2ec4102fc0 0x7f2ec4197b10 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:36.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.985+0000 7f2ecc762700 1 -- 172.21.15.151:0/104730051 >> 172.21.15.151:0/104730051 conn(0x7f2ec40fd400 msgr2=0x7f2ec40fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:49:36.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.985+0000 7f2ecc762700 1 -- 172.21.15.151:0/104730051 shutdown_connections 2024-03-04T15:49:36.988 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:36.985+0000 7f2ecc762700 1 -- 172.21.15.151:0/104730051 wait complete. 2024-03-04T15:49:36.989 INFO:teuthology.orchestra.run.smithi151.stderr:pool 'datapool' created 2024-03-04T15:49:37.230 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:36 smithi151 bash[23756]: cluster 2024-03-04T15:49:34.839261+0000 mgr.y (mgr.14152) 343 : cluster 0 pgmap v328: 65 pgs: 4 creating+peering, 1 active+clean, 60 unknown; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:37.230 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:36 smithi151 bash[23756]: audit 2024-03-04T15:49:35.717522+0000 mon.a (mon.0) 667 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-04T15:49:37.230 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:36 smithi151 bash[23756]: cluster 2024-03-04T15:49:35.727752+0000 mon.a (mon.0) 668 : cluster 0 osdmap e63: 8 total, 8 up, 8 in 2024-03-04T15:49:37.230 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:36 smithi151 bash[23756]: audit 2024-03-04T15:49:36.378051+0000 mon.a (mon.0) 669 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:37.230 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:36 smithi151 bash[23756]: audit 2024-03-04T15:49:36.383055+0000 mon.a (mon.0) 670 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:37.230 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:36 smithi151 bash[23756]: audit 2024-03-04T15:49:36.384048+0000 mon.a (mon.0) 671 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:37.230 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:36 smithi151 bash[23756]: audit 2024-03-04T15:49:36.384762+0000 mon.a (mon.0) 672 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:49:37.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:36 smithi117 bash[21480]: cluster 2024-03-04T15:49:34.839261+0000 mgr.y (mgr.14152) 343 : cluster 0 pgmap v328: 65 pgs: 4 creating+peering, 1 active+clean, 60 unknown; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:37.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:36 smithi117 bash[21480]: audit 2024-03-04T15:49:35.717522+0000 mon.a (mon.0) 667 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-04T15:49:37.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:36 smithi117 bash[21480]: cluster 2024-03-04T15:49:35.727752+0000 mon.a (mon.0) 668 : cluster 0 osdmap e63: 8 total, 8 up, 8 in 2024-03-04T15:49:37.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:36 smithi117 bash[21480]: audit 2024-03-04T15:49:36.378051+0000 mon.a (mon.0) 669 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:37.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:36 smithi117 bash[21480]: audit 2024-03-04T15:49:36.383055+0000 mon.a (mon.0) 670 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:37.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:36 smithi117 bash[21480]: audit 2024-03-04T15:49:36.384048+0000 mon.a (mon.0) 671 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:37.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:36 smithi117 bash[21480]: audit 2024-03-04T15:49:36.384762+0000 mon.a (mon.0) 672 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:49:37.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:36 smithi117 bash[28843]: cluster 2024-03-04T15:49:34.839261+0000 mgr.y (mgr.14152) 343 : cluster 0 pgmap v328: 65 pgs: 4 creating+peering, 1 active+clean, 60 unknown; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:49:37.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:36 smithi117 bash[28843]: audit 2024-03-04T15:49:35.717522+0000 mon.a (mon.0) 667 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-03-04T15:49:37.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:36 smithi117 bash[28843]: cluster 2024-03-04T15:49:35.727752+0000 mon.a (mon.0) 668 : cluster 0 osdmap e63: 8 total, 8 up, 8 in 2024-03-04T15:49:37.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:36 smithi117 bash[28843]: audit 2024-03-04T15:49:36.378051+0000 mon.a (mon.0) 669 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:37.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:36 smithi117 bash[28843]: audit 2024-03-04T15:49:36.383055+0000 mon.a (mon.0) 670 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:37.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:36 smithi117 bash[28843]: audit 2024-03-04T15:49:36.384048+0000 mon.a (mon.0) 671 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:37.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:36 smithi117 bash[28843]: audit 2024-03-04T15:49:36.384762+0000 mon.a (mon.0) 672 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:49:37.808 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- rbd pool init datapool 2024-03-04T15:49:38.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:37 smithi151 bash[23756]: cephadm 2024-03-04T15:49:36.388984+0000 mgr.y (mgr.14152) 344 : cephadm 1 Checking dashboard <-> RGW credentials 2024-03-04T15:49:38.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:37 smithi151 bash[23756]: audit 2024-03-04T15:49:36.549959+0000 mon.a (mon.0) 673 : audit 1 from='client.? 172.21.15.151:0/104730051' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-04T15:49:38.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:37 smithi151 bash[23756]: cluster 2024-03-04T15:49:36.840294+0000 mgr.y (mgr.14152) 345 : cluster 0 pgmap v330: 65 pgs: 26 active+clean, 19 creating+peering, 20 unknown; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 247 B/s rd, 494 B/s wr, 0 op/s 2024-03-04T15:49:38.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:37 smithi151 bash[23756]: audit 2024-03-04T15:49:36.971701+0000 mon.a (mon.0) 674 : audit 1 from='client.? 172.21.15.151:0/104730051' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-03-04T15:49:38.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:37 smithi151 bash[23756]: cluster 2024-03-04T15:49:36.974659+0000 mon.a (mon.0) 675 : cluster 0 osdmap e64: 8 total, 8 up, 8 in 2024-03-04T15:49:38.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:37 smithi151 bash[23756]: audit 2024-03-04T15:49:36.981079+0000 mon.c (mon.2) 15 : audit 1 from='client.? 172.21.15.117:0/3875759579' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-04T15:49:38.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:37 smithi151 bash[23756]: audit 2024-03-04T15:49:36.981675+0000 mon.a (mon.0) 676 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-04T15:49:38.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:37 smithi151 bash[23756]: cluster 2024-03-04T15:49:37.381464+0000 mon.a (mon.0) 677 : cluster 1 Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-04T15:49:38.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:37 smithi151 bash[23756]: cluster 2024-03-04T15:49:37.381527+0000 mon.a (mon.0) 678 : cluster 1 Cluster is now healthy 2024-03-04T15:49:38.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:37 smithi117 bash[21480]: cephadm 2024-03-04T15:49:36.388984+0000 mgr.y (mgr.14152) 344 : cephadm 1 Checking dashboard <-> RGW credentials 2024-03-04T15:49:38.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:37 smithi117 bash[21480]: audit 2024-03-04T15:49:36.549959+0000 mon.a (mon.0) 673 : audit 1 from='client.? 172.21.15.151:0/104730051' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-04T15:49:38.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:37 smithi117 bash[21480]: cluster 2024-03-04T15:49:36.840294+0000 mgr.y (mgr.14152) 345 : cluster 0 pgmap v330: 65 pgs: 26 active+clean, 19 creating+peering, 20 unknown; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 247 B/s rd, 494 B/s wr, 0 op/s 2024-03-04T15:49:38.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:37 smithi117 bash[21480]: audit 2024-03-04T15:49:36.971701+0000 mon.a (mon.0) 674 : audit 1 from='client.? 172.21.15.151:0/104730051' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-03-04T15:49:38.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:37 smithi117 bash[21480]: cluster 2024-03-04T15:49:36.974659+0000 mon.a (mon.0) 675 : cluster 0 osdmap e64: 8 total, 8 up, 8 in 2024-03-04T15:49:38.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:37 smithi117 bash[21480]: audit 2024-03-04T15:49:36.981079+0000 mon.c (mon.2) 15 : audit 1 from='client.? 172.21.15.117:0/3875759579' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-04T15:49:38.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:37 smithi117 bash[21480]: audit 2024-03-04T15:49:36.981675+0000 mon.a (mon.0) 676 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-04T15:49:38.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:37 smithi117 bash[21480]: cluster 2024-03-04T15:49:37.381464+0000 mon.a (mon.0) 677 : cluster 1 Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-04T15:49:38.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:37 smithi117 bash[21480]: cluster 2024-03-04T15:49:37.381527+0000 mon.a (mon.0) 678 : cluster 1 Cluster is now healthy 2024-03-04T15:49:38.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:37 smithi117 bash[28843]: cephadm 2024-03-04T15:49:36.388984+0000 mgr.y (mgr.14152) 344 : cephadm 1 Checking dashboard <-> RGW credentials 2024-03-04T15:49:38.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:37 smithi117 bash[28843]: audit 2024-03-04T15:49:36.549959+0000 mon.a (mon.0) 673 : audit 1 from='client.? 172.21.15.151:0/104730051' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-03-04T15:49:38.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:37 smithi117 bash[28843]: cluster 2024-03-04T15:49:36.840294+0000 mgr.y (mgr.14152) 345 : cluster 0 pgmap v330: 65 pgs: 26 active+clean, 19 creating+peering, 20 unknown; 577 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 247 B/s rd, 494 B/s wr, 0 op/s 2024-03-04T15:49:38.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:37 smithi117 bash[28843]: audit 2024-03-04T15:49:36.971701+0000 mon.a (mon.0) 674 : audit 1 from='client.? 172.21.15.151:0/104730051' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-03-04T15:49:38.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:37 smithi117 bash[28843]: cluster 2024-03-04T15:49:36.974659+0000 mon.a (mon.0) 675 : cluster 0 osdmap e64: 8 total, 8 up, 8 in 2024-03-04T15:49:38.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:37 smithi117 bash[28843]: audit 2024-03-04T15:49:36.981079+0000 mon.c (mon.2) 15 : audit 1 from='client.? 172.21.15.117:0/3875759579' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-04T15:49:38.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:37 smithi117 bash[28843]: audit 2024-03-04T15:49:36.981675+0000 mon.a (mon.0) 676 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-03-04T15:49:38.323 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:37 smithi117 bash[28843]: cluster 2024-03-04T15:49:37.381464+0000 mon.a (mon.0) 677 : cluster 1 Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-04T15:49:38.323 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:37 smithi117 bash[28843]: cluster 2024-03-04T15:49:37.381527+0000 mon.a (mon.0) 678 : cluster 1 Cluster is now healthy 2024-03-04T15:49:39.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:39 smithi151 bash[23756]: audit 2024-03-04T15:49:37.979618+0000 mon.a (mon.0) 679 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-04T15:49:39.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:39 smithi151 bash[23756]: cluster 2024-03-04T15:49:37.990122+0000 mon.a (mon.0) 680 : cluster 0 osdmap e65: 8 total, 8 up, 8 in 2024-03-04T15:49:39.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:39 smithi117 bash[21480]: audit 2024-03-04T15:49:37.979618+0000 mon.a (mon.0) 679 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-04T15:49:39.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:39 smithi117 bash[21480]: cluster 2024-03-04T15:49:37.990122+0000 mon.a (mon.0) 680 : cluster 0 osdmap e65: 8 total, 8 up, 8 in 2024-03-04T15:49:39.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:39 smithi117 bash[28843]: audit 2024-03-04T15:49:37.979618+0000 mon.a (mon.0) 679 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-03-04T15:49:39.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:39 smithi117 bash[28843]: cluster 2024-03-04T15:49:37.990122+0000 mon.a (mon.0) 680 : cluster 0 osdmap e65: 8 total, 8 up, 8 in 2024-03-04T15:49:40.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:40 smithi117 bash[21480]: cluster 2024-03-04T15:49:38.841214+0000 mgr.y (mgr.14152) 346 : cluster 0 pgmap v333: 100 pgs: 68 active+clean, 17 creating+peering, 15 unknown; 578 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-03-04T15:49:40.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:40 smithi117 bash[21480]: cluster 2024-03-04T15:49:39.187072+0000 mon.a (mon.0) 681 : cluster 0 osdmap e66: 8 total, 8 up, 8 in 2024-03-04T15:49:40.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:40 smithi117 bash[21480]: audit 2024-03-04T15:49:39.190881+0000 mon.c (mon.2) 16 : audit 1 from='client.? 172.21.15.117:0/3875759579' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-04T15:49:40.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:40 smithi117 bash[21480]: audit 2024-03-04T15:49:39.193991+0000 mon.a (mon.0) 682 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-04T15:49:40.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:40 smithi117 bash[21480]: audit 2024-03-04T15:49:39.196508+0000 mon.a (mon.0) 683 : audit 1 from='client.? 172.21.15.117:0/2165456101' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-04T15:49:40.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:40 smithi117 bash[21480]: cluster 2024-03-04T15:49:39.220836+0000 mon.a (mon.0) 684 : cluster 3 Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:49:40.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:40 smithi117 bash[28843]: cluster 2024-03-04T15:49:38.841214+0000 mgr.y (mgr.14152) 346 : cluster 0 pgmap v333: 100 pgs: 68 active+clean, 17 creating+peering, 15 unknown; 578 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-03-04T15:49:40.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:40 smithi117 bash[28843]: cluster 2024-03-04T15:49:39.187072+0000 mon.a (mon.0) 681 : cluster 0 osdmap e66: 8 total, 8 up, 8 in 2024-03-04T15:49:40.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:40 smithi117 bash[28843]: audit 2024-03-04T15:49:39.190881+0000 mon.c (mon.2) 16 : audit 1 from='client.? 172.21.15.117:0/3875759579' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-04T15:49:40.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:40 smithi117 bash[28843]: audit 2024-03-04T15:49:39.193991+0000 mon.a (mon.0) 682 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-04T15:49:40.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:40 smithi117 bash[28843]: audit 2024-03-04T15:49:39.196508+0000 mon.a (mon.0) 683 : audit 1 from='client.? 172.21.15.117:0/2165456101' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-04T15:49:40.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:40 smithi117 bash[28843]: cluster 2024-03-04T15:49:39.220836+0000 mon.a (mon.0) 684 : cluster 3 Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:49:40.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:40 smithi151 bash[23756]: cluster 2024-03-04T15:49:38.841214+0000 mgr.y (mgr.14152) 346 : cluster 0 pgmap v333: 100 pgs: 68 active+clean, 17 creating+peering, 15 unknown; 578 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 5 op/s 2024-03-04T15:49:40.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:40 smithi151 bash[23756]: cluster 2024-03-04T15:49:39.187072+0000 mon.a (mon.0) 681 : cluster 0 osdmap e66: 8 total, 8 up, 8 in 2024-03-04T15:49:40.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:40 smithi151 bash[23756]: audit 2024-03-04T15:49:39.190881+0000 mon.c (mon.2) 16 : audit 1 from='client.? 172.21.15.117:0/3875759579' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-04T15:49:40.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:40 smithi151 bash[23756]: audit 2024-03-04T15:49:39.193991+0000 mon.a (mon.0) 682 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-04T15:49:40.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:40 smithi151 bash[23756]: audit 2024-03-04T15:49:39.196508+0000 mon.a (mon.0) 683 : audit 1 from='client.? 172.21.15.117:0/2165456101' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-03-04T15:49:40.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:40 smithi151 bash[23756]: cluster 2024-03-04T15:49:39.220836+0000 mon.a (mon.0) 684 : cluster 3 Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:49:41.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:41 smithi117 bash[28843]: audit 2024-03-04T15:49:40.193497+0000 mon.a (mon.0) 685 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-04T15:49:41.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:41 smithi117 bash[28843]: audit 2024-03-04T15:49:40.193701+0000 mon.a (mon.0) 686 : audit 1 from='client.? 172.21.15.117:0/2165456101' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-04T15:49:41.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:41 smithi117 bash[28843]: cluster 2024-03-04T15:49:40.203090+0000 mon.a (mon.0) 687 : cluster 0 osdmap e67: 8 total, 8 up, 8 in 2024-03-04T15:49:41.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:41 smithi117 bash[28843]: audit 2024-03-04T15:49:40.212090+0000 mon.c (mon.2) 17 : audit 1 from='client.? 172.21.15.117:0/3875759579' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-04T15:49:41.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:41 smithi117 bash[28843]: audit 2024-03-04T15:49:40.213311+0000 mon.a (mon.0) 688 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-04T15:49:41.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:41 smithi117 bash[28843]: audit 2024-03-04T15:49:40.214702+0000 mon.a (mon.0) 689 : audit 1 from='client.? 172.21.15.117:0/2165456101' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-04T15:49:41.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:41 smithi117 bash[21480]: audit 2024-03-04T15:49:40.193497+0000 mon.a (mon.0) 685 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-04T15:49:41.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:41 smithi117 bash[21480]: audit 2024-03-04T15:49:40.193701+0000 mon.a (mon.0) 686 : audit 1 from='client.? 172.21.15.117:0/2165456101' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-04T15:49:41.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:41 smithi117 bash[21480]: cluster 2024-03-04T15:49:40.203090+0000 mon.a (mon.0) 687 : cluster 0 osdmap e67: 8 total, 8 up, 8 in 2024-03-04T15:49:41.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:41 smithi117 bash[21480]: audit 2024-03-04T15:49:40.212090+0000 mon.c (mon.2) 17 : audit 1 from='client.? 172.21.15.117:0/3875759579' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-04T15:49:41.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:41 smithi117 bash[21480]: audit 2024-03-04T15:49:40.213311+0000 mon.a (mon.0) 688 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-04T15:49:41.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:41 smithi117 bash[21480]: audit 2024-03-04T15:49:40.214702+0000 mon.a (mon.0) 689 : audit 1 from='client.? 172.21.15.117:0/2165456101' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-04T15:49:41.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:41 smithi151 bash[23756]: audit 2024-03-04T15:49:40.193497+0000 mon.a (mon.0) 685 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-04T15:49:41.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:41 smithi151 bash[23756]: audit 2024-03-04T15:49:40.193701+0000 mon.a (mon.0) 686 : audit 1 from='client.? 172.21.15.117:0/2165456101' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-03-04T15:49:41.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:41 smithi151 bash[23756]: cluster 2024-03-04T15:49:40.203090+0000 mon.a (mon.0) 687 : cluster 0 osdmap e67: 8 total, 8 up, 8 in 2024-03-04T15:49:41.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:41 smithi151 bash[23756]: audit 2024-03-04T15:49:40.212090+0000 mon.c (mon.2) 17 : audit 1 from='client.? 172.21.15.117:0/3875759579' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-04T15:49:41.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:41 smithi151 bash[23756]: audit 2024-03-04T15:49:40.213311+0000 mon.a (mon.0) 688 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-04T15:49:41.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:41 smithi151 bash[23756]: audit 2024-03-04T15:49:40.214702+0000 mon.a (mon.0) 689 : audit 1 from='client.? 172.21.15.117:0/2165456101' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-03-04T15:49:42.562 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T15:49:42.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[21480]: cluster 2024-03-04T15:49:40.842311+0000 mgr.y (mgr.14152) 347 : cluster 0 pgmap v336: 132 pgs: 107 active+clean, 3 creating+activating, 2 creating+peering, 20 unknown; 578 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-04T15:49:42.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[21480]: audit 2024-03-04T15:49:41.204224+0000 mon.a (mon.0) 690 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-04T15:49:42.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[21480]: audit 2024-03-04T15:49:41.204394+0000 mon.a (mon.0) 691 : audit 1 from='client.? 172.21.15.117:0/2165456101' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-04T15:49:42.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[21480]: cluster 2024-03-04T15:49:41.216266+0000 mon.a (mon.0) 692 : cluster 0 osdmap e68: 8 total, 8 up, 8 in 2024-03-04T15:49:42.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[21480]: audit 2024-03-04T15:49:41.492377+0000 mon.a (mon.0) 693 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:42.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[21480]: audit 2024-03-04T15:49:41.498295+0000 mon.a (mon.0) 694 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:42.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[21480]: audit 2024-03-04T15:49:41.503856+0000 mon.a (mon.0) 695 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:42.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[21480]: audit 2024-03-04T15:49:41.569790+0000 mon.a (mon.0) 696 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:49:42.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[21480]: audit 2024-03-04T15:49:41.571748+0000 mon.a (mon.0) 697 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:42.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[21480]: audit 2024-03-04T15:49:41.572882+0000 mon.a (mon.0) 698 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:49:42.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[21480]: cephadm 2024-03-04T15:49:41.577914+0000 mgr.y (mgr.14152) 348 : cephadm 1 Checking dashboard <-> RGW credentials 2024-03-04T15:49:42.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[21480]: audit 2024-03-04T15:49:41.852645+0000 mon.a (mon.0) 699 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:42.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[28843]: cluster 2024-03-04T15:49:40.842311+0000 mgr.y (mgr.14152) 347 : cluster 0 pgmap v336: 132 pgs: 107 active+clean, 3 creating+activating, 2 creating+peering, 20 unknown; 578 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-04T15:49:42.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[28843]: audit 2024-03-04T15:49:41.204224+0000 mon.a (mon.0) 690 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-04T15:49:42.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[28843]: audit 2024-03-04T15:49:41.204394+0000 mon.a (mon.0) 691 : audit 1 from='client.? 172.21.15.117:0/2165456101' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-04T15:49:42.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[28843]: cluster 2024-03-04T15:49:41.216266+0000 mon.a (mon.0) 692 : cluster 0 osdmap e68: 8 total, 8 up, 8 in 2024-03-04T15:49:42.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[28843]: audit 2024-03-04T15:49:41.492377+0000 mon.a (mon.0) 693 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:42.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[28843]: audit 2024-03-04T15:49:41.498295+0000 mon.a (mon.0) 694 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:42.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[28843]: audit 2024-03-04T15:49:41.503856+0000 mon.a (mon.0) 695 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:42.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[28843]: audit 2024-03-04T15:49:41.569790+0000 mon.a (mon.0) 696 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:49:42.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[28843]: audit 2024-03-04T15:49:41.571748+0000 mon.a (mon.0) 697 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:42.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[28843]: audit 2024-03-04T15:49:41.572882+0000 mon.a (mon.0) 698 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:49:42.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[28843]: cephadm 2024-03-04T15:49:41.577914+0000 mgr.y (mgr.14152) 348 : cephadm 1 Checking dashboard <-> RGW credentials 2024-03-04T15:49:42.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:42 smithi117 bash[28843]: audit 2024-03-04T15:49:41.852645+0000 mon.a (mon.0) 699 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:42.582 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:42 smithi151 bash[23756]: cluster 2024-03-04T15:49:40.842311+0000 mgr.y (mgr.14152) 347 : cluster 0 pgmap v336: 132 pgs: 107 active+clean, 3 creating+activating, 2 creating+peering, 20 unknown; 578 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 4.0 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-03-04T15:49:42.582 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:42 smithi151 bash[23756]: audit 2024-03-04T15:49:41.204224+0000 mon.a (mon.0) 690 : audit 1 from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-04T15:49:42.582 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:42 smithi151 bash[23756]: audit 2024-03-04T15:49:41.204394+0000 mon.a (mon.0) 691 : audit 1 from='client.? 172.21.15.117:0/2165456101' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-03-04T15:49:42.583 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:42 smithi151 bash[23756]: cluster 2024-03-04T15:49:41.216266+0000 mon.a (mon.0) 692 : cluster 0 osdmap e68: 8 total, 8 up, 8 in 2024-03-04T15:49:42.583 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:42 smithi151 bash[23756]: audit 2024-03-04T15:49:41.492377+0000 mon.a (mon.0) 693 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:42.583 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:42 smithi151 bash[23756]: audit 2024-03-04T15:49:41.498295+0000 mon.a (mon.0) 694 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:42.583 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:42 smithi151 bash[23756]: audit 2024-03-04T15:49:41.503856+0000 mon.a (mon.0) 695 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:42.583 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:42 smithi151 bash[23756]: audit 2024-03-04T15:49:41.569790+0000 mon.a (mon.0) 696 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:49:42.583 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:42 smithi151 bash[23756]: audit 2024-03-04T15:49:41.571748+0000 mon.a (mon.0) 697 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:42.584 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:42 smithi151 bash[23756]: audit 2024-03-04T15:49:41.572882+0000 mon.a (mon.0) 698 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:49:42.584 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:42 smithi151 bash[23756]: cephadm 2024-03-04T15:49:41.577914+0000 mgr.y (mgr.14152) 348 : cephadm 1 Checking dashboard <-> RGW credentials 2024-03-04T15:49:42.584 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:42 smithi151 bash[23756]: audit 2024-03-04T15:49:41.852645+0000 mon.a (mon.0) 699 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:44.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:44 smithi151 bash[23756]: cluster 2024-03-04T15:49:42.843611+0000 mgr.y (mgr.14152) 349 : cluster 0 pgmap v338: 132 pgs: 117 active+clean, 3 creating+activating, 12 unknown; 579 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 22 KiB/s rd, 2.1 KiB/s wr, 50 op/s 2024-03-04T15:49:44.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:44 smithi117 bash[21480]: cluster 2024-03-04T15:49:42.843611+0000 mgr.y (mgr.14152) 349 : cluster 0 pgmap v338: 132 pgs: 117 active+clean, 3 creating+activating, 12 unknown; 579 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 22 KiB/s rd, 2.1 KiB/s wr, 50 op/s 2024-03-04T15:49:44.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:44 smithi117 bash[28843]: cluster 2024-03-04T15:49:42.843611+0000 mgr.y (mgr.14152) 349 : cluster 0 pgmap v338: 132 pgs: 117 active+clean, 3 creating+activating, 12 unknown; 579 KiB data, 246 MiB used, 715 GiB / 715 GiB avail; 22 KiB/s rd, 2.1 KiB/s wr, 50 op/s 2024-03-04T15:49:46.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:46 smithi117 bash[21480]: cluster 2024-03-04T15:49:44.844761+0000 mgr.y (mgr.14152) 350 : cluster 0 pgmap v339: 132 pgs: 129 active+clean, 3 creating+activating; 580 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 3.3 KiB/s wr, 97 op/s 2024-03-04T15:49:46.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:46 smithi117 bash[21480]: audit 2024-03-04T15:49:45.601450+0000 mon.b (mon.1) 26 : audit 1 from='client.? 172.21.15.151:0/95846283' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-04T15:49:46.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:46 smithi117 bash[21480]: audit 2024-03-04T15:49:45.602797+0000 mon.a (mon.0) 700 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-04T15:49:46.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:46 smithi117 bash[28843]: cluster 2024-03-04T15:49:44.844761+0000 mgr.y (mgr.14152) 350 : cluster 0 pgmap v339: 132 pgs: 129 active+clean, 3 creating+activating; 580 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 3.3 KiB/s wr, 97 op/s 2024-03-04T15:49:46.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:46 smithi117 bash[28843]: audit 2024-03-04T15:49:45.601450+0000 mon.b (mon.1) 26 : audit 1 from='client.? 172.21.15.151:0/95846283' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-04T15:49:46.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:46 smithi117 bash[28843]: audit 2024-03-04T15:49:45.602797+0000 mon.a (mon.0) 700 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-04T15:49:46.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:46 smithi151 bash[23756]: cluster 2024-03-04T15:49:44.844761+0000 mgr.y (mgr.14152) 350 : cluster 0 pgmap v339: 132 pgs: 129 active+clean, 3 creating+activating; 580 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 41 KiB/s rd, 3.3 KiB/s wr, 97 op/s 2024-03-04T15:49:46.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:46 smithi151 bash[23756]: audit 2024-03-04T15:49:45.601450+0000 mon.b (mon.1) 26 : audit 1 from='client.? 172.21.15.151:0/95846283' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-04T15:49:46.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:46 smithi151 bash[23756]: audit 2024-03-04T15:49:45.602797+0000 mon.a (mon.0) 700 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-03-04T15:49:47.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:47 smithi117 bash[21480]: audit 2024-03-04T15:49:46.222812+0000 mon.a (mon.0) 701 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-03-04T15:49:47.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:47 smithi117 bash[21480]: cluster 2024-03-04T15:49:46.229932+0000 mon.a (mon.0) 702 : cluster 0 osdmap e69: 8 total, 8 up, 8 in 2024-03-04T15:49:47.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:47 smithi117 bash[28843]: audit 2024-03-04T15:49:46.222812+0000 mon.a (mon.0) 701 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-03-04T15:49:47.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:47 smithi117 bash[28843]: cluster 2024-03-04T15:49:46.229932+0000 mon.a (mon.0) 702 : cluster 0 osdmap e69: 8 total, 8 up, 8 in 2024-03-04T15:49:47.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:47 smithi151 bash[23756]: audit 2024-03-04T15:49:46.222812+0000 mon.a (mon.0) 701 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-03-04T15:49:47.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:47 smithi151 bash[23756]: cluster 2024-03-04T15:49:46.229932+0000 mon.a (mon.0) 702 : cluster 0 osdmap e69: 8 total, 8 up, 8 in 2024-03-04T15:49:48.528 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:48 smithi151 bash[23756]: cluster 2024-03-04T15:49:46.845626+0000 mgr.y (mgr.14152) 351 : cluster 0 pgmap v341: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 5.4 KiB/s wr, 172 op/s 2024-03-04T15:49:48.528 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:48 smithi151 bash[23756]: cluster 2024-03-04T15:49:47.222775+0000 mon.a (mon.0) 703 : cluster 1 Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-04T15:49:48.528 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:48 smithi151 bash[23756]: cluster 2024-03-04T15:49:47.222833+0000 mon.a (mon.0) 704 : cluster 1 Cluster is now healthy 2024-03-04T15:49:48.528 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:48 smithi151 bash[23756]: cluster 2024-03-04T15:49:47.239489+0000 mon.a (mon.0) 705 : cluster 0 osdmap e70: 8 total, 8 up, 8 in 2024-03-04T15:49:48.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:48 smithi117 bash[21480]: cluster 2024-03-04T15:49:46.845626+0000 mgr.y (mgr.14152) 351 : cluster 0 pgmap v341: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 5.4 KiB/s wr, 172 op/s 2024-03-04T15:49:48.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:48 smithi117 bash[21480]: cluster 2024-03-04T15:49:47.222775+0000 mon.a (mon.0) 703 : cluster 1 Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-04T15:49:48.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:48 smithi117 bash[21480]: cluster 2024-03-04T15:49:47.222833+0000 mon.a (mon.0) 704 : cluster 1 Cluster is now healthy 2024-03-04T15:49:48.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:48 smithi117 bash[21480]: cluster 2024-03-04T15:49:47.239489+0000 mon.a (mon.0) 705 : cluster 0 osdmap e70: 8 total, 8 up, 8 in 2024-03-04T15:49:48.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:48 smithi117 bash[28843]: cluster 2024-03-04T15:49:46.845626+0000 mgr.y (mgr.14152) 351 : cluster 0 pgmap v341: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 73 KiB/s rd, 5.4 KiB/s wr, 172 op/s 2024-03-04T15:49:48.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:48 smithi117 bash[28843]: cluster 2024-03-04T15:49:47.222775+0000 mon.a (mon.0) 703 : cluster 1 Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-03-04T15:49:48.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:48 smithi117 bash[28843]: cluster 2024-03-04T15:49:47.222833+0000 mon.a (mon.0) 704 : cluster 1 Cluster is now healthy 2024-03-04T15:49:48.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:48 smithi117 bash[28843]: cluster 2024-03-04T15:49:47.239489+0000 mon.a (mon.0) 705 : cluster 0 osdmap e70: 8 total, 8 up, 8 in 2024-03-04T15:49:49.135 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.151 --placement '1;smithi151=iscsi.a' 2024-03-04T15:49:49.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:49 smithi151 bash[23756]: cluster 2024-03-04T15:49:48.241593+0000 mon.a (mon.0) 706 : cluster 0 osdmap e71: 8 total, 8 up, 8 in 2024-03-04T15:49:49.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:49 smithi117 bash[21480]: cluster 2024-03-04T15:49:48.241593+0000 mon.a (mon.0) 706 : cluster 0 osdmap e71: 8 total, 8 up, 8 in 2024-03-04T15:49:49.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:49 smithi117 bash[28843]: cluster 2024-03-04T15:49:48.241593+0000 mon.a (mon.0) 706 : cluster 0 osdmap e71: 8 total, 8 up, 8 in 2024-03-04T15:49:50.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:50 smithi117 bash[21480]: cluster 2024-03-04T15:49:48.846600+0000 mgr.y (mgr.14152) 352 : cluster 0 pgmap v344: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 4.8 KiB/s wr, 151 op/s 2024-03-04T15:49:50.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:50 smithi117 bash[28843]: cluster 2024-03-04T15:49:48.846600+0000 mgr.y (mgr.14152) 352 : cluster 0 pgmap v344: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 4.8 KiB/s wr, 151 op/s 2024-03-04T15:49:50.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:50 smithi151 bash[23756]: cluster 2024-03-04T15:49:48.846600+0000 mgr.y (mgr.14152) 352 : cluster 0 pgmap v344: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 64 KiB/s rd, 4.8 KiB/s wr, 151 op/s 2024-03-04T15:49:52.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:52 smithi117 bash[21480]: cluster 2024-03-04T15:49:50.847772+0000 mgr.y (mgr.14152) 353 : cluster 0 pgmap v345: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 39 KiB/s rd, 2.7 KiB/s wr, 93 op/s 2024-03-04T15:49:52.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:52 smithi117 bash[28843]: cluster 2024-03-04T15:49:50.847772+0000 mgr.y (mgr.14152) 353 : cluster 0 pgmap v345: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 39 KiB/s rd, 2.7 KiB/s wr, 93 op/s 2024-03-04T15:49:52.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:52 smithi151 bash[23756]: cluster 2024-03-04T15:49:50.847772+0000 mgr.y (mgr.14152) 353 : cluster 0 pgmap v345: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 39 KiB/s rd, 2.7 KiB/s wr, 93 op/s 2024-03-04T15:49:53.888 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T15:49:54.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:54 smithi117 bash[21480]: cluster 2024-03-04T15:49:52.848700+0000 mgr.y (mgr.14152) 354 : cluster 0 pgmap v346: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 772 B/s rd, 309 B/s wr, 2 op/s 2024-03-04T15:49:54.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:54 smithi117 bash[28843]: cluster 2024-03-04T15:49:52.848700+0000 mgr.y (mgr.14152) 354 : cluster 0 pgmap v346: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 772 B/s rd, 309 B/s wr, 2 op/s 2024-03-04T15:49:54.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:54 smithi151 bash[23756]: cluster 2024-03-04T15:49:52.848700+0000 mgr.y (mgr.14152) 354 : cluster 0 pgmap v346: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 772 B/s rd, 309 B/s wr, 2 op/s 2024-03-04T15:49:56.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:56 smithi117 bash[21480]: cluster 2024-03-04T15:49:54.849860+0000 mgr.y (mgr.14152) 355 : cluster 0 pgmap v347: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-04T15:49:56.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:56 smithi117 bash[28843]: cluster 2024-03-04T15:49:54.849860+0000 mgr.y (mgr.14152) 355 : cluster 0 pgmap v347: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-04T15:49:56.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:56 smithi151 bash[23756]: cluster 2024-03-04T15:49:54.849860+0000 mgr.y (mgr.14152) 355 : cluster 0 pgmap v347: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-03-04T15:49:56.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.929+0000 7f1c4d3eb700 1 -- 172.21.15.151:0/905993263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c48102fd0 msgr2=0x7f1c48103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:56.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.929+0000 7f1c4d3eb700 1 --2- 172.21.15.151:0/905993263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c48102fd0 0x7f1c48103450 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1c3c009990 tx=0x7f1c3c009ca0 comp rx=0 tx=0).stop 2024-03-04T15:49:56.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.929+0000 7f1c4d3eb700 1 -- 172.21.15.151:0/905993263 shutdown_connections 2024-03-04T15:49:56.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.929+0000 7f1c4d3eb700 1 --2- 172.21.15.151:0/905993263 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c48101e70 0x7f1c48102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:56.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.929+0000 7f1c4d3eb700 1 --2- 172.21.15.151:0/905993263 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c48103990 0x7f1c4810a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:56.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.929+0000 7f1c4d3eb700 1 --2- 172.21.15.151:0/905993263 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c48102fd0 0x7f1c48103450 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:56.933 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.929+0000 7f1c4d3eb700 1 -- 172.21.15.151:0/905993263 >> 172.21.15.151:0/905993263 conn(0x7f1c480fd410 msgr2=0x7f1c480ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:49:56.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.929+0000 7f1c4d3eb700 1 -- 172.21.15.151:0/905993263 shutdown_connections 2024-03-04T15:49:56.934 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.929+0000 7f1c4d3eb700 1 -- 172.21.15.151:0/905993263 wait complete. 2024-03-04T15:49:56.935 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.929+0000 7f1c4d3eb700 1 Processor -- start 2024-03-04T15:49:56.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.929+0000 7f1c4d3eb700 1 -- start start 2024-03-04T15:49:56.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.929+0000 7f1c4d3eb700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c48101e70 0x7f1c48197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:56.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.929+0000 7f1c4d3eb700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c48102fd0 0x7f1c48197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:56.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.929+0000 7f1c4d3eb700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c48103990 0x7f1c4819ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:56.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.929+0000 7f1c4d3eb700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4810cf30 con 0x7f1c48101e70 2024-03-04T15:49:56.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.929+0000 7f1c4d3eb700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4810cdb0 con 0x7f1c48103990 2024-03-04T15:49:56.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.929+0000 7f1c4d3eb700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c4810d0b0 con 0x7f1c48102fd0 2024-03-04T15:49:56.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c477fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c48103990 0x7f1c4819ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:56.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c477fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c48103990 0x7f1c4819ebe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40688/0 (socket says 172.21.15.151:40688) 2024-03-04T15:49:56.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c477fe700 1 -- 172.21.15.151:0/1387152993 learned_addr learned my addr 172.21.15.151:0/1387152993 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-04T15:49:56.936 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c46ffd700 1 --2- 172.21.15.151:0/1387152993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c48101e70 0x7f1c48197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:56.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c467fc700 1 --2- 172.21.15.151:0/1387152993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c48102fd0 0x7f1c48197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:56.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c477fe700 1 -- 172.21.15.151:0/1387152993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c48102fd0 msgr2=0x7f1c48197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:56.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c477fe700 1 --2- 172.21.15.151:0/1387152993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c48102fd0 0x7f1c48197bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:56.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c477fe700 1 -- 172.21.15.151:0/1387152993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c48101e70 msgr2=0x7f1c48197670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:56.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c477fe700 1 --2- 172.21.15.151:0/1387152993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c48101e70 0x7f1c48197670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:56.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c477fe700 1 -- 172.21.15.151:0/1387152993 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c4819f280 con 0x7f1c48103990 2024-03-04T15:49:56.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c467fc700 1 --2- 172.21.15.151:0/1387152993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c48102fd0 0x7f1c48197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T15:49:56.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c46ffd700 1 --2- 172.21.15.151:0/1387152993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c48101e70 0x7f1c48197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T15:49:56.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c477fe700 1 --2- 172.21.15.151:0/1387152993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c48103990 0x7f1c4819ebe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1c40007ee0 tx=0x7f1c4000d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:49:56.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c2ffff700 1 -- 172.21.15.151:0/1387152993 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c4000dbd0 con 0x7f1c48103990 2024-03-04T15:49:56.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c4d3eb700 1 -- 172.21.15.151:0/1387152993 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c4819f570 con 0x7f1c48103990 2024-03-04T15:49:56.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c2ffff700 1 -- 172.21.15.151:0/1387152993 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f1c4000dd70 con 0x7f1c48103990 2024-03-04T15:49:56.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c2ffff700 1 -- 172.21.15.151:0/1387152993 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c40005890 con 0x7f1c48103990 2024-03-04T15:49:56.937 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c4d3eb700 1 -- 172.21.15.151:0/1387152993 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c4819fb00 con 0x7f1c48103990 2024-03-04T15:49:56.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c4d3eb700 1 -- 172.21.15.151:0/1387152993 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c4804faa0 con 0x7f1c48103990 2024-03-04T15:49:56.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c2ffff700 1 -- 172.21.15.151:0/1387152993 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 15) v1 ==== 91397+0+0 (secure 0 0 0) 0x7f1c4001a020 con 0x7f1c48103990 2024-03-04T15:49:56.938 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c2ffff700 1 --2- 172.21.15.151:0/1387152993 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f1c3006d570 0x7f1c3006fa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:49:56.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c2ffff700 1 -- 172.21.15.151:0/1387152993 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(71..71 src has 1..71) v4 ==== 7176+0+0 (secure 0 0 0) 0x7f1c4008c770 con 0x7f1c48103990 2024-03-04T15:49:56.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.933+0000 7f1c46ffd700 1 --2- 172.21.15.151:0/1387152993 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f1c3006d570 0x7f1c3006fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:49:56.939 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.937+0000 7f1c46ffd700 1 --2- 172.21.15.151:0/1387152993 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f1c3006d570 0x7f1c3006fa30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1c38005fd0 tx=0x7f1c38005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:49:56.942 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:56.937+0000 7f1c2ffff700 1 -- 172.21.15.151:0/1387152993 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f1c40010070 con 0x7f1c48103990 2024-03-04T15:49:57.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:57.161+0000 7f1c4d3eb700 1 -- 172.21.15.151:0/1387152993 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.151", "placement": "1;smithi151=iscsi.a", "target": ["mon-mgr", ""]}) v1 -- 0x7f1c4802da50 con 0x7f1c3006d570 2024-03-04T15:49:57.183 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:57.181+0000 7f1c2ffff700 1 -- 172.21.15.151:0/1387152993 <== mgr.14152 v2:172.21.15.117:6800/2865911351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+35 (secure 0 0 0) 0x7f1c4802da50 con 0x7f1c3006d570 2024-03-04T15:49:57.183 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled iscsi.datapool update... 2024-03-04T15:49:57.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:57.181+0000 7f1c4d3eb700 1 -- 172.21.15.151:0/1387152993 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f1c3006d570 msgr2=0x7f1c3006fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:57.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:57.181+0000 7f1c4d3eb700 1 --2- 172.21.15.151:0/1387152993 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f1c3006d570 0x7f1c3006fa30 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1c38005fd0 tx=0x7f1c38005ee0 comp rx=0 tx=0).stop 2024-03-04T15:49:57.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:57.181+0000 7f1c4d3eb700 1 -- 172.21.15.151:0/1387152993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c48103990 msgr2=0x7f1c4819ebe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:49:57.185 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:57.181+0000 7f1c4d3eb700 1 --2- 172.21.15.151:0/1387152993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c48103990 0x7f1c4819ebe0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f1c40007ee0 tx=0x7f1c4000d3b0 comp rx=0 tx=0).stop 2024-03-04T15:49:57.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:57.181+0000 7f1c4d3eb700 1 -- 172.21.15.151:0/1387152993 shutdown_connections 2024-03-04T15:49:57.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:57.181+0000 7f1c4d3eb700 1 --2- 172.21.15.151:0/1387152993 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1c48101e70 0x7f1c48197670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:57.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:57.181+0000 7f1c4d3eb700 1 --2- 172.21.15.151:0/1387152993 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f1c3006d570 0x7f1c3006fa30 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:57.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:57.181+0000 7f1c4d3eb700 1 --2- 172.21.15.151:0/1387152993 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1c48103990 0x7f1c4819ebe0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:57.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:57.181+0000 7f1c4d3eb700 1 --2- 172.21.15.151:0/1387152993 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f1c48102fd0 0x7f1c48197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:49:57.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:57.181+0000 7f1c4d3eb700 1 -- 172.21.15.151:0/1387152993 >> 172.21.15.151:0/1387152993 conn(0x7f1c480fd410 msgr2=0x7f1c480fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:49:57.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:57.181+0000 7f1c4d3eb700 1 -- 172.21.15.151:0/1387152993 shutdown_connections 2024-03-04T15:49:57.186 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:49:57.181+0000 7f1c4d3eb700 1 -- 172.21.15.151:0/1387152993 wait complete. 2024-03-04T15:49:58.014 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-03-04T15:49:58.015 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-04T15:49:58.015 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-03-04T15:49:58.032 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-04T15:49:58.032 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-03-04T15:49:58.045 DEBUG:teuthology.orchestra.run.smithi151:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@iscsi.iscsi.a.service 2024-03-04T15:49:58.088 INFO:tasks.cephadm:Adding prometheus.a on smithi151 2024-03-04T15:49:58.088 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph orch apply prometheus '1;smithi151=a' 2024-03-04T15:49:58.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[21480]: cluster 2024-03-04T15:49:56.851264+0000 mgr.y (mgr.14152) 356 : cluster 0 pgmap v348: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 212 B/s wr, 0 op/s 2024-03-04T15:49:58.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[21480]: cluster 2024-03-04T15:49:56.975956+0000 mon.a (mon.0) 707 : cluster 0 osdmap e72: 8 total, 8 up, 8 in 2024-03-04T15:49:58.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[21480]: audit 2024-03-04T15:49:57.169413+0000 mgr.y (mgr.14152) 357 : audit 0 from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.151", "placement": "1;smithi151=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:49:58.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[21480]: cephadm 2024-03-04T15:49:57.175738+0000 mgr.y (mgr.14152) 358 : cephadm 1 Saving service iscsi.datapool spec with placement smithi151=iscsi.a;count:1 2024-03-04T15:49:58.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[21480]: audit 2024-03-04T15:49:57.183171+0000 mon.a (mon.0) 708 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:58.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[21480]: audit 2024-03-04T15:49:57.211012+0000 mon.a (mon.0) 709 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:49:58.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[21480]: audit 2024-03-04T15:49:57.885531+0000 mon.a (mon.0) 710 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:58.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[21480]: audit 2024-03-04T15:49:57.887815+0000 mon.a (mon.0) 711 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:49:58.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[21480]: audit 2024-03-04T15:49:57.899065+0000 mon.a (mon.0) 712 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:58.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[21480]: audit 2024-03-04T15:49:57.906435+0000 mon.a (mon.0) 713 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-04T15:49:58.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[21480]: audit 2024-03-04T15:49:57.911720+0000 mon.a (mon.0) 714 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-03-04T15:49:58.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[21480]: audit 2024-03-04T15:49:57.941698+0000 mon.a (mon.0) 715 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:58.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[28843]: cluster 2024-03-04T15:49:56.851264+0000 mgr.y (mgr.14152) 356 : cluster 0 pgmap v348: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 212 B/s wr, 0 op/s 2024-03-04T15:49:58.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[28843]: cluster 2024-03-04T15:49:56.975956+0000 mon.a (mon.0) 707 : cluster 0 osdmap e72: 8 total, 8 up, 8 in 2024-03-04T15:49:58.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[28843]: audit 2024-03-04T15:49:57.169413+0000 mgr.y (mgr.14152) 357 : audit 0 from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.151", "placement": "1;smithi151=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:49:58.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[28843]: cephadm 2024-03-04T15:49:57.175738+0000 mgr.y (mgr.14152) 358 : cephadm 1 Saving service iscsi.datapool spec with placement smithi151=iscsi.a;count:1 2024-03-04T15:49:58.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[28843]: audit 2024-03-04T15:49:57.183171+0000 mon.a (mon.0) 708 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:58.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[28843]: audit 2024-03-04T15:49:57.211012+0000 mon.a (mon.0) 709 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:49:58.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[28843]: audit 2024-03-04T15:49:57.885531+0000 mon.a (mon.0) 710 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:58.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[28843]: audit 2024-03-04T15:49:57.887815+0000 mon.a (mon.0) 711 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:49:58.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[28843]: audit 2024-03-04T15:49:57.899065+0000 mon.a (mon.0) 712 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:58.323 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[28843]: audit 2024-03-04T15:49:57.906435+0000 mon.a (mon.0) 713 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-04T15:49:58.323 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[28843]: audit 2024-03-04T15:49:57.911720+0000 mon.a (mon.0) 714 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-03-04T15:49:58.323 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:57 smithi117 bash[28843]: audit 2024-03-04T15:49:57.941698+0000 mon.a (mon.0) 715 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:58.351 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:57 smithi151 bash[23756]: cluster 2024-03-04T15:49:56.851264+0000 mgr.y (mgr.14152) 356 : cluster 0 pgmap v348: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 212 B/s wr, 0 op/s 2024-03-04T15:49:58.352 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:57 smithi151 bash[23756]: cluster 2024-03-04T15:49:56.975956+0000 mon.a (mon.0) 707 : cluster 0 osdmap e72: 8 total, 8 up, 8 in 2024-03-04T15:49:58.352 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:57 smithi151 bash[23756]: audit 2024-03-04T15:49:57.169413+0000 mgr.y (mgr.14152) 357 : audit 0 from='client.24418 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.151", "placement": "1;smithi151=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:49:58.352 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:57 smithi151 bash[23756]: cephadm 2024-03-04T15:49:57.175738+0000 mgr.y (mgr.14152) 358 : cephadm 1 Saving service iscsi.datapool spec with placement smithi151=iscsi.a;count:1 2024-03-04T15:49:58.352 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:57 smithi151 bash[23756]: audit 2024-03-04T15:49:57.183171+0000 mon.a (mon.0) 708 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:58.352 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:57 smithi151 bash[23756]: audit 2024-03-04T15:49:57.211012+0000 mon.a (mon.0) 709 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:49:58.352 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:57 smithi151 bash[23756]: audit 2024-03-04T15:49:57.885531+0000 mon.a (mon.0) 710 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:58.352 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:57 smithi151 bash[23756]: audit 2024-03-04T15:49:57.887815+0000 mon.a (mon.0) 711 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:49:58.352 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:57 smithi151 bash[23756]: audit 2024-03-04T15:49:57.899065+0000 mon.a (mon.0) 712 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:49:58.352 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:57 smithi151 bash[23756]: audit 2024-03-04T15:49:57.906435+0000 mon.a (mon.0) 713 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-03-04T15:49:58.352 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:57 smithi151 bash[23756]: audit 2024-03-04T15:49:57.911720+0000 mon.a (mon.0) 714 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-03-04T15:49:58.352 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:57 smithi151 bash[23756]: audit 2024-03-04T15:49:57.941698+0000 mon.a (mon.0) 715 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:49:59.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:58 smithi117 bash[21480]: cephadm 2024-03-04T15:49:57.943773+0000 mgr.y (mgr.14152) 359 : cephadm 1 Deploying daemon iscsi.iscsi.a on smithi151 2024-03-04T15:49:59.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:58 smithi117 bash[28843]: cephadm 2024-03-04T15:49:57.943773+0000 mgr.y (mgr.14152) 359 : cephadm 1 Deploying daemon iscsi.iscsi.a on smithi151 2024-03-04T15:49:59.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:58 smithi151 bash[23756]: cephadm 2024-03-04T15:49:57.943773+0000 mgr.y (mgr.14152) 359 : cephadm 1 Deploying daemon iscsi.iscsi.a on smithi151 2024-03-04T15:50:00.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:49:59 smithi151 bash[23756]: cluster 2024-03-04T15:49:58.852422+0000 mgr.y (mgr.14152) 360 : cluster 0 pgmap v350: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 204 B/s wr, 0 op/s 2024-03-04T15:50:00.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:49:59 smithi117 bash[28843]: cluster 2024-03-04T15:49:58.852422+0000 mgr.y (mgr.14152) 360 : cluster 0 pgmap v350: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 204 B/s wr, 0 op/s 2024-03-04T15:50:00.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:49:59 smithi117 bash[21480]: cluster 2024-03-04T15:49:58.852422+0000 mgr.y (mgr.14152) 360 : cluster 0 pgmap v350: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 204 B/s wr, 0 op/s 2024-03-04T15:50:00.606 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:00 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:00.606 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 15:50:00 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:00.606 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 15:50:00 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:00.607 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:00 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:00.607 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 15:50:00 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:00.607 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 15:50:00 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:00.607 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:50:00 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:00.884 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:00 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:00.884 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 15:50:00 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:00.884 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 15:50:00 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:00.885 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 15:50:00 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:00.885 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 15:50:00 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:00.885 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:00 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:00.886 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:50:00 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:00.886 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:50:00 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:00.886 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:50:00 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:00.886 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:50:00 smithi151 systemd[1]: Started Ceph iscsi.iscsi.a for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T15:50:01.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:00 smithi151 bash[23756]: cluster 2024-03-04T15:50:00.000142+0000 mon.a (mon.0) 716 : cluster 1 overall HEALTH_OK 2024-03-04T15:50:01.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:00 smithi151 bash[23756]: audit 2024-03-04T15:50:00.910979+0000 mon.a (mon.0) 717 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:01.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:00 smithi151 bash[23756]: audit 2024-03-04T15:50:00.923672+0000 mon.a (mon.0) 718 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:01.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:00 smithi151 bash[23756]: audit 2024-03-04T15:50:00.975057+0000 mon.a (mon.0) 719 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:01.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:00 smithi117 bash[21480]: cluster 2024-03-04T15:50:00.000142+0000 mon.a (mon.0) 716 : cluster 1 overall HEALTH_OK 2024-03-04T15:50:01.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:00 smithi117 bash[21480]: audit 2024-03-04T15:50:00.910979+0000 mon.a (mon.0) 717 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:01.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:00 smithi117 bash[21480]: audit 2024-03-04T15:50:00.923672+0000 mon.a (mon.0) 718 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:01.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:00 smithi117 bash[21480]: audit 2024-03-04T15:50:00.975057+0000 mon.a (mon.0) 719 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:01.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:00 smithi117 bash[28843]: cluster 2024-03-04T15:50:00.000142+0000 mon.a (mon.0) 716 : cluster 1 overall HEALTH_OK 2024-03-04T15:50:01.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:00 smithi117 bash[28843]: audit 2024-03-04T15:50:00.910979+0000 mon.a (mon.0) 717 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:01.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:00 smithi117 bash[28843]: audit 2024-03-04T15:50:00.923672+0000 mon.a (mon.0) 718 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:01.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:00 smithi117 bash[28843]: audit 2024-03-04T15:50:00.975057+0000 mon.a (mon.0) 719 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:02.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:01 smithi117 bash[21480]: cluster 2024-03-04T15:50:00.853477+0000 mgr.y (mgr.14152) 361 : cluster 0 pgmap v351: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 204 B/s wr, 0 op/s 2024-03-04T15:50:02.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:01 smithi117 bash[21480]: cephadm 2024-03-04T15:50:00.976853+0000 mgr.y (mgr.14152) 362 : cephadm 1 Checking pool "datapool" exists for service iscsi.datapool 2024-03-04T15:50:02.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:01 smithi117 bash[21480]: audit 2024-03-04T15:50:00.996685+0000 mon.a (mon.0) 720 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:02.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:01 smithi117 bash[21480]: audit 2024-03-04T15:50:01.061618+0000 mon.a (mon.0) 721 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:50:02.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:01 smithi117 bash[28843]: cluster 2024-03-04T15:50:00.853477+0000 mgr.y (mgr.14152) 361 : cluster 0 pgmap v351: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 204 B/s wr, 0 op/s 2024-03-04T15:50:02.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:01 smithi117 bash[28843]: cephadm 2024-03-04T15:50:00.976853+0000 mgr.y (mgr.14152) 362 : cephadm 1 Checking pool "datapool" exists for service iscsi.datapool 2024-03-04T15:50:02.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:01 smithi117 bash[28843]: audit 2024-03-04T15:50:00.996685+0000 mon.a (mon.0) 720 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:02.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:01 smithi117 bash[28843]: audit 2024-03-04T15:50:01.061618+0000 mon.a (mon.0) 721 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:50:02.339 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:01 smithi151 bash[23756]: cluster 2024-03-04T15:50:00.853477+0000 mgr.y (mgr.14152) 361 : cluster 0 pgmap v351: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 204 B/s wr, 0 op/s 2024-03-04T15:50:02.339 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:01 smithi151 bash[23756]: cephadm 2024-03-04T15:50:00.976853+0000 mgr.y (mgr.14152) 362 : cephadm 1 Checking pool "datapool" exists for service iscsi.datapool 2024-03-04T15:50:02.339 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:01 smithi151 bash[23756]: audit 2024-03-04T15:50:00.996685+0000 mon.a (mon.0) 720 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:02.339 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:01 smithi151 bash[23756]: audit 2024-03-04T15:50:01.061618+0000 mon.a (mon.0) 721 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:50:02.857 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T15:50:03.007 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:50:02 smithi151 bash[48108]: debug Started the configuration object watcher 2024-03-04T15:50:03.007 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:50:02 smithi151 bash[48108]: debug Checking for config object changes every 1s 2024-03-04T15:50:03.007 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:50:02 smithi151 bash[48108]: debug Processing osd blocklist entries for this node 2024-03-04T15:50:03.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:03 smithi117 bash[21480]: cluster 2024-03-04T15:50:02.022206+0000 mon.a (mon.0) 722 : cluster 0 mgrmap e16: y(active, since 9m), standbys: x 2024-03-04T15:50:03.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:03 smithi117 bash[21480]: audit 2024-03-04T15:50:02.373692+0000 mon.a (mon.0) 723 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:03.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:03 smithi117 bash[28843]: cluster 2024-03-04T15:50:02.022206+0000 mon.a (mon.0) 722 : cluster 0 mgrmap e16: y(active, since 9m), standbys: x 2024-03-04T15:50:03.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:03 smithi117 bash[28843]: audit 2024-03-04T15:50:02.373692+0000 mon.a (mon.0) 723 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:03.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:03 smithi151 bash[23756]: cluster 2024-03-04T15:50:02.022206+0000 mon.a (mon.0) 722 : cluster 0 mgrmap e16: y(active, since 9m), standbys: x 2024-03-04T15:50:03.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:03 smithi151 bash[23756]: audit 2024-03-04T15:50:02.373692+0000 mon.a (mon.0) 723 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:03.432 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:50:03 smithi151 bash[48108]: debug Reading the configuration object to update local LIO configuration 2024-03-04T15:50:03.432 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:50:03 smithi151 bash[48108]: debug Configuration does not have an entry for this host(smithi151.front.sepia.ceph.com) - nothing to define to LIO 2024-03-04T15:50:03.432 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:50:03 smithi151 bash[48108]: debug * Running on http://[::]:5000/ (Press CTRL+C to quit) 2024-03-04T15:50:04.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:04 smithi117 bash[21480]: cluster 2024-03-04T15:50:02.854467+0000 mgr.y (mgr.14152) 363 : cluster 0 pgmap v352: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:50:04.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:04 smithi117 bash[21480]: audit 2024-03-04T15:50:03.085728+0000 mon.b (mon.1) 27 : audit 0 from='client.? 172.21.15.151:0/3754638790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T15:50:04.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:04 smithi117 bash[28843]: cluster 2024-03-04T15:50:02.854467+0000 mgr.y (mgr.14152) 363 : cluster 0 pgmap v352: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:50:04.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:04 smithi117 bash[28843]: audit 2024-03-04T15:50:03.085728+0000 mon.b (mon.1) 27 : audit 0 from='client.? 172.21.15.151:0/3754638790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T15:50:04.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:04 smithi151 bash[23756]: cluster 2024-03-04T15:50:02.854467+0000 mgr.y (mgr.14152) 363 : cluster 0 pgmap v352: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:50:04.417 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:04 smithi151 bash[23756]: audit 2024-03-04T15:50:03.085728+0000 mon.b (mon.1) 27 : audit 0 from='client.? 172.21.15.151:0/3754638790' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-03-04T15:50:06.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.004+0000 7ffb30aca700 1 -- 172.21.15.151:0/3143325079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb2c102fd0 msgr2=0x7ffb2c103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:06.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.004+0000 7ffb30aca700 1 --2- 172.21.15.151:0/3143325079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb2c102fd0 0x7ffb2c103450 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ffb2000b0a0 tx=0x7ffb2000b3b0 comp rx=0 tx=0).stop 2024-03-04T15:50:06.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.004+0000 7ffb30aca700 1 -- 172.21.15.151:0/3143325079 shutdown_connections 2024-03-04T15:50:06.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.004+0000 7ffb30aca700 1 --2- 172.21.15.151:0/3143325079 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb2c103990 0x7ffb2c10a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:06.010 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.004+0000 7ffb30aca700 1 --2- 172.21.15.151:0/3143325079 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb2c101e70 0x7ffb2c102290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:06.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.004+0000 7ffb30aca700 1 --2- 172.21.15.151:0/3143325079 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb2c102fd0 0x7ffb2c103450 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:06.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.004+0000 7ffb30aca700 1 -- 172.21.15.151:0/3143325079 >> 172.21.15.151:0/3143325079 conn(0x7ffb2c0fd410 msgr2=0x7ffb2c0ff850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:50:06.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.004+0000 7ffb30aca700 1 -- 172.21.15.151:0/3143325079 shutdown_connections 2024-03-04T15:50:06.011 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.004+0000 7ffb30aca700 1 -- 172.21.15.151:0/3143325079 wait complete. 2024-03-04T15:50:06.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb30aca700 1 Processor -- start 2024-03-04T15:50:06.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb30aca700 1 -- start start 2024-03-04T15:50:06.012 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb30aca700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb2c101e70 0x7ffb2c075b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:06.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb30aca700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb2c102fd0 0x7ffb2c0741d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:06.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb30aca700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb2c103990 0x7ffb2c074710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:06.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb30aca700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c10cf00 con 0x7ffb2c103990 2024-03-04T15:50:06.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb30aca700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c10cd80 con 0x7ffb2c101e70 2024-03-04T15:50:06.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb30aca700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2c10d080 con 0x7ffb2c102fd0 2024-03-04T15:50:06.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb2a59c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb2c101e70 0x7ffb2c075b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:50:06.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb2a59c700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb2c101e70 0x7ffb2c075b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:40370/0 (socket says 172.21.15.151:40370) 2024-03-04T15:50:06.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb2a59c700 1 -- 172.21.15.151:0/1653271925 learned_addr learned my addr 172.21.15.151:0/1653271925 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-04T15:50:06.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb2ad9d700 1 --2- 172.21.15.151:0/1653271925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb2c103990 0x7ffb2c074710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:50:06.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb29d9b700 1 --2- 172.21.15.151:0/1653271925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb2c102fd0 0x7ffb2c0741d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:50:06.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb2a59c700 1 -- 172.21.15.151:0/1653271925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb2c102fd0 msgr2=0x7ffb2c0741d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:06.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb2a59c700 1 --2- 172.21.15.151:0/1653271925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb2c102fd0 0x7ffb2c0741d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:06.016 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb2a59c700 1 -- 172.21.15.151:0/1653271925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb2c103990 msgr2=0x7ffb2c074710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:06.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb2a59c700 1 --2- 172.21.15.151:0/1653271925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb2c103990 0x7ffb2c074710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:06.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb2a59c700 1 -- 172.21.15.151:0/1653271925 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2c074fe0 con 0x7ffb2c101e70 2024-03-04T15:50:06.017 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb2a59c700 1 --2- 172.21.15.151:0/1653271925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb2c101e70 0x7ffb2c075b30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00c920 tx=0x7ffb1c00cce0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:50:06.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb1b7fe700 1 -- 172.21.15.151:0/1653271925 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c007b00 con 0x7ffb2c101e70 2024-03-04T15:50:06.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb1b7fe700 1 -- 172.21.15.151:0/1653271925 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ffb1c004cb0 con 0x7ffb2c101e70 2024-03-04T15:50:06.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb1b7fe700 1 -- 172.21.15.151:0/1653271925 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c005530 con 0x7ffb2c101e70 2024-03-04T15:50:06.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb30aca700 1 -- 172.21.15.151:0/1653271925 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2c1a17d0 con 0x7ffb2c101e70 2024-03-04T15:50:06.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.008+0000 7ffb30aca700 1 -- 172.21.15.151:0/1653271925 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb2c1a1c90 con 0x7ffb2c101e70 2024-03-04T15:50:06.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.012+0000 7ffb30aca700 1 -- 172.21.15.151:0/1653271925 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb2c10d6c0 con 0x7ffb2c101e70 2024-03-04T15:50:06.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.012+0000 7ffb1b7fe700 1 -- 172.21.15.151:0/1653271925 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) v1 ==== 91448+0+0 (secure 0 0 0) 0x7ffb1c004750 con 0x7ffb2c101e70 2024-03-04T15:50:06.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.012+0000 7ffb1b7fe700 1 --2- 172.21.15.151:0/1653271925 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7ffb1406d720 0x7ffb1406fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:06.018 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.012+0000 7ffb1b7fe700 1 -- 172.21.15.151:0/1653271925 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7148+0+0 (secure 0 0 0) 0x7ffb1c08d7b0 con 0x7ffb2c101e70 2024-03-04T15:50:06.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.012+0000 7ffb29d9b700 1 --2- 172.21.15.151:0/1653271925 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7ffb1406d720 0x7ffb1406fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:50:06.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.012+0000 7ffb29d9b700 1 --2- 172.21.15.151:0/1653271925 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7ffb1406d720 0x7ffb1406fbe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffb20000f80 tx=0x7ffb2000b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:50:06.020 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.016+0000 7ffb1b7fe700 1 -- 172.21.15.151:0/1653271925 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ffb1c014030 con 0x7ffb2c101e70 2024-03-04T15:50:06.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:06 smithi151 bash[23756]: cluster 2024-03-04T15:50:04.855540+0000 mgr.y (mgr.14152) 364 : cluster 0 pgmap v353: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 102 B/s rd, 102 B/s wr, 0 op/s 2024-03-04T15:50:06.241 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.236+0000 7ffb30aca700 1 -- 172.21.15.151:0/1653271925 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi151=a", "target": ["mon-mgr", ""]}) v1 -- 0x7ffb2c02daa0 con 0x7ffb1406d720 2024-03-04T15:50:06.255 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.252+0000 7ffb1b7fe700 1 -- 172.21.15.151:0/1653271925 <== mgr.14152 v2:172.21.15.117:6800/2865911351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7ffb2c02daa0 con 0x7ffb1406d720 2024-03-04T15:50:06.255 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled prometheus update... 2024-03-04T15:50:06.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.252+0000 7ffb30aca700 1 -- 172.21.15.151:0/1653271925 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7ffb1406d720 msgr2=0x7ffb1406fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:06.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.252+0000 7ffb30aca700 1 --2- 172.21.15.151:0/1653271925 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7ffb1406d720 0x7ffb1406fbe0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ffb20000f80 tx=0x7ffb2000b7b0 comp rx=0 tx=0).stop 2024-03-04T15:50:06.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.252+0000 7ffb30aca700 1 -- 172.21.15.151:0/1653271925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb2c101e70 msgr2=0x7ffb2c075b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:06.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.252+0000 7ffb30aca700 1 --2- 172.21.15.151:0/1653271925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb2c101e70 0x7ffb2c075b30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ffb1c00c920 tx=0x7ffb1c00cce0 comp rx=0 tx=0).stop 2024-03-04T15:50:06.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.252+0000 7ffb30aca700 1 -- 172.21.15.151:0/1653271925 shutdown_connections 2024-03-04T15:50:06.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.252+0000 7ffb30aca700 1 --2- 172.21.15.151:0/1653271925 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ffb2c103990 0x7ffb2c074710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:06.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.252+0000 7ffb30aca700 1 --2- 172.21.15.151:0/1653271925 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7ffb1406d720 0x7ffb1406fbe0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:06.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.252+0000 7ffb30aca700 1 --2- 172.21.15.151:0/1653271925 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ffb2c101e70 0x7ffb2c075b30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:06.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.252+0000 7ffb30aca700 1 --2- 172.21.15.151:0/1653271925 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ffb2c102fd0 0x7ffb2c0741d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:06.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.252+0000 7ffb30aca700 1 -- 172.21.15.151:0/1653271925 >> 172.21.15.151:0/1653271925 conn(0x7ffb2c0fd410 msgr2=0x7ffb2c0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:50:06.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.252+0000 7ffb30aca700 1 -- 172.21.15.151:0/1653271925 shutdown_connections 2024-03-04T15:50:06.258 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:06.252+0000 7ffb30aca700 1 -- 172.21.15.151:0/1653271925 wait complete. 2024-03-04T15:50:06.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:06 smithi117 bash[21480]: cluster 2024-03-04T15:50:04.855540+0000 mgr.y (mgr.14152) 364 : cluster 0 pgmap v353: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 102 B/s rd, 102 B/s wr, 0 op/s 2024-03-04T15:50:06.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:06 smithi117 bash[28843]: cluster 2024-03-04T15:50:04.855540+0000 mgr.y (mgr.14152) 364 : cluster 0 pgmap v353: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 102 B/s rd, 102 B/s wr, 0 op/s 2024-03-04T15:50:06.983 DEBUG:teuthology.orchestra.run.smithi151:prometheus.a> sudo journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@prometheus.a.service 2024-03-04T15:50:06.985 INFO:tasks.cephadm:Adding node-exporter.a on smithi117 2024-03-04T15:50:06.985 INFO:tasks.cephadm:Adding node-exporter.b on smithi151 2024-03-04T15:50:06.985 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph orch apply node-exporter '2;smithi117=a;smithi151=b' 2024-03-04T15:50:07.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:07 smithi117 bash[21480]: audit 2024-03-04T15:50:06.244672+0000 mgr.y (mgr.14152) 365 : audit 0 from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi151=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:50:07.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:07 smithi117 bash[21480]: cephadm 2024-03-04T15:50:06.248272+0000 mgr.y (mgr.14152) 366 : cephadm 1 Saving service prometheus spec with placement smithi151=a;count:1 2024-03-04T15:50:07.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:07 smithi117 bash[21480]: audit 2024-03-04T15:50:06.255736+0000 mon.a (mon.0) 724 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:07.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:07 smithi117 bash[28843]: audit 2024-03-04T15:50:06.244672+0000 mgr.y (mgr.14152) 365 : audit 0 from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi151=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:50:07.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:07 smithi117 bash[28843]: cephadm 2024-03-04T15:50:06.248272+0000 mgr.y (mgr.14152) 366 : cephadm 1 Saving service prometheus spec with placement smithi151=a;count:1 2024-03-04T15:50:07.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:07 smithi117 bash[28843]: audit 2024-03-04T15:50:06.255736+0000 mon.a (mon.0) 724 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:07.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:07 smithi151 bash[23756]: audit 2024-03-04T15:50:06.244672+0000 mgr.y (mgr.14152) 365 : audit 0 from='client.24454 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi151=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:50:07.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:07 smithi151 bash[23756]: cephadm 2024-03-04T15:50:06.248272+0000 mgr.y (mgr.14152) 366 : cephadm 1 Saving service prometheus spec with placement smithi151=a;count:1 2024-03-04T15:50:07.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:07 smithi151 bash[23756]: audit 2024-03-04T15:50:06.255736+0000 mon.a (mon.0) 724 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:08.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:08 smithi117 bash[21480]: cluster 2024-03-04T15:50:06.856779+0000 mgr.y (mgr.14152) 367 : cluster 0 pgmap v354: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 102 B/s rd, 102 B/s wr, 0 op/s 2024-03-04T15:50:08.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:08 smithi117 bash[28843]: cluster 2024-03-04T15:50:06.856779+0000 mgr.y (mgr.14152) 367 : cluster 0 pgmap v354: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 102 B/s rd, 102 B/s wr, 0 op/s 2024-03-04T15:50:08.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:08 smithi151 bash[23756]: cluster 2024-03-04T15:50:06.856779+0000 mgr.y (mgr.14152) 367 : cluster 0 pgmap v354: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 102 B/s rd, 102 B/s wr, 0 op/s 2024-03-04T15:50:10.065 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:09 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:10.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:10 smithi117 bash[21480]: cluster 2024-03-04T15:50:08.857869+0000 mgr.y (mgr.14152) 368 : cluster 0 pgmap v355: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 516 B/s rd, 86 B/s wr, 1 op/s 2024-03-04T15:50:10.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:10 smithi117 bash[21480]: audit 2024-03-04T15:50:09.066471+0000 mon.a (mon.0) 725 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:10.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:10 smithi117 bash[21480]: audit 2024-03-04T15:50:09.077326+0000 mon.a (mon.0) 726 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:10.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:10 smithi117 bash[21480]: audit 2024-03-04T15:50:09.079801+0000 mon.a (mon.0) 727 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:50:10.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:10 smithi117 bash[21480]: audit 2024-03-04T15:50:09.081743+0000 mon.a (mon.0) 728 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:50:10.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:10 smithi117 bash[21480]: audit 2024-03-04T15:50:09.091861+0000 mon.a (mon.0) 729 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:10.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:10 smithi117 bash[21480]: cephadm 2024-03-04T15:50:09.382727+0000 mgr.y (mgr.14152) 369 : cephadm 1 Deploying daemon prometheus.a on smithi151 2024-03-04T15:50:10.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:10 smithi117 bash[28843]: cluster 2024-03-04T15:50:08.857869+0000 mgr.y (mgr.14152) 368 : cluster 0 pgmap v355: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 516 B/s rd, 86 B/s wr, 1 op/s 2024-03-04T15:50:10.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:10 smithi117 bash[28843]: audit 2024-03-04T15:50:09.066471+0000 mon.a (mon.0) 725 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:10.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:10 smithi117 bash[28843]: audit 2024-03-04T15:50:09.077326+0000 mon.a (mon.0) 726 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:10.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:10 smithi117 bash[28843]: audit 2024-03-04T15:50:09.079801+0000 mon.a (mon.0) 727 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:50:10.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:10 smithi117 bash[28843]: audit 2024-03-04T15:50:09.081743+0000 mon.a (mon.0) 728 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:50:10.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:10 smithi117 bash[28843]: audit 2024-03-04T15:50:09.091861+0000 mon.a (mon.0) 729 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:10.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:10 smithi117 bash[28843]: cephadm 2024-03-04T15:50:09.382727+0000 mgr.y (mgr.14152) 369 : cephadm 1 Deploying daemon prometheus.a on smithi151 2024-03-04T15:50:10.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:10 smithi151 bash[23756]: cluster 2024-03-04T15:50:08.857869+0000 mgr.y (mgr.14152) 368 : cluster 0 pgmap v355: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 516 B/s rd, 86 B/s wr, 1 op/s 2024-03-04T15:50:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:10 smithi151 bash[23756]: audit 2024-03-04T15:50:09.066471+0000 mon.a (mon.0) 725 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:10 smithi151 bash[23756]: audit 2024-03-04T15:50:09.077326+0000 mon.a (mon.0) 726 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:10 smithi151 bash[23756]: audit 2024-03-04T15:50:09.079801+0000 mon.a (mon.0) 727 : audit 0 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:50:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:10 smithi151 bash[23756]: audit 2024-03-04T15:50:09.081743+0000 mon.a (mon.0) 728 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:50:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:10 smithi151 bash[23756]: audit 2024-03-04T15:50:09.091861+0000 mon.a (mon.0) 729 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:10 smithi151 bash[23756]: cephadm 2024-03-04T15:50:09.382727+0000 mgr.y (mgr.14152) 369 : cephadm 1 Deploying daemon prometheus.a on smithi151 2024-03-04T15:50:11.520 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T15:50:12.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:12 smithi117 bash[21480]: cluster 2024-03-04T15:50:10.858974+0000 mgr.y (mgr.14152) 370 : cluster 0 pgmap v356: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 85 B/s wr, 1 op/s 2024-03-04T15:50:12.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:12 smithi117 bash[28843]: cluster 2024-03-04T15:50:10.858974+0000 mgr.y (mgr.14152) 370 : cluster 0 pgmap v356: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 85 B/s wr, 1 op/s 2024-03-04T15:50:12.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:12 smithi151 bash[23756]: cluster 2024-03-04T15:50:10.858974+0000 mgr.y (mgr.14152) 370 : cluster 0 pgmap v356: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 85 B/s wr, 1 op/s 2024-03-04T15:50:13.078 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:50:12 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:50:13.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:13 smithi151 bash[23756]: audit 2024-03-04T15:50:12.772725+0000 mgr.y (mgr.14152) 371 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:50:13.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:13 smithi117 bash[21480]: audit 2024-03-04T15:50:12.772725+0000 mgr.y (mgr.14152) 371 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:50:13.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:13 smithi117 bash[28843]: audit 2024-03-04T15:50:12.772725+0000 mgr.y (mgr.14152) 371 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:50:14.419 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:14 smithi151 bash[23756]: cluster 2024-03-04T15:50:12.859815+0000 mgr.y (mgr.14152) 372 : cluster 0 pgmap v357: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 85 B/s wr, 1 op/s 2024-03-04T15:50:14.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:14 smithi117 bash[21480]: cluster 2024-03-04T15:50:12.859815+0000 mgr.y (mgr.14152) 372 : cluster 0 pgmap v357: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 85 B/s wr, 1 op/s 2024-03-04T15:50:14.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:14 smithi117 bash[28843]: cluster 2024-03-04T15:50:12.859815+0000 mgr.y (mgr.14152) 372 : cluster 0 pgmap v357: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 85 B/s wr, 1 op/s 2024-03-04T15:50:16.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:16 smithi151 bash[23756]: cluster 2024-03-04T15:50:14.860831+0000 mgr.y (mgr.14152) 373 : cluster 0 pgmap v358: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-03-04T15:50:16.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:16 smithi117 bash[21480]: cluster 2024-03-04T15:50:14.860831+0000 mgr.y (mgr.14152) 373 : cluster 0 pgmap v358: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-03-04T15:50:16.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:16 smithi117 bash[28843]: cluster 2024-03-04T15:50:14.860831+0000 mgr.y (mgr.14152) 373 : cluster 0 pgmap v358: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-03-04T15:50:18.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:18 smithi151 bash[23756]: cluster 2024-03-04T15:50:16.862009+0000 mgr.y (mgr.14152) 374 : cluster 0 pgmap v359: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:50:18.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:18 smithi117 bash[21480]: cluster 2024-03-04T15:50:16.862009+0000 mgr.y (mgr.14152) 374 : cluster 0 pgmap v359: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:50:18.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:18 smithi117 bash[28843]: cluster 2024-03-04T15:50:16.862009+0000 mgr.y (mgr.14152) 374 : cluster 0 pgmap v359: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:50:19.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d25a2700 1 -- 172.21.15.151:0/149736347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f65cc102fc0 msgr2=0x7f65cc103440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:19.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d25a2700 1 --2- 172.21.15.151:0/149736347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f65cc102fc0 0x7f65cc103440 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f65c000b0a0 tx=0x7f65c000b3b0 comp rx=0 tx=0).stop 2024-03-04T15:50:19.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d25a2700 1 -- 172.21.15.151:0/149736347 shutdown_connections 2024-03-04T15:50:19.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d25a2700 1 --2- 172.21.15.151:0/149736347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65cc101e60 0x7f65cc102280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:19.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d25a2700 1 --2- 172.21.15.151:0/149736347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65cc103980 0x7f65cc10a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:19.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d25a2700 1 --2- 172.21.15.151:0/149736347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f65cc102fc0 0x7f65cc103440 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:19.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d25a2700 1 -- 172.21.15.151:0/149736347 >> 172.21.15.151:0/149736347 conn(0x7f65cc0fd400 msgr2=0x7f65cc0ff840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:50:19.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d25a2700 1 -- 172.21.15.151:0/149736347 shutdown_connections 2024-03-04T15:50:19.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d25a2700 1 -- 172.21.15.151:0/149736347 wait complete. 2024-03-04T15:50:19.164 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d25a2700 1 Processor -- start 2024-03-04T15:50:19.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d25a2700 1 -- start start 2024-03-04T15:50:19.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d25a2700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65cc101e60 0x7f65cc197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:19.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d25a2700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f65cc102fc0 0x7f65cc197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:19.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d25a2700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65cc103980 0x7f65cc19ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:19.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d25a2700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc10cf50 con 0x7f65cc101e60 2024-03-04T15:50:19.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d25a2700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc10cdd0 con 0x7f65cc102fc0 2024-03-04T15:50:19.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d25a2700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65cc10d0d0 con 0x7f65cc103980 2024-03-04T15:50:19.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65cb7fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f65cc102fc0 0x7f65cc197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:50:19.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65cb7fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f65cc102fc0 0x7f65cc197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:50568/0 (socket says 172.21.15.151:50568) 2024-03-04T15:50:19.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65cb7fe700 1 -- 172.21.15.151:0/4168728957 learned_addr learned my addr 172.21.15.151:0/4168728957 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-04T15:50:19.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65cbfff700 1 --2- 172.21.15.151:0/4168728957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65cc101e60 0x7f65cc197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:50:19.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65cb7fe700 1 -- 172.21.15.151:0/4168728957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65cc103980 msgr2=0x7f65cc19ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:19.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d0b3f700 1 --2- 172.21.15.151:0/4168728957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65cc103980 0x7f65cc19ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:50:19.165 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65cb7fe700 1 --2- 172.21.15.151:0/4168728957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65cc103980 0x7f65cc19ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:19.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65cb7fe700 1 -- 172.21.15.151:0/4168728957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65cc101e60 msgr2=0x7f65cc197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:19.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65cb7fe700 1 --2- 172.21.15.151:0/4168728957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65cc101e60 0x7f65cc197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:19.166 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65cb7fe700 1 -- 172.21.15.151:0/4168728957 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65cc19f2b0 con 0x7f65cc102fc0 2024-03-04T15:50:19.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65cb7fe700 1 --2- 172.21.15.151:0/4168728957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f65cc102fc0 0x7f65cc197b90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009510 tx=0x7f65c000cf10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:50:19.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65c97fa700 1 -- 172.21.15.151:0/4168728957 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0021070 con 0x7f65cc102fc0 2024-03-04T15:50:19.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65c97fa700 1 -- 172.21.15.151:0/4168728957 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f65c001d8d0 con 0x7f65cc102fc0 2024-03-04T15:50:19.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d25a2700 1 -- 172.21.15.151:0/4168728957 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65cc19f540 con 0x7f65cc102fc0 2024-03-04T15:50:19.167 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65c97fa700 1 -- 172.21.15.151:0/4168728957 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c00054c0 con 0x7f65cc102fc0 2024-03-04T15:50:19.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d25a2700 1 -- 172.21.15.151:0/4168728957 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65cc19fac0 con 0x7f65cc102fc0 2024-03-04T15:50:19.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65d25a2700 1 -- 172.21.15.151:0/4168728957 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65cc04faa0 con 0x7f65cc102fc0 2024-03-04T15:50:19.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.160+0000 7f65c97fa700 1 -- 172.21.15.151:0/4168728957 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 16) v1 ==== 91448+0+0 (secure 0 0 0) 0x7f65c0024070 con 0x7f65cc102fc0 2024-03-04T15:50:19.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.164+0000 7f65c97fa700 1 --2- 172.21.15.151:0/4168728957 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f65b406d720 0x7f65b406fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:19.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.164+0000 7f65c97fa700 1 -- 172.21.15.151:0/4168728957 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 7148+0+0 (secure 0 0 0) 0x7f65c000f020 con 0x7f65cc102fc0 2024-03-04T15:50:19.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.164+0000 7f65cbfff700 1 --2- 172.21.15.151:0/4168728957 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f65b406d720 0x7f65b406fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:50:19.168 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.164+0000 7f65cbfff700 1 --2- 172.21.15.151:0/4168728957 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f65b406d720 0x7f65b406fbe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f65bc006fd0 tx=0x7f65bc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:50:19.171 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.168+0000 7f65c97fa700 1 -- 172.21.15.151:0/4168728957 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f65c002b020 con 0x7f65cc102fc0 2024-03-04T15:50:19.386 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.380+0000 7f65d25a2700 1 -- 172.21.15.151:0/4168728957 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi117=a;smithi151=b", "target": ["mon-mgr", ""]}) v1 -- 0x7f65cc062220 con 0x7f65b406d720 2024-03-04T15:50:19.399 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.396+0000 7f65c97fa700 1 -- 172.21.15.151:0/4168728957 <== mgr.14152 v2:172.21.15.117:6800/2865911351 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7f65cc062220 con 0x7f65b406d720 2024-03-04T15:50:19.400 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled node-exporter update... 2024-03-04T15:50:19.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.396+0000 7f65d25a2700 1 -- 172.21.15.151:0/4168728957 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f65b406d720 msgr2=0x7f65b406fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:19.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.396+0000 7f65d25a2700 1 --2- 172.21.15.151:0/4168728957 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f65b406d720 0x7f65b406fbe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f65bc006fd0 tx=0x7f65bc008040 comp rx=0 tx=0).stop 2024-03-04T15:50:19.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.396+0000 7f65d25a2700 1 -- 172.21.15.151:0/4168728957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f65cc102fc0 msgr2=0x7f65cc197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:19.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.396+0000 7f65d25a2700 1 --2- 172.21.15.151:0/4168728957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f65cc102fc0 0x7f65cc197b90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f65c0009510 tx=0x7f65c000cf10 comp rx=0 tx=0).stop 2024-03-04T15:50:19.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.396+0000 7f65d25a2700 1 -- 172.21.15.151:0/4168728957 shutdown_connections 2024-03-04T15:50:19.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.396+0000 7f65d25a2700 1 --2- 172.21.15.151:0/4168728957 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f65cc101e60 0x7f65cc197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:19.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.396+0000 7f65d25a2700 1 --2- 172.21.15.151:0/4168728957 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7f65b406d720 0x7f65b406fbe0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:19.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.396+0000 7f65d25a2700 1 --2- 172.21.15.151:0/4168728957 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f65cc103980 0x7f65cc19ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:19.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.396+0000 7f65d25a2700 1 --2- 172.21.15.151:0/4168728957 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f65cc102fc0 0x7f65cc197b90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:19.402 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.396+0000 7f65d25a2700 1 -- 172.21.15.151:0/4168728957 >> 172.21.15.151:0/4168728957 conn(0x7f65cc0fd400 msgr2=0x7f65cc0fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:50:19.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.396+0000 7f65d25a2700 1 -- 172.21.15.151:0/4168728957 shutdown_connections 2024-03-04T15:50:19.403 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:19.396+0000 7f65d25a2700 1 -- 172.21.15.151:0/4168728957 wait complete. 2024-03-04T15:50:20.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:20 smithi151 bash[23756]: cluster 2024-03-04T15:50:18.863023+0000 mgr.y (mgr.14152) 375 : cluster 0 pgmap v360: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:50:20.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:20 smithi151 bash[23756]: audit 2024-03-04T15:50:19.389404+0000 mgr.y (mgr.14152) 376 : audit 0 from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi117=a;smithi151=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:50:20.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:20 smithi151 bash[23756]: cephadm 2024-03-04T15:50:19.393111+0000 mgr.y (mgr.14152) 377 : cephadm 1 Saving service node-exporter spec with placement smithi117=a;smithi151=b;count:2 2024-03-04T15:50:20.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:20 smithi151 bash[23756]: audit 2024-03-04T15:50:19.400189+0000 mon.a (mon.0) 730 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:20.702 DEBUG:teuthology.orchestra.run.smithi117:node-exporter.a> sudo journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@node-exporter.a.service 2024-03-04T15:50:20.705 DEBUG:teuthology.orchestra.run.smithi151:node-exporter.b> sudo journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@node-exporter.b.service 2024-03-04T15:50:20.707 INFO:tasks.cephadm:Adding alertmanager.a on smithi117 2024-03-04T15:50:20.707 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph orch apply alertmanager '1;smithi117=a' 2024-03-04T15:50:20.713 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:20 smithi117 bash[21480]: cluster 2024-03-04T15:50:18.863023+0000 mgr.y (mgr.14152) 375 : cluster 0 pgmap v360: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:50:20.713 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:20 smithi117 bash[21480]: audit 2024-03-04T15:50:19.389404+0000 mgr.y (mgr.14152) 376 : audit 0 from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi117=a;smithi151=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:50:20.713 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:20 smithi117 bash[21480]: cephadm 2024-03-04T15:50:19.393111+0000 mgr.y (mgr.14152) 377 : cephadm 1 Saving service node-exporter spec with placement smithi117=a;smithi151=b;count:2 2024-03-04T15:50:20.713 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:20 smithi117 bash[21480]: audit 2024-03-04T15:50:19.400189+0000 mon.a (mon.0) 730 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:20.713 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:20 smithi117 bash[28843]: cluster 2024-03-04T15:50:18.863023+0000 mgr.y (mgr.14152) 375 : cluster 0 pgmap v360: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:50:20.713 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:20 smithi117 bash[28843]: audit 2024-03-04T15:50:19.389404+0000 mgr.y (mgr.14152) 376 : audit 0 from='client.24460 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi117=a;smithi151=b", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:50:20.714 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:20 smithi117 bash[28843]: cephadm 2024-03-04T15:50:19.393111+0000 mgr.y (mgr.14152) 377 : cephadm 1 Saving service node-exporter spec with placement smithi117=a;smithi151=b;count:2 2024-03-04T15:50:20.714 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:20 smithi117 bash[28843]: audit 2024-03-04T15:50:19.400189+0000 mon.a (mon.0) 730 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:22.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:22 smithi151 bash[23756]: cluster 2024-03-04T15:50:20.864132+0000 mgr.y (mgr.14152) 378 : cluster 0 pgmap v361: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:50:22.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:22 smithi117 bash[21480]: cluster 2024-03-04T15:50:20.864132+0000 mgr.y (mgr.14152) 378 : cluster 0 pgmap v361: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:50:22.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:22 smithi117 bash[28843]: cluster 2024-03-04T15:50:20.864132+0000 mgr.y (mgr.14152) 378 : cluster 0 pgmap v361: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:50:23.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:50:22 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:50:23.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:23 smithi151 bash[23756]: audit 2024-03-04T15:50:22.779296+0000 mon.a (mon.0) 731 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2024-03-04T15:50:23.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:23 smithi151 bash[23756]: audit 2024-03-04T15:50:22.779844+0000 mon.a (mon.0) 732 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-03-04T15:50:23.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:23 smithi151 bash[23756]: audit 2024-03-04T15:50:22.780376+0000 mon.a (mon.0) 733 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-03-04T15:50:23.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:23 smithi151 bash[23756]: audit 2024-03-04T15:50:22.780580+0000 mgr.y (mgr.14152) 379 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:50:23.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:23 smithi151 bash[23756]: audit 2024-03-04T15:50:22.780790+0000 mon.a (mon.0) 734 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-03-04T15:50:23.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:23 smithi117 bash[28843]: audit 2024-03-04T15:50:22.779296+0000 mon.a (mon.0) 731 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2024-03-04T15:50:23.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:23 smithi117 bash[28843]: audit 2024-03-04T15:50:22.779844+0000 mon.a (mon.0) 732 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-03-04T15:50:23.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:23 smithi117 bash[28843]: audit 2024-03-04T15:50:22.780376+0000 mon.a (mon.0) 733 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-03-04T15:50:23.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:23 smithi117 bash[28843]: audit 2024-03-04T15:50:22.780580+0000 mgr.y (mgr.14152) 379 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:50:23.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:23 smithi117 bash[28843]: audit 2024-03-04T15:50:22.780790+0000 mon.a (mon.0) 734 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-03-04T15:50:23.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:23 smithi117 bash[21480]: audit 2024-03-04T15:50:22.779296+0000 mon.a (mon.0) 731 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]: dispatch 2024-03-04T15:50:23.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:23 smithi117 bash[21480]: audit 2024-03-04T15:50:22.779844+0000 mon.a (mon.0) 732 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-03-04T15:50:23.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:23 smithi117 bash[21480]: audit 2024-03-04T15:50:22.780376+0000 mon.a (mon.0) 733 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-03-04T15:50:23.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:23 smithi117 bash[21480]: audit 2024-03-04T15:50:22.780580+0000 mgr.y (mgr.14152) 379 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:50:23.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:23 smithi117 bash[21480]: audit 2024-03-04T15:50:22.780790+0000 mon.a (mon.0) 734 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]: dispatch 2024-03-04T15:50:24.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:24 smithi151 bash[23756]: cluster 2024-03-04T15:50:22.865024+0000 mgr.y (mgr.14152) 380 : cluster 0 pgmap v362: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:50:24.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:24 smithi151 bash[23756]: audit 2024-03-04T15:50:23.410464+0000 mon.a (mon.0) 735 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]': finished 2024-03-04T15:50:24.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:24 smithi151 bash[23756]: audit 2024-03-04T15:50:23.410677+0000 mon.a (mon.0) 736 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-03-04T15:50:24.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:24 smithi151 bash[23756]: audit 2024-03-04T15:50:23.410839+0000 mon.a (mon.0) 737 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-03-04T15:50:24.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:24 smithi151 bash[23756]: audit 2024-03-04T15:50:23.410992+0000 mon.a (mon.0) 738 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-03-04T15:50:24.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:24 smithi151 bash[23756]: cluster 2024-03-04T15:50:23.420927+0000 mon.a (mon.0) 739 : cluster 0 osdmap e73: 8 total, 8 up, 8 in 2024-03-04T15:50:24.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:24 smithi117 bash[28843]: cluster 2024-03-04T15:50:22.865024+0000 mgr.y (mgr.14152) 380 : cluster 0 pgmap v362: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:50:24.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:24 smithi117 bash[28843]: audit 2024-03-04T15:50:23.410464+0000 mon.a (mon.0) 735 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]': finished 2024-03-04T15:50:24.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:24 smithi117 bash[28843]: audit 2024-03-04T15:50:23.410677+0000 mon.a (mon.0) 736 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-03-04T15:50:24.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:24 smithi117 bash[28843]: audit 2024-03-04T15:50:23.410839+0000 mon.a (mon.0) 737 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-03-04T15:50:24.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:24 smithi117 bash[28843]: audit 2024-03-04T15:50:23.410992+0000 mon.a (mon.0) 738 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-03-04T15:50:24.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:24 smithi117 bash[28843]: cluster 2024-03-04T15:50:23.420927+0000 mon.a (mon.0) 739 : cluster 0 osdmap e73: 8 total, 8 up, 8 in 2024-03-04T15:50:24.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:24 smithi117 bash[21480]: cluster 2024-03-04T15:50:22.865024+0000 mgr.y (mgr.14152) 380 : cluster 0 pgmap v362: 132 pgs: 132 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:50:24.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:24 smithi117 bash[21480]: audit 2024-03-04T15:50:23.410464+0000 mon.a (mon.0) 735 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 2]}]': finished 2024-03-04T15:50:24.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:24 smithi117 bash[21480]: audit 2024-03-04T15:50:23.410677+0000 mon.a (mon.0) 736 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-03-04T15:50:24.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:24 smithi117 bash[21480]: audit 2024-03-04T15:50:23.410839+0000 mon.a (mon.0) 737 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-03-04T15:50:24.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:24 smithi117 bash[21480]: audit 2024-03-04T15:50:23.410992+0000 mon.a (mon.0) 738 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1b", "id": [1, 2]}]': finished 2024-03-04T15:50:24.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:24 smithi117 bash[21480]: cluster 2024-03-04T15:50:23.420927+0000 mon.a (mon.0) 739 : cluster 0 osdmap e73: 8 total, 8 up, 8 in 2024-03-04T15:50:25.681 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:25 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:25.682 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 15:50:25 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:25.682 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 15:50:25 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:25.682 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 15:50:25 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:25.683 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 15:50:25 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:25.683 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:50:25 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:25.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:25 smithi151 bash[23756]: cluster 2024-03-04T15:50:24.428031+0000 mon.a (mon.0) 740 : cluster 0 osdmap e74: 8 total, 8 up, 8 in 2024-03-04T15:50:25.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:25 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:25.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:25 smithi117 bash[21480]: cluster 2024-03-04T15:50:24.428031+0000 mon.a (mon.0) 740 : cluster 0 osdmap e74: 8 total, 8 up, 8 in 2024-03-04T15:50:25.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:25 smithi117 bash[28843]: cluster 2024-03-04T15:50:24.428031+0000 mon.a (mon.0) 740 : cluster 0 osdmap e74: 8 total, 8 up, 8 in 2024-03-04T15:50:25.826 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T15:50:25.985 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:25 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:25.986 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 15:50:25 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:25.986 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 15:50:25 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:25.986 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 15:50:25 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:25.987 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:50:25 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:25.987 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:25 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:25.987 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:25 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:25.987 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:25 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:25.987 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:25 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:25.987 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:25 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:25.988 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 15:50:25 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:26.431 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:25 smithi151 systemd[1]: Started Ceph prometheus.a for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T15:50:26.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:26 smithi117 bash[21480]: cluster 2024-03-04T15:50:24.865716+0000 mgr.y (mgr.14152) 381 : cluster 0 pgmap v365: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:50:26.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:26 smithi117 bash[21480]: audit 2024-03-04T15:50:26.043796+0000 mon.a (mon.0) 741 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:26.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:26 smithi117 bash[21480]: audit 2024-03-04T15:50:26.056662+0000 mon.a (mon.0) 742 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:26.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:26 smithi117 bash[21480]: audit 2024-03-04T15:50:26.111536+0000 mon.a (mon.0) 743 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:26.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:26 smithi117 bash[21480]: audit 2024-03-04T15:50:26.120129+0000 mon.a (mon.0) 744 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-04T15:50:26.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:26 smithi117 bash[28843]: cluster 2024-03-04T15:50:24.865716+0000 mgr.y (mgr.14152) 381 : cluster 0 pgmap v365: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:50:26.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:26 smithi117 bash[28843]: audit 2024-03-04T15:50:26.043796+0000 mon.a (mon.0) 741 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:26.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:26 smithi117 bash[28843]: audit 2024-03-04T15:50:26.056662+0000 mon.a (mon.0) 742 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:26.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:26 smithi117 bash[28843]: audit 2024-03-04T15:50:26.111536+0000 mon.a (mon.0) 743 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:26.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:26 smithi117 bash[28843]: audit 2024-03-04T15:50:26.120129+0000 mon.a (mon.0) 744 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-04T15:50:26.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:26 smithi151 bash[23756]: cluster 2024-03-04T15:50:24.865716+0000 mgr.y (mgr.14152) 381 : cluster 0 pgmap v365: 132 pgs: 1 peering, 131 active+clean; 582 KiB data, 247 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:50:26.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:26 smithi151 bash[23756]: audit 2024-03-04T15:50:26.043796+0000 mon.a (mon.0) 741 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:26.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:26 smithi151 bash[23756]: audit 2024-03-04T15:50:26.056662+0000 mon.a (mon.0) 742 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:26.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:26 smithi151 bash[23756]: audit 2024-03-04T15:50:26.111536+0000 mon.a (mon.0) 743 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' 2024-03-04T15:50:26.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:26 smithi151 bash[23756]: audit 2024-03-04T15:50:26.120129+0000 mon.a (mon.0) 744 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-03-04T15:50:27.431 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[24709]: ignoring --setuser ceph since I am not root 2024-03-04T15:50:27.431 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[24709]: ignoring --setgroup ceph since I am not root 2024-03-04T15:50:27.431 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[24709]: debug 2024-03-04T15:50:27.208+0000 7f35f7fb8700 1 -- 172.21.15.151:0/2000669744 <== mon.1 v2:172.21.15.151:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556c50e3c5a0 con 0x556c51bbcc00 2024-03-04T15:50:27.431 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[24709]: debug 2024-03-04T15:50:27.360+0000 7f3601d9c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T15:50:27.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:27 smithi117 bash[21734]: ignoring --setuser ceph since I am not root 2024-03-04T15:50:27.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:27 smithi117 bash[21734]: ignoring --setgroup ceph since I am not root 2024-03-04T15:50:27.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:27 smithi117 bash[21734]: debug 2024-03-04T15:50:27.324+0000 7f0eae041200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-03-04T15:50:27.932 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.566Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-03-04T15:50:27.932 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.566Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-03-04T15:50:27.932 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.566Z caller=main.go:570 level=info host_details="(Linux 5.15.0-97-generic #107-Ubuntu SMP Wed Feb 7 13:26:48 UTC 2024 x86_64 smithi151 (none))" 2024-03-04T15:50:27.932 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.566Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-03-04T15:50:27.932 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.566Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-03-04T15:50:27.932 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.567Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-03-04T15:50:27.932 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.568Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-03-04T15:50:27.932 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.569Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-03-04T15:50:27.933 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.569Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-03-04T15:50:27.933 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.571Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-03-04T15:50:27.933 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.571Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.724µs 2024-03-04T15:50:27.933 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.571Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-03-04T15:50:27.933 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.573Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-03-04T15:50:27.933 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.573Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=21.551µs wal_replay_duration=2.616764ms wbl_replay_duration=119ns total_replay_duration=2.664457ms 2024-03-04T15:50:27.933 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.575Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-03-04T15:50:27.933 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.575Z caller=main.go:1029 level=info msg="TSDB started" 2024-03-04T15:50:27.933 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.575Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-03-04T15:50:27.933 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.576Z caller=refresh.go:80 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.117:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.117:8765: connect: connection refused" 2024-03-04T15:50:27.933 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.576Z caller=refresh.go:80 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.117:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.117:8765: connect: connection refused" 2024-03-04T15:50:27.933 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.576Z caller=refresh.go:80 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.117:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.117:8765: connect: connection refused" 2024-03-04T15:50:27.933 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.597Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=22.553136ms db_storage=832ns remote_storage=1.355µs web_handler=816ns query_engine=810ns scrape=275.856µs scrape_sd=96.274µs notify=802ns notify_sd=1.393µs rules=21.932341ms tracing=5.783µs 2024-03-04T15:50:27.933 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.597Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-03-04T15:50:27.934 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:27 smithi151 bash[49217]: ts=2024-03-04T15:50:27.597Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-03-04T15:50:28.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:28 smithi151 bash[23756]: audit 2024-03-04T15:50:27.112533+0000 mon.a (mon.0) 745 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-04T15:50:28.406 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:28 smithi151 bash[23756]: cluster 2024-03-04T15:50:27.124296+0000 mon.a (mon.0) 746 : cluster 0 mgrmap e17: y(active, since 10m), standbys: x 2024-03-04T15:50:28.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:28 smithi117 bash[21480]: audit 2024-03-04T15:50:27.112533+0000 mon.a (mon.0) 745 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-04T15:50:28.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:28 smithi117 bash[21480]: cluster 2024-03-04T15:50:27.124296+0000 mon.a (mon.0) 746 : cluster 0 mgrmap e17: y(active, since 10m), standbys: x 2024-03-04T15:50:28.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:28 smithi117 bash[28843]: audit 2024-03-04T15:50:27.112533+0000 mon.a (mon.0) 745 : audit 1 from='mgr.14152 172.21.15.117:0/2521650902' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-03-04T15:50:28.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:28 smithi117 bash[28843]: cluster 2024-03-04T15:50:27.124296+0000 mon.a (mon.0) 746 : cluster 0 mgrmap e17: y(active, since 10m), standbys: x 2024-03-04T15:50:29.229 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:28 smithi117 bash[21734]: debug 2024-03-04T15:50:28.880+0000 7f0eae041200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T15:50:29.230 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:28 smithi117 bash[21734]: debug 2024-03-04T15:50:28.976+0000 7f0eae041200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T15:50:29.331 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:28 smithi151 bash[24709]: debug 2024-03-04T15:50:28.976+0000 7f3601d9c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-03-04T15:50:29.331 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:29 smithi151 bash[24709]: debug 2024-03-04T15:50:29.072+0000 7f3601d9c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-03-04T15:50:29.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:29 smithi117 bash[21734]: debug 2024-03-04T15:50:29.224+0000 7f0eae041200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T15:50:29.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.600+0000 7feb3ae50700 1 -- 172.21.15.151:0/946314094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb34101f60 msgr2=0x7feb341023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:29.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.600+0000 7feb3ae50700 1 --2- 172.21.15.151:0/946314094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb34101f60 0x7feb341023c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7feb200099c0 tx=0x7feb20009cd0 comp rx=0 tx=0).stop 2024-03-04T15:50:29.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.600+0000 7feb3ae50700 1 -- 172.21.15.151:0/946314094 shutdown_connections 2024-03-04T15:50:29.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.600+0000 7feb3ae50700 1 --2- 172.21.15.151:0/946314094 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34100d60 0x7feb34101180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:29.607 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.600+0000 7feb3ae50700 1 --2- 172.21.15.151:0/946314094 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb34102900 0x7feb341091b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:29.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.600+0000 7feb3ae50700 1 --2- 172.21.15.151:0/946314094 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb34101f60 0x7feb341023c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:29.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.600+0000 7feb3ae50700 1 -- 172.21.15.151:0/946314094 >> 172.21.15.151:0/946314094 conn(0x7feb340fc300 msgr2=0x7feb340fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:50:29.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.600+0000 7feb3ae50700 1 -- 172.21.15.151:0/946314094 shutdown_connections 2024-03-04T15:50:29.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.600+0000 7feb3ae50700 1 -- 172.21.15.151:0/946314094 wait complete. 2024-03-04T15:50:29.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.600+0000 7feb3ae50700 1 Processor -- start 2024-03-04T15:50:29.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.600+0000 7feb3ae50700 1 -- start start 2024-03-04T15:50:29.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.600+0000 7feb3ae50700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34100d60 0x7feb3406af10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:29.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.600+0000 7feb3ae50700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb34101f60 0x7feb3419c380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:29.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.600+0000 7feb3ae50700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb34102900 0x7feb3419e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:29.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.600+0000 7feb3ae50700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3410bde0 con 0x7feb34100d60 2024-03-04T15:50:29.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.600+0000 7feb3ae50700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3410bc60 con 0x7feb34102900 2024-03-04T15:50:29.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.600+0000 7feb3ae50700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3410bf60 con 0x7feb34101f60 2024-03-04T15:50:29.608 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb393ed700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb34102900 0x7feb3419e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:50:29.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb393ed700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb34102900 0x7feb3419e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:59622/0 (socket says 172.21.15.151:59622) 2024-03-04T15:50:29.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb393ed700 1 -- 172.21.15.151:0/202127734 learned_addr learned my addr 172.21.15.151:0/202127734 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-04T15:50:29.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb33fff700 1 --2- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb34101f60 0x7feb3419c380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:50:29.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb393ed700 1 -- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb34101f60 msgr2=0x7feb3419c380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:29.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb38bec700 1 --2- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34100d60 0x7feb3406af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:50:29.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb393ed700 1 --2- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7feb34101f60 0x7feb3419c380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:29.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb393ed700 1 -- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34100d60 msgr2=0x7feb3406af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:29.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb393ed700 1 --2- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34100d60 0x7feb3406af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:29.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb393ed700 1 -- 172.21.15.151:0/202127734 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3419f290 con 0x7feb34102900 2024-03-04T15:50:29.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb38bec700 1 --2- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7feb34100d60 0x7feb3406af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T15:50:29.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb393ed700 1 --2- 172.21.15.151:0/202127734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb34102900 0x7feb3419e9c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feb2c007ee0 tx=0x7feb2c00d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:50:29.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb31ffb700 1 -- 172.21.15.151:0/202127734 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c00dbd0 con 0x7feb34102900 2024-03-04T15:50:29.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb31ffb700 1 -- 172.21.15.151:0/202127734 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7feb2c00dd70 con 0x7feb34102900 2024-03-04T15:50:29.609 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb31ffb700 1 -- 172.21.15.151:0/202127734 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb2c005890 con 0x7feb34102900 2024-03-04T15:50:29.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb3ae50700 1 -- 172.21.15.151:0/202127734 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb3419f580 con 0x7feb34102900 2024-03-04T15:50:29.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb3ae50700 1 -- 172.21.15.151:0/202127734 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3419fa70 con 0x7feb34102900 2024-03-04T15:50:29.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb3ae50700 1 -- 172.21.15.151:0/202127734 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb3404faa0 con 0x7feb34102900 2024-03-04T15:50:29.610 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb31ffb700 1 -- 172.21.15.151:0/202127734 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 17) v1 ==== 91462+0+0 (secure 0 0 0) 0x7feb2c01a020 con 0x7feb34102900 2024-03-04T15:50:29.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb31ffb700 1 --2- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7feb2406d770 0x7feb2406fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:29.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb38bec700 1 -- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7feb2406d770 msgr2=0x7feb2406fc30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2865911351 2024-03-04T15:50:29.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb31ffb700 1 -- 172.21.15.151:0/202127734 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 7236+0+0 (secure 0 0 0) 0x7feb2c08d780 con 0x7feb34102900 2024-03-04T15:50:29.611 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.604+0000 7feb38bec700 1 --2- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7feb2406d770 0x7feb2406fc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-03-04T15:50:29.613 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.608+0000 7feb31ffb700 1 -- 172.21.15.151:0/202127734 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7feb2c010070 con 0x7feb34102900 2024-03-04T15:50:29.681 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:29 smithi151 bash[24709]: debug 2024-03-04T15:50:29.324+0000 7f3601d9c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-03-04T15:50:29.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.808+0000 7feb38bec700 1 -- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7feb2406d770 msgr2=0x7feb2406fc30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2865911351 2024-03-04T15:50:29.811 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.808+0000 7feb38bec700 1 --2- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7feb2406d770 0x7feb2406fc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-03-04T15:50:29.836 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:29.832+0000 7feb3ae50700 1 -- 172.21.15.151:0/202127734 --> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi117=a", "target": ["mon-mgr", ""]}) v1 -- 0x7feb3402dc80 con 0x7feb2406d770 2024-03-04T15:50:30.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:30.212+0000 7feb38bec700 1 -- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7feb2406d770 msgr2=0x7feb2406fc30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2865911351 2024-03-04T15:50:30.215 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:30.212+0000 7feb38bec700 1 --2- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7feb2406d770 0x7feb2406fc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-03-04T15:50:30.320 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:30 smithi117 bash[21734]: debug 2024-03-04T15:50:30.016+0000 7f0eae041200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T15:50:30.320 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:30 smithi117 bash[21734]: debug 2024-03-04T15:50:30.128+0000 7f0eae041200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T15:50:30.320 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:30 smithi117 bash[21734]: debug 2024-03-04T15:50:30.232+0000 7f0eae041200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T15:50:30.431 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:30 smithi151 bash[24709]: debug 2024-03-04T15:50:30.132+0000 7f3601d9c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-03-04T15:50:30.431 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:30 smithi151 bash[24709]: debug 2024-03-04T15:50:30.244+0000 7f3601d9c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-03-04T15:50:30.431 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:30 smithi151 bash[24709]: debug 2024-03-04T15:50:30.344+0000 7f3601d9c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-03-04T15:50:31.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:31.016+0000 7feb38bec700 1 -- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7feb2406d770 msgr2=0x7feb2406fc30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2865911351 2024-03-04T15:50:31.019 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:31.016+0000 7feb38bec700 1 --2- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7feb2406d770 0x7feb2406fc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-03-04T15:50:31.070 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:30 smithi117 bash[21734]: debug 2024-03-04T15:50:30.588+0000 7f0eae041200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T15:50:31.070 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:30 smithi117 bash[21734]: debug 2024-03-04T15:50:30.700+0000 7f0eae041200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T15:50:31.070 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:30 smithi117 bash[21734]: debug 2024-03-04T15:50:30.804+0000 7f0eae041200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T15:50:31.181 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:30 smithi151 bash[24709]: debug 2024-03-04T15:50:30.700+0000 7f3601d9c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-03-04T15:50:31.182 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:30 smithi151 bash[24709]: debug 2024-03-04T15:50:30.812+0000 7f3601d9c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-03-04T15:50:31.182 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:30 smithi151 bash[24709]: debug 2024-03-04T15:50:30.916+0000 7f3601d9c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-03-04T15:50:31.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:31 smithi117 bash[21734]: debug 2024-03-04T15:50:31.160+0000 7f0eae041200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T15:50:31.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:31 smithi117 bash[21734]: debug 2024-03-04T15:50:31.400+0000 7f0eae041200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T15:50:31.681 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:31 smithi151 bash[24709]: debug 2024-03-04T15:50:31.268+0000 7f3601d9c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-03-04T15:50:31.681 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:31 smithi151 bash[24709]: debug 2024-03-04T15:50:31.508+0000 7f3601d9c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-03-04T15:50:32.050 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:31 smithi117 bash[21734]: debug 2024-03-04T15:50:31.660+0000 7f0eae041200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T15:50:32.050 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:31 smithi117 bash[21734]: debug 2024-03-04T15:50:31.776+0000 7f0eae041200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T15:50:32.160 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:31 smithi151 bash[24709]: debug 2024-03-04T15:50:31.772+0000 7f3601d9c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-03-04T15:50:32.160 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:31 smithi151 bash[24709]: debug 2024-03-04T15:50:31.884+0000 7f3601d9c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-03-04T15:50:32.320 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:32 smithi117 bash[21734]: debug 2024-03-04T15:50:32.048+0000 7f0eae041200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T15:50:32.320 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:32 smithi117 bash[21734]: debug 2024-03-04T15:50:32.292+0000 7f0eae041200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T15:50:32.431 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:32 smithi151 bash[24709]: debug 2024-03-04T15:50:32.156+0000 7f3601d9c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-03-04T15:50:32.431 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:32 smithi151 bash[24709]: debug 2024-03-04T15:50:32.396+0000 7f3601d9c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-03-04T15:50:32.622 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:32.620+0000 7feb38bec700 1 -- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7feb2406d770 msgr2=0x7feb2406fc30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.117:6800/2865911351 2024-03-04T15:50:32.623 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:32.620+0000 7feb38bec700 1 --2- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7feb2406d770 0x7feb2406fc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-03-04T15:50:33.070 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:32 smithi117 bash[21734]: debug 2024-03-04T15:50:32.700+0000 7f0eae041200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T15:50:33.070 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:32 smithi117 bash[21734]: debug 2024-03-04T15:50:32.820+0000 7f0eae041200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T15:50:33.173 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:32 smithi151 bash[24709]: debug 2024-03-04T15:50:32.808+0000 7f3601d9c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-03-04T15:50:33.173 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:32 smithi151 bash[24709]: debug 2024-03-04T15:50:32.924+0000 7f3601d9c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-03-04T15:50:33.431 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:33 smithi151 bash[24709]: debug 2024-03-04T15:50:33.168+0000 7f3601d9c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T15:50:33.569 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:33 smithi117 bash[21734]: debug 2024-03-04T15:50:33.068+0000 7f0eae041200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-03-04T15:50:34.489 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:34 smithi117 bash[21734]: debug 2024-03-04T15:50:34.076+0000 7f0eae041200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T15:50:34.489 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:34 smithi117 bash[21734]: debug 2024-03-04T15:50:34.192+0000 7f0eae041200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T15:50:34.588 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:34 smithi151 bash[24709]: debug 2024-03-04T15:50:34.180+0000 7f3601d9c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-03-04T15:50:34.588 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:34 smithi151 bash[24709]: debug 2024-03-04T15:50:34.292+0000 7f3601d9c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-03-04T15:50:34.819 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:34 smithi117 bash[21734]: debug 2024-03-04T15:50:34.488+0000 7f0eae041200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T15:50:34.820 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:34 smithi117 bash[21734]: debug 2024-03-04T15:50:34.600+0000 7f0eae041200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T15:50:34.931 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:34 smithi151 bash[24709]: debug 2024-03-04T15:50:34.584+0000 7f3601d9c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-03-04T15:50:34.931 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:34 smithi151 bash[24709]: debug 2024-03-04T15:50:34.696+0000 7f3601d9c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-03-04T15:50:35.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:35.212+0000 7feb31ffb700 1 -- 172.21.15.151:0/202127734 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mgrmap(e 18) v1 ==== 91078+0+0 (secure 0 0 0) 0x7feb2c056ff0 con 0x7feb34102900 2024-03-04T15:50:35.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:35.212+0000 7feb31ffb700 1 -- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7feb2406d770 msgr2=0x7feb2406fc30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-03-04T15:50:35.216 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:35.212+0000 7feb31ffb700 1 --2- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:6800/2865911351,v1:172.21.15.117:6801/2865911351] conn(0x7feb2406d770 0x7feb2406fc30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:35.319 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21734]: debug 2024-03-04T15:50:35.052+0000 7f0eae041200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T15:50:35.320 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21734]: debug 2024-03-04T15:50:35.172+0000 7f0eae041200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T15:50:35.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[28843]: cluster 2024-03-04T15:50:35.195719+0000 mon.a (mon.0) 747 : cluster 1 Active manager daemon y restarted 2024-03-04T15:50:35.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[28843]: cluster 2024-03-04T15:50:35.196431+0000 mon.a (mon.0) 748 : cluster 1 Activating manager daemon y 2024-03-04T15:50:35.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[28843]: cluster 2024-03-04T15:50:35.208389+0000 mon.a (mon.0) 749 : cluster 0 osdmap e75: 8 total, 8 up, 8 in 2024-03-04T15:50:35.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[28843]: cluster 2024-03-04T15:50:35.209994+0000 mon.a (mon.0) 750 : cluster 0 mgrmap e18: y(active, starting, since 0.0137219s), standbys: x 2024-03-04T15:50:35.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[28843]: audit 2024-03-04T15:50:35.222316+0000 mon.c (mon.2) 18 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:50:35.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[28843]: audit 2024-03-04T15:50:35.222785+0000 mon.c (mon.2) 19 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:50:35.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[28843]: audit 2024-03-04T15:50:35.223284+0000 mon.c (mon.2) 20 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:50:35.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[28843]: audit 2024-03-04T15:50:35.223992+0000 mon.c (mon.2) 21 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-04T15:50:35.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[28843]: audit 2024-03-04T15:50:35.224398+0000 mon.c (mon.2) 22 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-04T15:50:35.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[28843]: audit 2024-03-04T15:50:35.224942+0000 mon.c (mon.2) 23 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:50:35.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[28843]: audit 2024-03-04T15:50:35.225780+0000 mon.c (mon.2) 24 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T15:50:35.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[28843]: audit 2024-03-04T15:50:35.226546+0000 mon.c (mon.2) 25 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:50:35.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[28843]: audit 2024-03-04T15:50:35.227109+0000 mon.c (mon.2) 26 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:50:35.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[28843]: audit 2024-03-04T15:50:35.227736+0000 mon.c (mon.2) 27 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:50:35.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[28843]: audit 2024-03-04T15:50:35.228330+0000 mon.c (mon.2) 28 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T15:50:35.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[28843]: audit 2024-03-04T15:50:35.228928+0000 mon.c (mon.2) 29 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T15:50:35.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[28843]: audit 2024-03-04T15:50:35.229539+0000 mon.c (mon.2) 30 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T15:50:35.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[28843]: audit 2024-03-04T15:50:35.230215+0000 mon.c (mon.2) 31 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T15:50:35.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[28843]: audit 2024-03-04T15:50:35.230679+0000 mon.c (mon.2) 32 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T15:50:35.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[28843]: audit 2024-03-04T15:50:35.231974+0000 mon.c (mon.2) 33 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T15:50:35.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21480]: cluster 2024-03-04T15:50:35.195719+0000 mon.a (mon.0) 747 : cluster 1 Active manager daemon y restarted 2024-03-04T15:50:35.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21480]: cluster 2024-03-04T15:50:35.196431+0000 mon.a (mon.0) 748 : cluster 1 Activating manager daemon y 2024-03-04T15:50:35.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21480]: cluster 2024-03-04T15:50:35.208389+0000 mon.a (mon.0) 749 : cluster 0 osdmap e75: 8 total, 8 up, 8 in 2024-03-04T15:50:35.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21480]: cluster 2024-03-04T15:50:35.209994+0000 mon.a (mon.0) 750 : cluster 0 mgrmap e18: y(active, starting, since 0.0137219s), standbys: x 2024-03-04T15:50:35.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21480]: audit 2024-03-04T15:50:35.222316+0000 mon.c (mon.2) 18 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:50:35.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21480]: audit 2024-03-04T15:50:35.222785+0000 mon.c (mon.2) 19 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:50:35.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21480]: audit 2024-03-04T15:50:35.223284+0000 mon.c (mon.2) 20 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:50:35.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21480]: audit 2024-03-04T15:50:35.223992+0000 mon.c (mon.2) 21 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-04T15:50:35.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21480]: audit 2024-03-04T15:50:35.224398+0000 mon.c (mon.2) 22 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-04T15:50:35.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21480]: audit 2024-03-04T15:50:35.224942+0000 mon.c (mon.2) 23 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:50:35.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21480]: audit 2024-03-04T15:50:35.225780+0000 mon.c (mon.2) 24 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T15:50:35.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21480]: audit 2024-03-04T15:50:35.226546+0000 mon.c (mon.2) 25 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:50:35.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21480]: audit 2024-03-04T15:50:35.227109+0000 mon.c (mon.2) 26 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:50:35.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21480]: audit 2024-03-04T15:50:35.227736+0000 mon.c (mon.2) 27 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:50:35.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21480]: audit 2024-03-04T15:50:35.228330+0000 mon.c (mon.2) 28 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T15:50:35.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21480]: audit 2024-03-04T15:50:35.228928+0000 mon.c (mon.2) 29 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T15:50:35.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21480]: audit 2024-03-04T15:50:35.229539+0000 mon.c (mon.2) 30 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T15:50:35.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21480]: audit 2024-03-04T15:50:35.230215+0000 mon.c (mon.2) 31 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T15:50:35.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21480]: audit 2024-03-04T15:50:35.230679+0000 mon.c (mon.2) 32 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T15:50:35.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21480]: audit 2024-03-04T15:50:35.231974+0000 mon.c (mon.2) 33 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T15:50:35.403 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[24709]: debug 2024-03-04T15:50:35.144+0000 7f3601d9c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-03-04T15:50:35.403 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[24709]: debug 2024-03-04T15:50:35.268+0000 7f3601d9c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-03-04T15:50:35.403 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[24709]: [04/Mar/2024:15:50:35] ENGINE Bus STARTING 2024-03-04T15:50:35.403 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[24709]: CherryPy Checker: 2024-03-04T15:50:35.403 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[23756]: cluster 2024-03-04T15:50:35.195719+0000 mon.a (mon.0) 747 : cluster 1 Active manager daemon y restarted 2024-03-04T15:50:35.403 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[23756]: cluster 2024-03-04T15:50:35.196431+0000 mon.a (mon.0) 748 : cluster 1 Activating manager daemon y 2024-03-04T15:50:35.403 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[23756]: cluster 2024-03-04T15:50:35.208389+0000 mon.a (mon.0) 749 : cluster 0 osdmap e75: 8 total, 8 up, 8 in 2024-03-04T15:50:35.403 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[23756]: cluster 2024-03-04T15:50:35.209994+0000 mon.a (mon.0) 750 : cluster 0 mgrmap e18: y(active, starting, since 0.0137219s), standbys: x 2024-03-04T15:50:35.403 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[23756]: audit 2024-03-04T15:50:35.222316+0000 mon.c (mon.2) 18 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-03-04T15:50:35.403 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[23756]: audit 2024-03-04T15:50:35.222785+0000 mon.c (mon.2) 19 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-03-04T15:50:35.403 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[23756]: audit 2024-03-04T15:50:35.223284+0000 mon.c (mon.2) 20 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-03-04T15:50:35.404 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[23756]: audit 2024-03-04T15:50:35.223992+0000 mon.c (mon.2) 21 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-03-04T15:50:35.404 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[23756]: audit 2024-03-04T15:50:35.224398+0000 mon.c (mon.2) 22 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-03-04T15:50:35.404 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[23756]: audit 2024-03-04T15:50:35.224942+0000 mon.c (mon.2) 23 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-03-04T15:50:35.404 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[23756]: audit 2024-03-04T15:50:35.225780+0000 mon.c (mon.2) 24 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-03-04T15:50:35.404 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[23756]: audit 2024-03-04T15:50:35.226546+0000 mon.c (mon.2) 25 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-03-04T15:50:35.404 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[23756]: audit 2024-03-04T15:50:35.227109+0000 mon.c (mon.2) 26 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-03-04T15:50:35.404 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[23756]: audit 2024-03-04T15:50:35.227736+0000 mon.c (mon.2) 27 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-03-04T15:50:35.404 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[23756]: audit 2024-03-04T15:50:35.228330+0000 mon.c (mon.2) 28 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-03-04T15:50:35.404 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[23756]: audit 2024-03-04T15:50:35.228928+0000 mon.c (mon.2) 29 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-03-04T15:50:35.404 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[23756]: audit 2024-03-04T15:50:35.229539+0000 mon.c (mon.2) 30 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-03-04T15:50:35.404 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[23756]: audit 2024-03-04T15:50:35.230215+0000 mon.c (mon.2) 31 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-03-04T15:50:35.404 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[23756]: audit 2024-03-04T15:50:35.230679+0000 mon.c (mon.2) 32 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-03-04T15:50:35.404 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[23756]: audit 2024-03-04T15:50:35.231974+0000 mon.c (mon.2) 33 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-03-04T15:50:35.681 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[24709]: The Application mounted at '' has an empty config. 2024-03-04T15:50:35.681 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[24709]: [04/Mar/2024:15:50:35] ENGINE Serving on http://:::9283 2024-03-04T15:50:35.682 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:35 smithi151 bash[24709]: [04/Mar/2024:15:50:35] ENGINE Bus STARTED 2024-03-04T15:50:35.819 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21734]: [04/Mar/2024:15:50:35] ENGINE Bus STARTING 2024-03-04T15:50:35.820 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21734]: CherryPy Checker: 2024-03-04T15:50:35.820 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21734]: The Application mounted at '' has an empty config. 2024-03-04T15:50:35.820 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21734]: [04/Mar/2024:15:50:35] ENGINE Serving on http://:::9283 2024-03-04T15:50:35.820 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:35 smithi117 bash[21734]: [04/Mar/2024:15:50:35] ENGINE Bus STARTED 2024-03-04T15:50:36.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:36.268+0000 7feb31ffb700 1 -- 172.21.15.151:0/202127734 <== mon.1 v2:172.21.15.151:3300/0 8 ==== mgrmap(e 19) v1 ==== 91205+0+0 (secure 0 0 0) 0x7feb2c004a90 con 0x7feb34102900 2024-03-04T15:50:36.273 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:36.268+0000 7feb31ffb700 1 --2- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7feb24076ff0 0x7feb240793e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:36.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:36.268+0000 7feb31ffb700 1 -- 172.21.15.151:0/202127734 --> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi117=a", "target": ["mon-mgr", ""]}) v1 -- 0x7feb240292b0 con 0x7feb24076ff0 2024-03-04T15:50:36.274 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:36.272+0000 7feb38bec700 1 --2- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7feb24076ff0 0x7feb240793e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:50:36.276 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:36.272+0000 7feb38bec700 1 --2- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7feb24076ff0 0x7feb240793e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7feb34101dc0 tx=0x7feb280093d0 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:50:36.289 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:36.284+0000 7feb31ffb700 1 -- 172.21.15.151:0/202127734 <== mgr.24428 v2:172.21.15.117:6800/2727720514 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7feb240292b0 con 0x7feb24076ff0 2024-03-04T15:50:36.289 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled alertmanager update... 2024-03-04T15:50:36.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:36.288+0000 7feb3ae50700 1 -- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7feb24076ff0 msgr2=0x7feb240793e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:36.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:36.288+0000 7feb3ae50700 1 --2- 172.21.15.151:0/202127734 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7feb24076ff0 0x7feb240793e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7feb34101dc0 tx=0x7feb280093d0 comp rx=0 tx=0).stop 2024-03-04T15:50:36.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:36.288+0000 7feb3ae50700 1 -- 172.21.15.151:0/202127734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb34102900 msgr2=0x7feb3419e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:36.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:36.288+0000 7feb3ae50700 1 --2- 172.21.15.151:0/202127734 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7feb34102900 0x7feb3419e9c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7feb2c007ee0 tx=0x7feb2c00d3b0 comp rx=0 tx=0).stop 2024-03-04T15:50:36.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:36.288+0000 7feb393ed700 1 -- 172.21.15.151:0/202127734 reap_dead start 2024-03-04T15:50:36.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:36.288+0000 7feb3ae50700 1 -- 172.21.15.151:0/202127734 shutdown_connections 2024-03-04T15:50:36.292 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:36.288+0000 7feb3ae50700 1 -- 172.21.15.151:0/202127734 >> 172.21.15.151:0/202127734 conn(0x7feb340fc300 msgr2=0x7feb340fde00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:50:36.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:36.288+0000 7feb3ae50700 1 -- 172.21.15.151:0/202127734 shutdown_connections 2024-03-04T15:50:36.293 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:36.288+0000 7feb3ae50700 1 -- 172.21.15.151:0/202127734 wait complete. 2024-03-04T15:50:36.554 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:50:36.555 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: cluster 2024-03-04T15:50:35.259926+0000 mon.a (mon.0) 751 : cluster 1 Manager daemon y is now available 2024-03-04T15:50:36.555 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: cluster 2024-03-04T15:50:35.295258+0000 mon.a (mon.0) 752 : cluster 0 Standby manager daemon x restarted 2024-03-04T15:50:36.555 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: cluster 2024-03-04T15:50:35.295377+0000 mon.a (mon.0) 753 : cluster 0 Standby manager daemon x started 2024-03-04T15:50:36.555 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: audit 2024-03-04T15:50:35.298848+0000 mon.b (mon.1) 28 : audit 0 from='mgr.? 172.21.15.151:0/449502918' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-04T15:50:36.555 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: audit 2024-03-04T15:50:35.300643+0000 mon.b (mon.1) 29 : audit 0 from='mgr.? 172.21.15.151:0/449502918' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T15:50:36.555 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: audit 2024-03-04T15:50:35.302412+0000 mon.c (mon.2) 34 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:50:36.555 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: audit 2024-03-04T15:50:35.304030+0000 mon.b (mon.1) 30 : audit 0 from='mgr.? 172.21.15.151:0/449502918' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-04T15:50:36.555 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: audit 2024-03-04T15:50:35.304428+0000 mon.b (mon.1) 31 : audit 0 from='mgr.? 172.21.15.151:0/449502918' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T15:50:36.555 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: audit 2024-03-04T15:50:35.304924+0000 mon.c (mon.2) 35 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:50:36.555 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: audit 2024-03-04T15:50:35.323362+0000 mon.a (mon.0) 754 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:36.555 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: audit 2024-03-04T15:50:35.331619+0000 mon.c (mon.2) 36 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:50:36.556 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: audit 2024-03-04T15:50:35.338205+0000 mon.c (mon.2) 37 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-04T15:50:36.556 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: audit 2024-03-04T15:50:35.338635+0000 mon.a (mon.0) 755 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-04T15:50:36.556 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: audit 2024-03-04T15:50:35.361686+0000 mon.c (mon.2) 38 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-04T15:50:36.556 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: audit 2024-03-04T15:50:35.362291+0000 mon.a (mon.0) 756 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-04T15:50:36.556 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: cephadm 2024-03-04T15:50:35.857406+0000 mgr.y (mgr.24428) 1 : cephadm 1 [04/Mar/2024:15:50:35] ENGINE Bus STARTING 2024-03-04T15:50:36.556 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: cephadm 2024-03-04T15:50:35.959573+0000 mgr.y (mgr.24428) 2 : cephadm 1 [04/Mar/2024:15:50:35] ENGINE Serving on http://172.21.15.117:8765 2024-03-04T15:50:36.556 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: cephadm 2024-03-04T15:50:36.078171+0000 mgr.y (mgr.24428) 3 : cephadm 1 [04/Mar/2024:15:50:36] ENGINE Serving on https://172.21.15.117:7150 2024-03-04T15:50:36.556 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: cephadm 2024-03-04T15:50:36.078480+0000 mgr.y (mgr.24428) 4 : cephadm 1 [04/Mar/2024:15:50:36] ENGINE Bus STARTED 2024-03-04T15:50:36.556 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: audit 2024-03-04T15:50:36.149561+0000 mon.a (mon.0) 757 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:36.556 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:36 smithi151 bash[23756]: audit 2024-03-04T15:50:36.157168+0000 mon.a (mon.0) 758 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:36.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: cluster 2024-03-04T15:50:35.259926+0000 mon.a (mon.0) 751 : cluster 1 Manager daemon y is now available 2024-03-04T15:50:36.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: cluster 2024-03-04T15:50:35.295258+0000 mon.a (mon.0) 752 : cluster 0 Standby manager daemon x restarted 2024-03-04T15:50:36.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: cluster 2024-03-04T15:50:35.295377+0000 mon.a (mon.0) 753 : cluster 0 Standby manager daemon x started 2024-03-04T15:50:36.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: audit 2024-03-04T15:50:35.298848+0000 mon.b (mon.1) 28 : audit 0 from='mgr.? 172.21.15.151:0/449502918' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-04T15:50:36.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: audit 2024-03-04T15:50:35.300643+0000 mon.b (mon.1) 29 : audit 0 from='mgr.? 172.21.15.151:0/449502918' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T15:50:36.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: audit 2024-03-04T15:50:35.302412+0000 mon.c (mon.2) 34 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:50:36.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: audit 2024-03-04T15:50:35.304030+0000 mon.b (mon.1) 30 : audit 0 from='mgr.? 172.21.15.151:0/449502918' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-04T15:50:36.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: audit 2024-03-04T15:50:35.304428+0000 mon.b (mon.1) 31 : audit 0 from='mgr.? 172.21.15.151:0/449502918' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T15:50:36.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: audit 2024-03-04T15:50:35.304924+0000 mon.c (mon.2) 35 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:50:36.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: audit 2024-03-04T15:50:35.323362+0000 mon.a (mon.0) 754 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:36.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: audit 2024-03-04T15:50:35.331619+0000 mon.c (mon.2) 36 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:50:36.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: audit 2024-03-04T15:50:35.338205+0000 mon.c (mon.2) 37 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-04T15:50:36.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: audit 2024-03-04T15:50:35.338635+0000 mon.a (mon.0) 755 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-04T15:50:36.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: audit 2024-03-04T15:50:35.361686+0000 mon.c (mon.2) 38 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-04T15:50:36.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: audit 2024-03-04T15:50:35.362291+0000 mon.a (mon.0) 756 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-04T15:50:36.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: cephadm 2024-03-04T15:50:35.857406+0000 mgr.y (mgr.24428) 1 : cephadm 1 [04/Mar/2024:15:50:35] ENGINE Bus STARTING 2024-03-04T15:50:36.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: cephadm 2024-03-04T15:50:35.959573+0000 mgr.y (mgr.24428) 2 : cephadm 1 [04/Mar/2024:15:50:35] ENGINE Serving on http://172.21.15.117:8765 2024-03-04T15:50:36.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: cephadm 2024-03-04T15:50:36.078171+0000 mgr.y (mgr.24428) 3 : cephadm 1 [04/Mar/2024:15:50:36] ENGINE Serving on https://172.21.15.117:7150 2024-03-04T15:50:36.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: cephadm 2024-03-04T15:50:36.078480+0000 mgr.y (mgr.24428) 4 : cephadm 1 [04/Mar/2024:15:50:36] ENGINE Bus STARTED 2024-03-04T15:50:36.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: audit 2024-03-04T15:50:36.149561+0000 mon.a (mon.0) 757 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:36.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[28843]: audit 2024-03-04T15:50:36.157168+0000 mon.a (mon.0) 758 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:36.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: cluster 2024-03-04T15:50:35.259926+0000 mon.a (mon.0) 751 : cluster 1 Manager daemon y is now available 2024-03-04T15:50:36.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: cluster 2024-03-04T15:50:35.295258+0000 mon.a (mon.0) 752 : cluster 0 Standby manager daemon x restarted 2024-03-04T15:50:36.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: cluster 2024-03-04T15:50:35.295377+0000 mon.a (mon.0) 753 : cluster 0 Standby manager daemon x started 2024-03-04T15:50:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: audit 2024-03-04T15:50:35.298848+0000 mon.b (mon.1) 28 : audit 0 from='mgr.? 172.21.15.151:0/449502918' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-03-04T15:50:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: audit 2024-03-04T15:50:35.300643+0000 mon.b (mon.1) 29 : audit 0 from='mgr.? 172.21.15.151:0/449502918' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-03-04T15:50:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: audit 2024-03-04T15:50:35.302412+0000 mon.c (mon.2) 34 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:50:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: audit 2024-03-04T15:50:35.304030+0000 mon.b (mon.1) 30 : audit 0 from='mgr.? 172.21.15.151:0/449502918' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-03-04T15:50:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: audit 2024-03-04T15:50:35.304428+0000 mon.b (mon.1) 31 : audit 0 from='mgr.? 172.21.15.151:0/449502918' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-03-04T15:50:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: audit 2024-03-04T15:50:35.304924+0000 mon.c (mon.2) 35 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:50:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: audit 2024-03-04T15:50:35.323362+0000 mon.a (mon.0) 754 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: audit 2024-03-04T15:50:35.331619+0000 mon.c (mon.2) 36 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:50:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: audit 2024-03-04T15:50:35.338205+0000 mon.c (mon.2) 37 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-04T15:50:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: audit 2024-03-04T15:50:35.338635+0000 mon.a (mon.0) 755 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-03-04T15:50:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: audit 2024-03-04T15:50:35.361686+0000 mon.c (mon.2) 38 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-04T15:50:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: audit 2024-03-04T15:50:35.362291+0000 mon.a (mon.0) 756 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-03-04T15:50:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: cephadm 2024-03-04T15:50:35.857406+0000 mgr.y (mgr.24428) 1 : cephadm 1 [04/Mar/2024:15:50:35] ENGINE Bus STARTING 2024-03-04T15:50:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: cephadm 2024-03-04T15:50:35.959573+0000 mgr.y (mgr.24428) 2 : cephadm 1 [04/Mar/2024:15:50:35] ENGINE Serving on http://172.21.15.117:8765 2024-03-04T15:50:36.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: cephadm 2024-03-04T15:50:36.078171+0000 mgr.y (mgr.24428) 3 : cephadm 1 [04/Mar/2024:15:50:36] ENGINE Serving on https://172.21.15.117:7150 2024-03-04T15:50:36.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: cephadm 2024-03-04T15:50:36.078480+0000 mgr.y (mgr.24428) 4 : cephadm 1 [04/Mar/2024:15:50:36] ENGINE Bus STARTED 2024-03-04T15:50:36.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: audit 2024-03-04T15:50:36.149561+0000 mon.a (mon.0) 757 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:36.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:36 smithi117 bash[21480]: audit 2024-03-04T15:50:36.157168+0000 mon.a (mon.0) 758 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:37.239 DEBUG:teuthology.orchestra.run.smithi117:alertmanager.a> sudo journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@alertmanager.a.service 2024-03-04T15:50:37.242 INFO:tasks.cephadm:Adding grafana.a on smithi151 2024-03-04T15:50:37.242 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph orch apply grafana '1;smithi151=a' 2024-03-04T15:50:37.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:37 smithi117 bash[21480]: cluster 2024-03-04T15:50:36.268745+0000 mon.a (mon.0) 759 : cluster 0 mgrmap e19: y(active, since 1.0725s), standbys: x 2024-03-04T15:50:37.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:37 smithi117 bash[21480]: audit 2024-03-04T15:50:36.277814+0000 mgr.y (mgr.24428) 5 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:50:37.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:37 smithi117 bash[21480]: audit 2024-03-04T15:50:36.278865+0000 mgr.y (mgr.24428) 6 : audit 0 from='client.24475 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi117=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:50:37.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:37 smithi117 bash[21480]: cluster 2024-03-04T15:50:36.281666+0000 mgr.y (mgr.24428) 7 : cluster 0 pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:50:37.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:37 smithi117 bash[21480]: cephadm 2024-03-04T15:50:36.282579+0000 mgr.y (mgr.24428) 8 : cephadm 1 Saving service alertmanager spec with placement smithi117=a;count:1 2024-03-04T15:50:37.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:37 smithi117 bash[21480]: audit 2024-03-04T15:50:36.287018+0000 mon.a (mon.0) 760 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:37.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:37 smithi117 bash[28843]: cluster 2024-03-04T15:50:36.268745+0000 mon.a (mon.0) 759 : cluster 0 mgrmap e19: y(active, since 1.0725s), standbys: x 2024-03-04T15:50:37.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:37 smithi117 bash[28843]: audit 2024-03-04T15:50:36.277814+0000 mgr.y (mgr.24428) 5 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:50:37.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:37 smithi117 bash[28843]: audit 2024-03-04T15:50:36.278865+0000 mgr.y (mgr.24428) 6 : audit 0 from='client.24475 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi117=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:50:37.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:37 smithi117 bash[28843]: cluster 2024-03-04T15:50:36.281666+0000 mgr.y (mgr.24428) 7 : cluster 0 pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:50:37.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:37 smithi117 bash[28843]: cephadm 2024-03-04T15:50:36.282579+0000 mgr.y (mgr.24428) 8 : cephadm 1 Saving service alertmanager spec with placement smithi117=a;count:1 2024-03-04T15:50:37.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:37 smithi117 bash[28843]: audit 2024-03-04T15:50:36.287018+0000 mon.a (mon.0) 760 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:37.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:37 smithi151 bash[23756]: cluster 2024-03-04T15:50:36.268745+0000 mon.a (mon.0) 759 : cluster 0 mgrmap e19: y(active, since 1.0725s), standbys: x 2024-03-04T15:50:37.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:37 smithi151 bash[23756]: audit 2024-03-04T15:50:36.277814+0000 mgr.y (mgr.24428) 5 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:50:37.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:37 smithi151 bash[23756]: audit 2024-03-04T15:50:36.278865+0000 mgr.y (mgr.24428) 6 : audit 0 from='client.24475 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi117=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:50:37.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:37 smithi151 bash[23756]: cluster 2024-03-04T15:50:36.281666+0000 mgr.y (mgr.24428) 7 : cluster 0 pgmap v3: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:50:37.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:37 smithi151 bash[23756]: cephadm 2024-03-04T15:50:36.282579+0000 mgr.y (mgr.24428) 8 : cephadm 1 Saving service alertmanager spec with placement smithi117=a;count:1 2024-03-04T15:50:37.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:37 smithi151 bash[23756]: audit 2024-03-04T15:50:36.287018+0000 mon.a (mon.0) 760 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:38.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:38 smithi117 bash[28843]: cluster 2024-03-04T15:50:37.225660+0000 mgr.y (mgr.24428) 9 : cluster 0 pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:50:38.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:38 smithi117 bash[28843]: cluster 2024-03-04T15:50:37.295077+0000 mon.a (mon.0) 761 : cluster 0 mgrmap e20: y(active, since 2s), standbys: x 2024-03-04T15:50:38.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:38 smithi117 bash[21480]: cluster 2024-03-04T15:50:37.225660+0000 mgr.y (mgr.24428) 9 : cluster 0 pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:50:38.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:38 smithi117 bash[21480]: cluster 2024-03-04T15:50:37.295077+0000 mon.a (mon.0) 761 : cluster 0 mgrmap e20: y(active, since 2s), standbys: x 2024-03-04T15:50:38.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:38 smithi151 bash[23756]: cluster 2024-03-04T15:50:37.225660+0000 mgr.y (mgr.24428) 9 : cluster 0 pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:50:38.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:38 smithi151 bash[23756]: cluster 2024-03-04T15:50:37.295077+0000 mon.a (mon.0) 761 : cluster 0 mgrmap e20: y(active, since 2s), standbys: x 2024-03-04T15:50:40.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:40 smithi117 bash[21480]: cluster 2024-03-04T15:50:39.226424+0000 mgr.y (mgr.24428) 10 : cluster 0 pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:50:40.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:40 smithi117 bash[21480]: cluster 2024-03-04T15:50:39.311247+0000 mon.a (mon.0) 762 : cluster 0 mgrmap e21: y(active, since 4s), standbys: x 2024-03-04T15:50:40.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:40 smithi117 bash[28843]: cluster 2024-03-04T15:50:39.226424+0000 mgr.y (mgr.24428) 10 : cluster 0 pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:50:40.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:40 smithi117 bash[28843]: cluster 2024-03-04T15:50:39.311247+0000 mon.a (mon.0) 762 : cluster 0 mgrmap e21: y(active, since 4s), standbys: x 2024-03-04T15:50:40.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:40 smithi151 bash[23756]: cluster 2024-03-04T15:50:39.226424+0000 mgr.y (mgr.24428) 10 : cluster 0 pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:50:40.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:40 smithi151 bash[23756]: cluster 2024-03-04T15:50:39.311247+0000 mon.a (mon.0) 762 : cluster 0 mgrmap e21: y(active, since 4s), standbys: x 2024-03-04T15:50:41.777 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T15:50:42.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:42 smithi117 bash[28843]: cluster 2024-03-04T15:50:41.226898+0000 mgr.y (mgr.24428) 11 : cluster 0 pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:50:42.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:42 smithi117 bash[28843]: audit 2024-03-04T15:50:41.463000+0000 mon.a (mon.0) 763 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:42.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:42 smithi117 bash[28843]: audit 2024-03-04T15:50:41.476203+0000 mon.a (mon.0) 764 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:42.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:42 smithi117 bash[28843]: audit 2024-03-04T15:50:41.544242+0000 mon.a (mon.0) 765 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:42.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:42 smithi117 bash[28843]: audit 2024-03-04T15:50:41.554191+0000 mon.a (mon.0) 766 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:42.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:42 smithi117 bash[21480]: cluster 2024-03-04T15:50:41.226898+0000 mgr.y (mgr.24428) 11 : cluster 0 pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:50:42.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:42 smithi117 bash[21480]: audit 2024-03-04T15:50:41.463000+0000 mon.a (mon.0) 763 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:42.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:42 smithi117 bash[21480]: audit 2024-03-04T15:50:41.476203+0000 mon.a (mon.0) 764 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:42.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:42 smithi117 bash[21480]: audit 2024-03-04T15:50:41.544242+0000 mon.a (mon.0) 765 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:42.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:42 smithi117 bash[21480]: audit 2024-03-04T15:50:41.554191+0000 mon.a (mon.0) 766 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:42.822 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:42 smithi151 bash[23756]: cluster 2024-03-04T15:50:41.226898+0000 mgr.y (mgr.24428) 11 : cluster 0 pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:50:42.822 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:42 smithi151 bash[23756]: audit 2024-03-04T15:50:41.463000+0000 mon.a (mon.0) 763 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:42.822 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:42 smithi151 bash[23756]: audit 2024-03-04T15:50:41.476203+0000 mon.a (mon.0) 764 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:42.822 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:42 smithi151 bash[23756]: audit 2024-03-04T15:50:41.544242+0000 mon.a (mon.0) 765 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:42.822 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:42 smithi151 bash[23756]: audit 2024-03-04T15:50:41.554191+0000 mon.a (mon.0) 766 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.526703+0000 mon.a (mon.0) 767 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.535064+0000 mon.a (mon.0) 768 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.540040+0000 mon.c (mon.2) 39 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.540427+0000 mon.a (mon.0) 769 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.541783+0000 mon.c (mon.2) 40 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.542118+0000 mon.a (mon.0) 770 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.543149+0000 mon.c (mon.2) 41 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.543440+0000 mon.a (mon.0) 771 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.544318+0000 mon.c (mon.2) 42 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.544596+0000 mon.a (mon.0) 772 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: cephadm 2024-03-04T15:50:42.545332+0000 mgr.y (mgr.24428) 12 : cephadm 1 Adjusting osd_memory_target on smithi151 to 3804M 2024-03-04T15:50:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.551537+0000 mon.a (mon.0) 773 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.561739+0000 mon.a (mon.0) 774 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.573245+0000 mon.a (mon.0) 775 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.577180+0000 mon.c (mon.2) 43 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.577527+0000 mon.a (mon.0) 776 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.578425+0000 mon.c (mon.2) 44 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.578710+0000 mon.a (mon.0) 777 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.579573+0000 mon.c (mon.2) 45 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.579887+0000 mon.a (mon.0) 778 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.580740+0000 mon.c (mon.2) 46 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.581026+0000 mon.a (mon.0) 779 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: cephadm 2024-03-04T15:50:42.581776+0000 mgr.y (mgr.24428) 13 : cephadm 1 Adjusting osd_memory_target on smithi117 to 3804M 2024-03-04T15:50:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.587619+0000 mon.a (mon.0) 780 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.592670+0000 mon.c (mon.2) 47 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:50:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:42.593472+0000 mon.c (mon.2) 48 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:50:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: cephadm 2024-03-04T15:50:42.594830+0000 mgr.y (mgr.24428) 14 : cephadm 1 Updating smithi117:/etc/ceph/ceph.conf 2024-03-04T15:50:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: cephadm 2024-03-04T15:50:42.595297+0000 mgr.y (mgr.24428) 15 : cephadm 1 Updating smithi151:/etc/ceph/ceph.conf 2024-03-04T15:50:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: cephadm 2024-03-04T15:50:42.860845+0000 mgr.y (mgr.24428) 16 : cephadm 1 Updating smithi117:/var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T15:50:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: cephadm 2024-03-04T15:50:42.861159+0000 mgr.y (mgr.24428) 17 : cephadm 1 Updating smithi151:/var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T15:50:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: cephadm 2024-03-04T15:50:43.025547+0000 mgr.y (mgr.24428) 18 : cephadm 1 Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-03-04T15:50:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: cephadm 2024-03-04T15:50:43.025782+0000 mgr.y (mgr.24428) 19 : cephadm 1 Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-03-04T15:50:43.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:43.363098+0000 mon.a (mon.0) 781 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:43.369135+0000 mon.a (mon.0) 782 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:43.379952+0000 mon.a (mon.0) 783 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:43.385969+0000 mon.a (mon.0) 784 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:43 smithi151 bash[23756]: audit 2024-03-04T15:50:43.395764+0000 mon.a (mon.0) 785 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.814 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.526703+0000 mon.a (mon.0) 767 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.814 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.535064+0000 mon.a (mon.0) 768 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.814 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.540040+0000 mon.c (mon.2) 39 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.814 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.540427+0000 mon.a (mon.0) 769 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.814 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.541783+0000 mon.c (mon.2) 40 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.814 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.542118+0000 mon.a (mon.0) 770 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.814 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.543149+0000 mon.c (mon.2) 41 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.814 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.543440+0000 mon.a (mon.0) 771 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.814 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.544318+0000 mon.c (mon.2) 42 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.814 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.544596+0000 mon.a (mon.0) 772 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: cephadm 2024-03-04T15:50:42.545332+0000 mgr.y (mgr.24428) 12 : cephadm 1 Adjusting osd_memory_target on smithi151 to 3804M 2024-03-04T15:50:43.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.551537+0000 mon.a (mon.0) 773 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.561739+0000 mon.a (mon.0) 774 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.573245+0000 mon.a (mon.0) 775 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.577180+0000 mon.c (mon.2) 43 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.577527+0000 mon.a (mon.0) 776 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.578425+0000 mon.c (mon.2) 44 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.578710+0000 mon.a (mon.0) 777 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.579573+0000 mon.c (mon.2) 45 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.815 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.526703+0000 mon.a (mon.0) 767 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.815 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.535064+0000 mon.a (mon.0) 768 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.540040+0000 mon.c (mon.2) 39 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.540427+0000 mon.a (mon.0) 769 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.541783+0000 mon.c (mon.2) 40 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.542118+0000 mon.a (mon.0) 770 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.543149+0000 mon.c (mon.2) 41 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.543440+0000 mon.a (mon.0) 771 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.544318+0000 mon.c (mon.2) 42 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.544596+0000 mon.a (mon.0) 772 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: cephadm 2024-03-04T15:50:42.545332+0000 mgr.y (mgr.24428) 12 : cephadm 1 Adjusting osd_memory_target on smithi151 to 3804M 2024-03-04T15:50:43.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.551537+0000 mon.a (mon.0) 773 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.561739+0000 mon.a (mon.0) 774 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.573245+0000 mon.a (mon.0) 775 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.577180+0000 mon.c (mon.2) 43 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.577527+0000 mon.a (mon.0) 776 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.578425+0000 mon.c (mon.2) 44 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.578710+0000 mon.a (mon.0) 777 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.579573+0000 mon.c (mon.2) 45 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.579887+0000 mon.a (mon.0) 778 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.580740+0000 mon.c (mon.2) 46 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.581026+0000 mon.a (mon.0) 779 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: cephadm 2024-03-04T15:50:42.581776+0000 mgr.y (mgr.24428) 13 : cephadm 1 Adjusting osd_memory_target on smithi117 to 3804M 2024-03-04T15:50:43.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.587619+0000 mon.a (mon.0) 780 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.592670+0000 mon.c (mon.2) 47 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:50:43.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:42.593472+0000 mon.c (mon.2) 48 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:50:43.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: cephadm 2024-03-04T15:50:42.594830+0000 mgr.y (mgr.24428) 14 : cephadm 1 Updating smithi117:/etc/ceph/ceph.conf 2024-03-04T15:50:43.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: cephadm 2024-03-04T15:50:42.595297+0000 mgr.y (mgr.24428) 15 : cephadm 1 Updating smithi151:/etc/ceph/ceph.conf 2024-03-04T15:50:43.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: cephadm 2024-03-04T15:50:42.860845+0000 mgr.y (mgr.24428) 16 : cephadm 1 Updating smithi117:/var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T15:50:43.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: cephadm 2024-03-04T15:50:42.861159+0000 mgr.y (mgr.24428) 17 : cephadm 1 Updating smithi151:/var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T15:50:43.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: cephadm 2024-03-04T15:50:43.025547+0000 mgr.y (mgr.24428) 18 : cephadm 1 Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-03-04T15:50:43.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: cephadm 2024-03-04T15:50:43.025782+0000 mgr.y (mgr.24428) 19 : cephadm 1 Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-03-04T15:50:43.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:43.363098+0000 mon.a (mon.0) 781 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:43.369135+0000 mon.a (mon.0) 782 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:43.379952+0000 mon.a (mon.0) 783 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:43.385969+0000 mon.a (mon.0) 784 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[21480]: audit 2024-03-04T15:50:43.395764+0000 mon.a (mon.0) 785 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.579887+0000 mon.a (mon.0) 778 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.580740+0000 mon.c (mon.2) 46 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.581026+0000 mon.a (mon.0) 779 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T15:50:43.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: cephadm 2024-03-04T15:50:42.581776+0000 mgr.y (mgr.24428) 13 : cephadm 1 Adjusting osd_memory_target on smithi117 to 3804M 2024-03-04T15:50:43.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.587619+0000 mon.a (mon.0) 780 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.592670+0000 mon.c (mon.2) 47 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:50:43.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:42.593472+0000 mon.c (mon.2) 48 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:50:43.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: cephadm 2024-03-04T15:50:42.594830+0000 mgr.y (mgr.24428) 14 : cephadm 1 Updating smithi117:/etc/ceph/ceph.conf 2024-03-04T15:50:43.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: cephadm 2024-03-04T15:50:42.595297+0000 mgr.y (mgr.24428) 15 : cephadm 1 Updating smithi151:/etc/ceph/ceph.conf 2024-03-04T15:50:43.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: cephadm 2024-03-04T15:50:42.860845+0000 mgr.y (mgr.24428) 16 : cephadm 1 Updating smithi117:/var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T15:50:43.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: cephadm 2024-03-04T15:50:42.861159+0000 mgr.y (mgr.24428) 17 : cephadm 1 Updating smithi151:/var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/config/ceph.conf 2024-03-04T15:50:43.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: cephadm 2024-03-04T15:50:43.025547+0000 mgr.y (mgr.24428) 18 : cephadm 1 Updating smithi117:/etc/ceph/ceph.client.admin.keyring 2024-03-04T15:50:43.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: cephadm 2024-03-04T15:50:43.025782+0000 mgr.y (mgr.24428) 19 : cephadm 1 Updating smithi151:/etc/ceph/ceph.client.admin.keyring 2024-03-04T15:50:43.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:43.363098+0000 mon.a (mon.0) 781 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:43.369135+0000 mon.a (mon.0) 782 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:43.379952+0000 mon.a (mon.0) 783 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:43.385969+0000 mon.a (mon.0) 784 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:43.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:43 smithi117 bash[28843]: audit 2024-03-04T15:50:43.395764+0000 mon.a (mon.0) 785 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:44.426 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:44 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:44.426 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:44 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:44.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:44 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:44.427 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:44 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:44.427 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:44 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:44.427 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:44 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:44.427 INFO:journalctl@ceph.osd.1.smithi117.stdout:Mar 04 15:50:44 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:44.428 INFO:journalctl@ceph.osd.1.smithi117.stdout:Mar 04 15:50:44 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:44.428 INFO:journalctl@ceph.osd.0.smithi117.stdout:Mar 04 15:50:44 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:44.428 INFO:journalctl@ceph.osd.0.smithi117.stdout:Mar 04 15:50:44 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:44.428 INFO:journalctl@ceph.rgw.foo.a.smithi117.stdout:Mar 04 15:50:44 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:44.428 INFO:journalctl@ceph.rgw.foo.a.smithi117.stdout:Mar 04 15:50:44 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:44.429 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 04 15:50:44 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:44.429 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 04 15:50:44 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:44.429 INFO:journalctl@ceph.osd.2.smithi117.stdout:Mar 04 15:50:44 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:44.429 INFO:journalctl@ceph.osd.2.smithi117.stdout:Mar 04 15:50:44 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:44.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:44 smithi117 bash[28843]: cephadm 2024-03-04T15:50:43.190636+0000 mgr.y (mgr.24428) 20 : cephadm 1 Updating smithi117:/var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T15:50:44.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:44 smithi117 bash[28843]: cephadm 2024-03-04T15:50:43.191711+0000 mgr.y (mgr.24428) 21 : cephadm 1 Updating smithi151:/var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T15:50:44.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:44 smithi117 bash[28843]: cluster 2024-03-04T15:50:43.228133+0000 mgr.y (mgr.24428) 22 : cluster 0 pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-04T15:50:44.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:44 smithi117 bash[28843]: cephadm 2024-03-04T15:50:43.403657+0000 mgr.y (mgr.24428) 23 : cephadm 1 Deploying daemon node-exporter.a on smithi117 2024-03-04T15:50:44.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:44 smithi117 bash[28843]: audit 2024-03-04T15:50:44.468283+0000 mon.a (mon.0) 786 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:44.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:44 smithi117 bash[28843]: audit 2024-03-04T15:50:44.479115+0000 mon.a (mon.0) 787 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:44.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:44 smithi117 bash[28843]: audit 2024-03-04T15:50:44.529856+0000 mon.a (mon.0) 788 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:44.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:44 smithi117 bash[21480]: cephadm 2024-03-04T15:50:43.190636+0000 mgr.y (mgr.24428) 20 : cephadm 1 Updating smithi117:/var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T15:50:44.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:44 smithi117 bash[21480]: cephadm 2024-03-04T15:50:43.191711+0000 mgr.y (mgr.24428) 21 : cephadm 1 Updating smithi151:/var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T15:50:44.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:44 smithi117 bash[21480]: cluster 2024-03-04T15:50:43.228133+0000 mgr.y (mgr.24428) 22 : cluster 0 pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-04T15:50:44.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:44 smithi117 bash[21480]: cephadm 2024-03-04T15:50:43.403657+0000 mgr.y (mgr.24428) 23 : cephadm 1 Deploying daemon node-exporter.a on smithi117 2024-03-04T15:50:44.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:44 smithi117 bash[21480]: audit 2024-03-04T15:50:44.468283+0000 mon.a (mon.0) 786 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:44.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:44 smithi117 bash[21480]: audit 2024-03-04T15:50:44.479115+0000 mon.a (mon.0) 787 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:44.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:44 smithi117 bash[21480]: audit 2024-03-04T15:50:44.529856+0000 mon.a (mon.0) 788 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:44.822 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:44 smithi117 systemd[1]: Started Ceph node-exporter.a for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T15:50:44.822 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:44 smithi117 bash[57413]: Unable to find image 'quay.io/prometheus/node-exporter:v1.5.0' locally 2024-03-04T15:50:44.915 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:44 smithi151 bash[23756]: cephadm 2024-03-04T15:50:43.190636+0000 mgr.y (mgr.24428) 20 : cephadm 1 Updating smithi117:/var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T15:50:44.915 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:44 smithi151 bash[23756]: cephadm 2024-03-04T15:50:43.191711+0000 mgr.y (mgr.24428) 21 : cephadm 1 Updating smithi151:/var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/config/ceph.client.admin.keyring 2024-03-04T15:50:44.915 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:44 smithi151 bash[23756]: cluster 2024-03-04T15:50:43.228133+0000 mgr.y (mgr.24428) 22 : cluster 0 pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 30 KiB/s rd, 0 B/s wr, 17 op/s 2024-03-04T15:50:44.915 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:44 smithi151 bash[23756]: cephadm 2024-03-04T15:50:43.403657+0000 mgr.y (mgr.24428) 23 : cephadm 1 Deploying daemon node-exporter.a on smithi117 2024-03-04T15:50:44.915 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:44 smithi151 bash[23756]: audit 2024-03-04T15:50:44.468283+0000 mon.a (mon.0) 786 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:44.915 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:44 smithi151 bash[23756]: audit 2024-03-04T15:50:44.479115+0000 mon.a (mon.0) 787 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:44.915 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:44 smithi151 bash[23756]: audit 2024-03-04T15:50:44.529856+0000 mon.a (mon.0) 788 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:45.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe631b63700 1 -- 172.21.15.151:0/2753520021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe62c101700 msgr2=0x7fe62c10db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:45.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe631b63700 1 --2- 172.21.15.151:0/2753520021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe62c101700 0x7fe62c10db70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe62400b0a0 tx=0x7fe62400b3b0 comp rx=0 tx=0).stop 2024-03-04T15:50:45.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe631b63700 1 -- 172.21.15.151:0/2753520021 shutdown_connections 2024-03-04T15:50:45.091 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe631b63700 1 --2- 172.21.15.151:0/2753520021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62c106dc0 0x7fe62c1071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:45.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe631b63700 1 --2- 172.21.15.151:0/2753520021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe62c101700 0x7fe62c10db70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:45.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe631b63700 1 --2- 172.21.15.151:0/2753520021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62c100d40 0x7fe62c1011c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:45.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe631b63700 1 -- 172.21.15.151:0/2753520021 >> 172.21.15.151:0/2753520021 conn(0x7fe62c076d80 msgr2=0x7fe62c077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:50:45.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe631b63700 1 -- 172.21.15.151:0/2753520021 shutdown_connections 2024-03-04T15:50:45.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe631b63700 1 -- 172.21.15.151:0/2753520021 wait complete. 2024-03-04T15:50:45.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe631b63700 1 Processor -- start 2024-03-04T15:50:45.092 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe631b63700 1 -- start start 2024-03-04T15:50:45.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe631b63700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe62c100d40 0x7fe62c19f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:45.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe631b63700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62c101700 0x7fe62c19fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:45.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe631b63700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62c106dc0 0x7fe62c199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:45.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe631b63700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c110520 con 0x7fe62c101700 2024-03-04T15:50:45.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe631b63700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c1103a0 con 0x7fe62c100d40 2024-03-04T15:50:45.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe631b63700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe62c1106a0 con 0x7fe62c106dc0 2024-03-04T15:50:45.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe62b7fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe62c100d40 0x7fe62c19f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:50:45.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe62b7fe700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe62c100d40 0x7fe62c19f850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:51158/0 (socket says 172.21.15.151:51158) 2024-03-04T15:50:45.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe62b7fe700 1 -- 172.21.15.151:0/2800533051 learned_addr learned my addr 172.21.15.151:0/2800533051 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-04T15:50:45.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe62bfff700 1 --2- 172.21.15.151:0/2800533051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62c106dc0 0x7fe62c199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:50:45.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe62affd700 1 --2- 172.21.15.151:0/2800533051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62c101700 0x7fe62c19fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:50:45.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe62b7fe700 1 -- 172.21.15.151:0/2800533051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62c106dc0 msgr2=0x7fe62c199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:45.094 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe62b7fe700 1 --2- 172.21.15.151:0/2800533051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62c106dc0 0x7fe62c199a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:45.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe62b7fe700 1 -- 172.21.15.151:0/2800533051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62c101700 msgr2=0x7fe62c19fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:45.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe62b7fe700 1 --2- 172.21.15.151:0/2800533051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62c101700 0x7fe62c19fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:45.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe62b7fe700 1 -- 172.21.15.151:0/2800533051 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe62c19a170 con 0x7fe62c100d40 2024-03-04T15:50:45.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe62affd700 1 --2- 172.21.15.151:0/2800533051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62c101700 0x7fe62c19fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T15:50:45.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe62bfff700 1 --2- 172.21.15.151:0/2800533051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62c106dc0 0x7fe62c199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T15:50:45.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe62b7fe700 1 --2- 172.21.15.151:0/2800533051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe62c100d40 0x7fe62c19f850 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00d950 tx=0x7fe61c00dd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:50:45.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe628ff9700 1 -- 172.21.15.151:0/2800533051 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c016070 con 0x7fe62c100d40 2024-03-04T15:50:45.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe628ff9700 1 -- 172.21.15.151:0/2800533051 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fe61c010460 con 0x7fe62c100d40 2024-03-04T15:50:45.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe631b63700 1 -- 172.21.15.151:0/2800533051 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe62c19a460 con 0x7fe62c100d40 2024-03-04T15:50:45.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe628ff9700 1 -- 172.21.15.151:0/2800533051 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe61c00f620 con 0x7fe62c100d40 2024-03-04T15:50:45.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe631b63700 1 -- 172.21.15.151:0/2800533051 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe62c1a6720 con 0x7fe62c100d40 2024-03-04T15:50:45.095 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.088+0000 7fe631b63700 1 -- 172.21.15.151:0/2800533051 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe62c04faa0 con 0x7fe62c100d40 2024-03-04T15:50:45.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.092+0000 7fe628ff9700 1 -- 172.21.15.151:0/2800533051 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7fe61c019070 con 0x7fe62c100d40 2024-03-04T15:50:45.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.092+0000 7fe628ff9700 1 --2- 172.21.15.151:0/2800533051 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fe61806d780 0x7fe61806fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:45.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.092+0000 7fe628ff9700 1 -- 172.21.15.151:0/2800533051 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7fe61c08d640 con 0x7fe62c100d40 2024-03-04T15:50:45.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.092+0000 7fe62affd700 1 --2- 172.21.15.151:0/2800533051 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fe61806d780 0x7fe61806fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:50:45.096 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.092+0000 7fe62affd700 1 --2- 172.21.15.151:0/2800533051 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fe61806d780 0x7fe61806fc40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe614005fd0 tx=0x7fe614005ee0 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:50:45.100 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.096+0000 7fe628ff9700 1 -- 172.21.15.151:0/2800533051 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fe61c05ae90 con 0x7fe62c100d40 2024-03-04T15:50:45.335 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.332+0000 7fe631b63700 1 -- 172.21.15.151:0/2800533051 --> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi151=a", "target": ["mon-mgr", ""]}) v1 -- 0x7fe62c06a220 con 0x7fe61806d780 2024-03-04T15:50:45.350 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.348+0000 7fe628ff9700 1 -- 172.21.15.151:0/2800533051 <== mgr.24428 v2:172.21.15.117:6800/2727720514 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7fe62c06a220 con 0x7fe61806d780 2024-03-04T15:50:45.351 INFO:teuthology.orchestra.run.smithi151.stdout:Scheduled grafana update... 2024-03-04T15:50:45.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.348+0000 7fe631b63700 1 -- 172.21.15.151:0/2800533051 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fe61806d780 msgr2=0x7fe61806fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:45.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.348+0000 7fe631b63700 1 --2- 172.21.15.151:0/2800533051 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fe61806d780 0x7fe61806fc40 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe614005fd0 tx=0x7fe614005ee0 comp rx=0 tx=0).stop 2024-03-04T15:50:45.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.348+0000 7fe631b63700 1 -- 172.21.15.151:0/2800533051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe62c100d40 msgr2=0x7fe62c19f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:45.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.348+0000 7fe631b63700 1 --2- 172.21.15.151:0/2800533051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe62c100d40 0x7fe62c19f850 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fe61c00d950 tx=0x7fe61c00dd10 comp rx=0 tx=0).stop 2024-03-04T15:50:45.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.348+0000 7fe631b63700 1 -- 172.21.15.151:0/2800533051 shutdown_connections 2024-03-04T15:50:45.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.348+0000 7fe631b63700 1 --2- 172.21.15.151:0/2800533051 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fe62c106dc0 0x7fe62c199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:45.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.348+0000 7fe631b63700 1 --2- 172.21.15.151:0/2800533051 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fe62c100d40 0x7fe62c19f850 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:45.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.348+0000 7fe631b63700 1 --2- 172.21.15.151:0/2800533051 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fe61806d780 0x7fe61806fc40 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:45.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.348+0000 7fe631b63700 1 --2- 172.21.15.151:0/2800533051 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fe62c101700 0x7fe62c19fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:45.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.348+0000 7fe631b63700 1 -- 172.21.15.151:0/2800533051 >> 172.21.15.151:0/2800533051 conn(0x7fe62c076d80 msgr2=0x7fe62c069b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:50:45.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.348+0000 7fe631b63700 1 -- 172.21.15.151:0/2800533051 shutdown_connections 2024-03-04T15:50:45.353 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:50:45.348+0000 7fe631b63700 1 -- 172.21.15.151:0/2800533051 wait complete. 2024-03-04T15:50:45.502 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:45 smithi117 bash[57413]: v1.5.0: Pulling from prometheus/node-exporter 2024-03-04T15:50:45.502 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:45 smithi117 bash[57413]: 22b70bddd3ac: Pulling fs layer 2024-03-04T15:50:45.502 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:45 smithi117 bash[57413]: 5c12815fee55: Pulling fs layer 2024-03-04T15:50:45.502 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:45 smithi117 bash[57413]: c0e87333d380: Pulling fs layer 2024-03-04T15:50:45.550 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:45 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:45.550 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:50:45 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:45.551 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 15:50:45 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:45.551 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 15:50:45 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:45.551 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 15:50:45 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:45.551 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 15:50:45 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:45.551 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 15:50:45 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:45.551 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 15:50:45 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:45.552 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:50:45 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:45.552 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:50:45 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:45.552 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:45 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:45.552 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:50:45 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:45.553 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 15:50:45 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:45.553 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 15:50:45 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:45.553 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:45 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:45.553 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:45 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:45.803 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:45 smithi117 bash[28843]: cephadm 2024-03-04T15:50:44.531345+0000 mgr.y (mgr.24428) 24 : cephadm 1 Deploying daemon node-exporter.b on smithi151 2024-03-04T15:50:45.803 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:45 smithi117 bash[28843]: audit 2024-03-04T15:50:45.349337+0000 mon.a (mon.0) 789 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:45.804 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:45 smithi117 bash[21480]: cephadm 2024-03-04T15:50:44.531345+0000 mgr.y (mgr.24428) 24 : cephadm 1 Deploying daemon node-exporter.b on smithi151 2024-03-04T15:50:45.804 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:45 smithi117 bash[21480]: audit 2024-03-04T15:50:45.349337+0000 mon.a (mon.0) 789 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:45.804 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:45 smithi117 bash[57413]: 5c12815fee55: Verifying Checksum 2024-03-04T15:50:45.804 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:45 smithi117 bash[57413]: 5c12815fee55: Download complete 2024-03-04T15:50:45.804 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:45 smithi117 bash[57413]: 22b70bddd3ac: Verifying Checksum 2024-03-04T15:50:45.804 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:45 smithi117 bash[57413]: 22b70bddd3ac: Download complete 2024-03-04T15:50:45.804 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:45 smithi117 bash[57413]: 22b70bddd3ac: Pull complete 2024-03-04T15:50:45.835 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:45 smithi151 bash[23756]: cephadm 2024-03-04T15:50:44.531345+0000 mgr.y (mgr.24428) 24 : cephadm 1 Deploying daemon node-exporter.b on smithi151 2024-03-04T15:50:45.835 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:45 smithi151 bash[23756]: audit 2024-03-04T15:50:45.349337+0000 mon.a (mon.0) 789 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:45.836 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:45 smithi151 systemd[1]: Started Ceph node-exporter.b for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T15:50:45.836 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:45 smithi151 bash[50213]: Unable to find image 'quay.io/prometheus/node-exporter:v1.5.0' locally 2024-03-04T15:50:46.070 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:45 smithi117 bash[57413]: c0e87333d380: Download complete 2024-03-04T15:50:46.367 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[57413]: 5c12815fee55: Pull complete 2024-03-04T15:50:46.368 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:50:46 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:46.477 DEBUG:teuthology.orchestra.run.smithi151:grafana.a> sudo journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@grafana.a.service 2024-03-04T15:50:46.480 INFO:tasks.cephadm:Setting up client nodes... 2024-03-04T15:50:46.481 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-04T15:50:46.597 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:50:46 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:50:46.820 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[57413]: c0e87333d380: Pull complete 2024-03-04T15:50:46.820 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[57413]: Digest: sha256:39c642b2b337e38c18e80266fb14383754178202f40103646337722a594d984c 2024-03-04T15:50:46.820 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[57413]: Status: Downloaded newer image for quay.io/prometheus/node-exporter:v1.5.0 2024-03-04T15:50:46.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[21480]: cluster 2024-03-04T15:50:45.228799+0000 mgr.y (mgr.24428) 25 : cluster 0 pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-04T15:50:46.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[21480]: audit 2024-03-04T15:50:45.339648+0000 mgr.y (mgr.24428) 26 : audit 0 from='client.24502 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi151=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:50:46.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[21480]: cephadm 2024-03-04T15:50:45.342714+0000 mgr.y (mgr.24428) 27 : cephadm 1 Saving service grafana spec with placement smithi151=a;count:1 2024-03-04T15:50:46.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[21480]: audit 2024-03-04T15:50:45.605434+0000 mon.a (mon.0) 790 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:46.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[21480]: audit 2024-03-04T15:50:45.613094+0000 mon.a (mon.0) 791 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:46.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[21480]: audit 2024-03-04T15:50:45.659556+0000 mon.a (mon.0) 792 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:46.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[21480]: audit 2024-03-04T15:50:45.668132+0000 mon.a (mon.0) 793 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:46.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[21480]: audit 2024-03-04T15:50:45.679180+0000 mon.a (mon.0) 794 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:46.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[21480]: cephadm 2024-03-04T15:50:45.702787+0000 mgr.y (mgr.24428) 28 : cephadm 1 Deploying daemon alertmanager.a on smithi117 2024-03-04T15:50:46.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[28843]: cluster 2024-03-04T15:50:45.228799+0000 mgr.y (mgr.24428) 25 : cluster 0 pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-04T15:50:46.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[28843]: audit 2024-03-04T15:50:45.339648+0000 mgr.y (mgr.24428) 26 : audit 0 from='client.24502 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi151=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:50:46.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[28843]: cephadm 2024-03-04T15:50:45.342714+0000 mgr.y (mgr.24428) 27 : cephadm 1 Saving service grafana spec with placement smithi151=a;count:1 2024-03-04T15:50:46.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[28843]: audit 2024-03-04T15:50:45.605434+0000 mon.a (mon.0) 790 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:46.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[28843]: audit 2024-03-04T15:50:45.613094+0000 mon.a (mon.0) 791 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:46.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[28843]: audit 2024-03-04T15:50:45.659556+0000 mon.a (mon.0) 792 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:46.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[28843]: audit 2024-03-04T15:50:45.668132+0000 mon.a (mon.0) 793 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:46.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[28843]: audit 2024-03-04T15:50:45.679180+0000 mon.a (mon.0) 794 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:46.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:46 smithi117 bash[28843]: cephadm 2024-03-04T15:50:45.702787+0000 mgr.y (mgr.24428) 28 : cephadm 1 Deploying daemon alertmanager.a on smithi117 2024-03-04T15:50:46.897 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:46 smithi151 bash[50213]: v1.5.0: Pulling from prometheus/node-exporter 2024-03-04T15:50:46.898 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:46 smithi151 bash[23756]: cluster 2024-03-04T15:50:45.228799+0000 mgr.y (mgr.24428) 25 : cluster 0 pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 24 KiB/s rd, 0 B/s wr, 13 op/s 2024-03-04T15:50:46.898 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:46 smithi151 bash[23756]: audit 2024-03-04T15:50:45.339648+0000 mgr.y (mgr.24428) 26 : audit 0 from='client.24502 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi151=a", "target": ["mon-mgr", ""]}]: dispatch 2024-03-04T15:50:46.898 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:46 smithi151 bash[23756]: cephadm 2024-03-04T15:50:45.342714+0000 mgr.y (mgr.24428) 27 : cephadm 1 Saving service grafana spec with placement smithi151=a;count:1 2024-03-04T15:50:46.898 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:46 smithi151 bash[23756]: audit 2024-03-04T15:50:45.605434+0000 mon.a (mon.0) 790 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:46.898 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:46 smithi151 bash[23756]: audit 2024-03-04T15:50:45.613094+0000 mon.a (mon.0) 791 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:46.898 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:46 smithi151 bash[23756]: audit 2024-03-04T15:50:45.659556+0000 mon.a (mon.0) 792 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:46.898 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:46 smithi151 bash[23756]: audit 2024-03-04T15:50:45.668132+0000 mon.a (mon.0) 793 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:46.898 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:46 smithi151 bash[23756]: audit 2024-03-04T15:50:45.679180+0000 mon.a (mon.0) 794 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:46.898 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:46 smithi151 bash[23756]: cephadm 2024-03-04T15:50:45.702787+0000 mgr.y (mgr.24428) 28 : cephadm 1 Deploying daemon alertmanager.a on smithi117 2024-03-04T15:50:47.181 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:46 smithi151 bash[50213]: 22b70bddd3ac: Pulling fs layer 2024-03-04T15:50:47.181 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:46 smithi151 bash[50213]: 5c12815fee55: Pulling fs layer 2024-03-04T15:50:47.182 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:46 smithi151 bash[50213]: c0e87333d380: Pulling fs layer 2024-03-04T15:50:47.182 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:47 smithi151 bash[50213]: 5c12815fee55: Verifying Checksum 2024-03-04T15:50:47.182 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:47 smithi151 bash[50213]: 5c12815fee55: Download complete 2024-03-04T15:50:47.182 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:47 smithi151 bash[50213]: 22b70bddd3ac: Verifying Checksum 2024-03-04T15:50:47.182 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:47 smithi151 bash[50213]: 22b70bddd3ac: Download complete 2024-03-04T15:50:47.614 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:47 smithi151 bash[50213]: c0e87333d380: Verifying Checksum 2024-03-04T15:50:47.614 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:47 smithi151 bash[50213]: c0e87333d380: Download complete 2024-03-04T15:50:47.614 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:47 smithi151 bash[50213]: 22b70bddd3ac: Pull complete 2024-03-04T15:50:47.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:47 smithi151 bash[23756]: audit 2024-03-04T15:50:46.288764+0000 mgr.y (mgr.24428) 29 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:50:47.932 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:47 smithi151 bash[50213]: 5c12815fee55: Pull complete 2024-03-04T15:50:48.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:47 smithi117 bash[21480]: audit 2024-03-04T15:50:46.288764+0000 mgr.y (mgr.24428) 29 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:50:48.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:47 smithi117 bash[28843]: audit 2024-03-04T15:50:46.288764+0000 mgr.y (mgr.24428) 29 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:50:48.431 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:48 smithi151 bash[50213]: c0e87333d380: Pull complete 2024-03-04T15:50:48.432 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:48 smithi151 bash[50213]: Digest: sha256:39c642b2b337e38c18e80266fb14383754178202f40103646337722a594d984c 2024-03-04T15:50:48.432 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:48 smithi151 bash[50213]: Status: Downloaded newer image for quay.io/prometheus/node-exporter:v1.5.0 2024-03-04T15:50:48.570 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.307Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-03-04T15:50:48.570 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.308Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-03-04T15:50:48.570 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.308Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-03-04T15:50:48.571 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.308Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-03-04T15:50:48.571 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.309Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-03-04T15:50:48.571 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.309Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-03-04T15:50:48.571 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.309Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-03-04T15:50:48.571 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.309Z caller=node_exporter.go:117 level=info collector=arp 2024-03-04T15:50:48.571 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.309Z caller=node_exporter.go:117 level=info collector=bcache 2024-03-04T15:50:48.571 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.309Z caller=node_exporter.go:117 level=info collector=bonding 2024-03-04T15:50:48.571 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.309Z caller=node_exporter.go:117 level=info collector=btrfs 2024-03-04T15:50:48.571 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.309Z caller=node_exporter.go:117 level=info collector=conntrack 2024-03-04T15:50:48.571 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.309Z caller=node_exporter.go:117 level=info collector=cpu 2024-03-04T15:50:48.571 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-03-04T15:50:48.571 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=diskstats 2024-03-04T15:50:48.571 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=dmi 2024-03-04T15:50:48.571 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=edac 2024-03-04T15:50:48.571 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=entropy 2024-03-04T15:50:48.572 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-03-04T15:50:48.572 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=filefd 2024-03-04T15:50:48.572 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=filesystem 2024-03-04T15:50:48.572 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=hwmon 2024-03-04T15:50:48.572 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=infiniband 2024-03-04T15:50:48.572 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=ipvs 2024-03-04T15:50:48.572 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=loadavg 2024-03-04T15:50:48.572 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=mdadm 2024-03-04T15:50:48.572 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=meminfo 2024-03-04T15:50:48.572 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=netclass 2024-03-04T15:50:48.572 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=netdev 2024-03-04T15:50:48.572 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=netstat 2024-03-04T15:50:48.572 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=nfs 2024-03-04T15:50:48.572 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=nfsd 2024-03-04T15:50:48.572 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=nvme 2024-03-04T15:50:48.573 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=os 2024-03-04T15:50:48.573 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-03-04T15:50:48.573 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=pressure 2024-03-04T15:50:48.573 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=rapl 2024-03-04T15:50:48.573 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=schedstat 2024-03-04T15:50:48.573 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=selinux 2024-03-04T15:50:48.573 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=sockstat 2024-03-04T15:50:48.573 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=softnet 2024-03-04T15:50:48.573 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=stat 2024-03-04T15:50:48.573 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=tapestats 2024-03-04T15:50:48.573 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=textfile 2024-03-04T15:50:48.573 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-03-04T15:50:48.573 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=time 2024-03-04T15:50:48.573 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-03-04T15:50:48.573 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=uname 2024-03-04T15:50:48.574 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=vmstat 2024-03-04T15:50:48.574 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=xfs 2024-03-04T15:50:48.574 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=node_exporter.go:117 level=info collector=zfs 2024-03-04T15:50:48.574 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-03-04T15:50:48.574 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[57413]: ts=2024-03-04T15:50:48.310Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-03-04T15:50:49.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[21480]: cluster 2024-03-04T15:50:47.229995+0000 mgr.y (mgr.24428) 30 : cluster 0 pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-04T15:50:49.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:48 smithi117 bash[28843]: cluster 2024-03-04T15:50:47.229995+0000 mgr.y (mgr.24428) 30 : cluster 0 pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-04T15:50:49.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:48 smithi151 bash[23756]: cluster 2024-03-04T15:50:47.229995+0000 mgr.y (mgr.24428) 30 : cluster 0 pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 11 op/s 2024-03-04T15:50:50.182 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.004Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-03-04T15:50:50.182 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.004Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-03-04T15:50:50.182 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.005Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-03-04T15:50:50.183 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.005Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-03-04T15:50:50.183 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-03-04T15:50:50.183 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-03-04T15:50:50.183 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-03-04T15:50:50.183 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=arp 2024-03-04T15:50:50.183 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=bcache 2024-03-04T15:50:50.183 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=bonding 2024-03-04T15:50:50.183 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=btrfs 2024-03-04T15:50:50.183 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=conntrack 2024-03-04T15:50:50.183 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=cpu 2024-03-04T15:50:50.183 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-03-04T15:50:50.183 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=diskstats 2024-03-04T15:50:50.183 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=dmi 2024-03-04T15:50:50.183 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=edac 2024-03-04T15:50:50.183 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=entropy 2024-03-04T15:50:50.184 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-03-04T15:50:50.184 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=filefd 2024-03-04T15:50:50.184 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=filesystem 2024-03-04T15:50:50.184 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=hwmon 2024-03-04T15:50:50.184 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=infiniband 2024-03-04T15:50:50.184 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=ipvs 2024-03-04T15:50:50.184 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=loadavg 2024-03-04T15:50:50.184 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=mdadm 2024-03-04T15:50:50.184 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=meminfo 2024-03-04T15:50:50.185 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=netclass 2024-03-04T15:50:50.185 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=netdev 2024-03-04T15:50:50.185 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=netstat 2024-03-04T15:50:50.185 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=nfs 2024-03-04T15:50:50.185 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=nfsd 2024-03-04T15:50:50.185 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=nvme 2024-03-04T15:50:50.185 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=os 2024-03-04T15:50:50.185 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-03-04T15:50:50.185 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=pressure 2024-03-04T15:50:50.185 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=rapl 2024-03-04T15:50:50.185 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=schedstat 2024-03-04T15:50:50.185 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=selinux 2024-03-04T15:50:50.185 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=sockstat 2024-03-04T15:50:50.185 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=softnet 2024-03-04T15:50:50.185 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=stat 2024-03-04T15:50:50.185 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=tapestats 2024-03-04T15:50:50.186 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=textfile 2024-03-04T15:50:50.186 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-03-04T15:50:50.186 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=time 2024-03-04T15:50:50.186 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-03-04T15:50:50.186 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=uname 2024-03-04T15:50:50.186 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=vmstat 2024-03-04T15:50:50.186 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=xfs 2024-03-04T15:50:50.186 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.006Z caller=node_exporter.go:117 level=info collector=zfs 2024-03-04T15:50:50.186 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.007Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-03-04T15:50:50.186 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[50213]: ts=2024-03-04T15:50:50.007Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-03-04T15:50:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[23756]: cluster 2024-03-04T15:50:49.230692+0000 mgr.y (mgr.24428) 31 : cluster 0 pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-04T15:50:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[23756]: audit 2024-03-04T15:50:50.338902+0000 mon.a (mon.0) 795 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:50 smithi151 bash[23756]: audit 2024-03-04T15:50:50.359759+0000 mon.c (mon.2) 49 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:50:51.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:50 smithi117 bash[28843]: cluster 2024-03-04T15:50:49.230692+0000 mgr.y (mgr.24428) 31 : cluster 0 pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-04T15:50:51.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:50 smithi117 bash[28843]: audit 2024-03-04T15:50:50.338902+0000 mon.a (mon.0) 795 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:51.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:50 smithi117 bash[28843]: audit 2024-03-04T15:50:50.359759+0000 mon.c (mon.2) 49 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:50:51.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:50 smithi117 bash[21480]: cluster 2024-03-04T15:50:49.230692+0000 mgr.y (mgr.24428) 31 : cluster 0 pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-04T15:50:51.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:50 smithi117 bash[21480]: audit 2024-03-04T15:50:50.338902+0000 mon.a (mon.0) 795 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:51.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:50 smithi117 bash[21480]: audit 2024-03-04T15:50:50.359759+0000 mon.c (mon.2) 49 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:50:52.251 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:50:52.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:52 smithi151 bash[23756]: cluster 2024-03-04T15:50:51.231415+0000 mgr.y (mgr.24428) 32 : cluster 0 pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-04T15:50:53.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:52 smithi117 bash[21480]: cluster 2024-03-04T15:50:51.231415+0000 mgr.y (mgr.24428) 32 : cluster 0 pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-04T15:50:53.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:52 smithi117 bash[28843]: cluster 2024-03-04T15:50:51.231415+0000 mgr.y (mgr.24428) 32 : cluster 0 pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-04T15:50:54.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:54 smithi117 bash[21480]: cluster 2024-03-04T15:50:53.232933+0000 mgr.y (mgr.24428) 33 : cluster 0 pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-04T15:50:54.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:54 smithi117 bash[28843]: cluster 2024-03-04T15:50:53.232933+0000 mgr.y (mgr.24428) 33 : cluster 0 pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-04T15:50:54.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:54 smithi151 bash[23756]: cluster 2024-03-04T15:50:53.232933+0000 mgr.y (mgr.24428) 33 : cluster 0 pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 18 KiB/s rd, 0 B/s wr, 10 op/s 2024-03-04T15:50:55.854 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:55 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:55.854 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:55 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:55.855 INFO:journalctl@ceph.osd.0.smithi117.stdout:Mar 04 15:50:55 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:55.855 INFO:journalctl@ceph.osd.1.smithi117.stdout:Mar 04 15:50:55 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:55.855 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:55 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:55.856 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:55 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:55.856 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 04 15:50:55 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:55.856 INFO:journalctl@ceph.osd.2.smithi117.stdout:Mar 04 15:50:55 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:55.856 INFO:journalctl@ceph.rgw.foo.a.smithi117.stdout:Mar 04 15:50:55 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:56.105 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:55 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:56.105 INFO:journalctl@ceph.osd.0.smithi117.stdout:Mar 04 15:50:55 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:56.106 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 15:50:55 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:56.106 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:50:56 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:56.106 INFO:journalctl@ceph.osd.1.smithi117.stdout:Mar 04 15:50:55 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:56.107 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 04 15:50:55 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:56.107 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:50:55 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:56.107 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:50:55 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:56.107 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:50:55 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:56.107 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:50:56 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:56.108 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:55 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:56.108 INFO:journalctl@ceph.osd.2.smithi117.stdout:Mar 04 15:50:55 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:56.108 INFO:journalctl@ceph.rgw.foo.a.smithi117.stdout:Mar 04 15:50:55 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:50:56.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd981c20700 1 -- 172.21.15.117:0/23800473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c100f80 msgr2=0x7fd97c101360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:56.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd981c20700 1 --2- 172.21.15.117:0/23800473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c100f80 0x7fd97c101360 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fd96c009920 tx=0x7fd96c009c30 comp rx=0 tx=0).stop 2024-03-04T15:50:56.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd981c20700 1 -- 172.21.15.117:0/23800473 shutdown_connections 2024-03-04T15:50:56.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd981c20700 1 --2- 172.21.15.117:0/23800473 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd97c106060 0x7fd97c10dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:56.284 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd981c20700 1 --2- 172.21.15.117:0/23800473 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c100f80 0x7fd97c101360 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:56.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd981c20700 1 --2- 172.21.15.117:0/23800473 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd97c101930 0x7fd97c105980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:56.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd981c20700 1 -- 172.21.15.117:0/23800473 >> 172.21.15.117:0/23800473 conn(0x7fd97c077dd0 msgr2=0x7fd97c0781e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:50:56.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd981c20700 1 -- 172.21.15.117:0/23800473 shutdown_connections 2024-03-04T15:50:56.285 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd981c20700 1 -- 172.21.15.117:0/23800473 wait complete. 2024-03-04T15:50:56.286 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd981c20700 1 Processor -- start 2024-03-04T15:50:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd981c20700 1 -- start start 2024-03-04T15:50:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd981c20700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c100f80 0x7fd97c19b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd981c20700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd97c101930 0x7fd97c19bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd981c20700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd97c106060 0x7fd97c1958c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd981c20700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c10f4b0 con 0x7fd97c106060 2024-03-04T15:50:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd981c20700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c10f330 con 0x7fd97c101930 2024-03-04T15:50:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd981c20700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd97c10f630 con 0x7fd97c100f80 2024-03-04T15:50:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd97b7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c100f80 0x7fd97c19b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:50:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd97b7fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c100f80 0x7fd97c19b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:33072/0 (socket says 172.21.15.117:33072) 2024-03-04T15:50:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd97b7fe700 1 -- 172.21.15.117:0/577912095 learned_addr learned my addr 172.21.15.117:0/577912095 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:50:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd97bfff700 1 --2- 172.21.15.117:0/577912095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd97c106060 0x7fd97c1958c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:50:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd97affd700 1 --2- 172.21.15.117:0/577912095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd97c101930 0x7fd97c19bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:50:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd97b7fe700 1 -- 172.21.15.117:0/577912095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd97c101930 msgr2=0x7fd97c19bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd97b7fe700 1 --2- 172.21.15.117:0/577912095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd97c101930 0x7fd97c19bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:56.287 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd97b7fe700 1 -- 172.21.15.117:0/577912095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd97c106060 msgr2=0x7fd97c1958c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:56.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd97b7fe700 1 --2- 172.21.15.117:0/577912095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd97c106060 0x7fd97c1958c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:56.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd97b7fe700 1 -- 172.21.15.117:0/577912095 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd97c196020 con 0x7fd97c100f80 2024-03-04T15:50:56.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd97b7fe700 1 --2- 172.21.15.117:0/577912095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c100f80 0x7fd97c19b700 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd96c005e90 tx=0x7fd96c00fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:50:56.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd978ff9700 1 -- 172.21.15.117:0/577912095 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c027070 con 0x7fd97c100f80 2024-03-04T15:50:56.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd978ff9700 1 -- 172.21.15.117:0/577912095 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd96c005040 con 0x7fd97c100f80 2024-03-04T15:50:56.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd981c20700 1 -- 172.21.15.117:0/577912095 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd97c1962b0 con 0x7fd97c100f80 2024-03-04T15:50:56.288 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.283+0000 7fd978ff9700 1 -- 172.21.15.117:0/577912095 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96c004910 con 0x7fd97c100f80 2024-03-04T15:50:56.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.287+0000 7fd981c20700 1 -- 172.21.15.117:0/577912095 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd97c1a25d0 con 0x7fd97c100f80 2024-03-04T15:50:56.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.287+0000 7fd978ff9700 1 -- 172.21.15.117:0/577912095 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7fd96c02a070 con 0x7fd97c100f80 2024-03-04T15:50:56.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.287+0000 7fd978ff9700 1 --2- 172.21.15.117:0/577912095 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fd96406d780 0x7fd96406fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:50:56.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.287+0000 7fd981c20700 1 -- 172.21.15.117:0/577912095 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd97c069c30 con 0x7fd97c100f80 2024-03-04T15:50:56.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.287+0000 7fd978ff9700 1 -- 172.21.15.117:0/577912095 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7fd96c08e3f0 con 0x7fd97c100f80 2024-03-04T15:50:56.289 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.287+0000 7fd97affd700 1 --2- 172.21.15.117:0/577912095 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fd96406d780 0x7fd96406fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:50:56.291 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.291+0000 7fd97affd700 1 --2- 172.21.15.117:0/577912095 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fd96406d780 0x7fd96406fc40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd970009750 tx=0x7fd970009310 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:50:56.294 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.291+0000 7fd978ff9700 1 -- 172.21.15.117:0/577912095 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd96c013030 con 0x7fd97c100f80 2024-03-04T15:50:56.569 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:50:56 smithi117 systemd[1]: Started Ceph alertmanager.a for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T15:50:56.586 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.583+0000 7fd981c20700 1 -- 172.21.15.117:0/577912095 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7fd97c196df0 con 0x7fd97c100f80 2024-03-04T15:50:56.593 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.591+0000 7fd978ff9700 1 -- 172.21.15.117:0/577912095 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7fd96c05bcc0 con 0x7fd97c100f80 2024-03-04T15:50:56.594 INFO:teuthology.orchestra.run.smithi117.stdout:[client.0] 2024-03-04T15:50:56.594 INFO:teuthology.orchestra.run.smithi117.stdout: key = AQDg7eVl61ciIxAAh5yOjXOX0W99+kpYSjoC3A== 2024-03-04T15:50:56.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.595+0000 7fd981c20700 1 -- 172.21.15.117:0/577912095 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fd96406d780 msgr2=0x7fd96406fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:56.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.595+0000 7fd981c20700 1 --2- 172.21.15.117:0/577912095 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fd96406d780 0x7fd96406fc40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fd970009750 tx=0x7fd970009310 comp rx=0 tx=0).stop 2024-03-04T15:50:56.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.595+0000 7fd981c20700 1 -- 172.21.15.117:0/577912095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c100f80 msgr2=0x7fd97c19b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:50:56.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.595+0000 7fd981c20700 1 --2- 172.21.15.117:0/577912095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c100f80 0x7fd97c19b700 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd96c005e90 tx=0x7fd96c00fa70 comp rx=0 tx=0).stop 2024-03-04T15:50:56.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.595+0000 7fd981c20700 1 -- 172.21.15.117:0/577912095 shutdown_connections 2024-03-04T15:50:56.596 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.595+0000 7fd981c20700 1 --2- 172.21.15.117:0/577912095 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fd96406d780 0x7fd96406fc40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:56.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.595+0000 7fd981c20700 1 --2- 172.21.15.117:0/577912095 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd97c106060 0x7fd97c1958c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:56.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.595+0000 7fd981c20700 1 --2- 172.21.15.117:0/577912095 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd97c100f80 0x7fd97c19b700 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:56.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.595+0000 7fd981c20700 1 --2- 172.21.15.117:0/577912095 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd97c101930 0x7fd97c19bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:50:56.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.595+0000 7fd981c20700 1 -- 172.21.15.117:0/577912095 >> 172.21.15.117:0/577912095 conn(0x7fd97c077dd0 msgr2=0x7fd97c0feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:50:56.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.595+0000 7fd981c20700 1 -- 172.21.15.117:0/577912095 shutdown_connections 2024-03-04T15:50:56.597 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:50:56.595+0000 7fd981c20700 1 -- 172.21.15.117:0/577912095 wait complete. 2024-03-04T15:50:56.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:50:56 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:50:57.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:57 smithi151 bash[23756]: cluster 2024-03-04T15:50:55.233652+0000 mgr.y (mgr.24428) 34 : cluster 0 pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:50:57.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:57 smithi151 bash[23756]: audit 2024-03-04T15:50:56.155961+0000 mon.a (mon.0) 796 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:57.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:57 smithi151 bash[23756]: audit 2024-03-04T15:50:56.164474+0000 mon.a (mon.0) 797 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:57.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:57 smithi151 bash[23756]: audit 2024-03-04T15:50:56.194050+0000 mon.a (mon.0) 798 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:57.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:57 smithi151 bash[23756]: audit 2024-03-04T15:50:56.199469+0000 mon.a (mon.0) 799 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:57.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:57 smithi151 bash[23756]: audit 2024-03-04T15:50:56.265265+0000 mon.c (mon.2) 50 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:50:57.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:57 smithi151 bash[23756]: audit 2024-03-04T15:50:56.588786+0000 mon.c (mon.2) 51 : audit 1 from='client.? 172.21.15.117:0/577912095' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T15:50:57.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:57 smithi151 bash[23756]: audit 2024-03-04T15:50:56.589269+0000 mon.a (mon.0) 800 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T15:50:57.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:57 smithi151 bash[23756]: audit 2024-03-04T15:50:56.591703+0000 mon.a (mon.0) 801 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-04T15:50:57.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:57 smithi117 bash[21480]: cluster 2024-03-04T15:50:55.233652+0000 mgr.y (mgr.24428) 34 : cluster 0 pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:50:57.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:57 smithi117 bash[21480]: audit 2024-03-04T15:50:56.155961+0000 mon.a (mon.0) 796 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:57.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:57 smithi117 bash[21480]: audit 2024-03-04T15:50:56.164474+0000 mon.a (mon.0) 797 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:57.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:57 smithi117 bash[21480]: audit 2024-03-04T15:50:56.194050+0000 mon.a (mon.0) 798 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:57.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:57 smithi117 bash[21480]: audit 2024-03-04T15:50:56.199469+0000 mon.a (mon.0) 799 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:57.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:57 smithi117 bash[21480]: audit 2024-03-04T15:50:56.265265+0000 mon.c (mon.2) 50 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:50:57.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:57 smithi117 bash[21480]: audit 2024-03-04T15:50:56.588786+0000 mon.c (mon.2) 51 : audit 1 from='client.? 172.21.15.117:0/577912095' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T15:50:57.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:57 smithi117 bash[21480]: audit 2024-03-04T15:50:56.589269+0000 mon.a (mon.0) 800 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T15:50:57.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:57 smithi117 bash[21480]: audit 2024-03-04T15:50:56.591703+0000 mon.a (mon.0) 801 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-04T15:50:57.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:57 smithi117 bash[28843]: cluster 2024-03-04T15:50:55.233652+0000 mgr.y (mgr.24428) 34 : cluster 0 pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:50:57.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:57 smithi117 bash[28843]: audit 2024-03-04T15:50:56.155961+0000 mon.a (mon.0) 796 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:57.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:57 smithi117 bash[28843]: audit 2024-03-04T15:50:56.164474+0000 mon.a (mon.0) 797 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:57.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:57 smithi117 bash[28843]: audit 2024-03-04T15:50:56.194050+0000 mon.a (mon.0) 798 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:57.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:57 smithi117 bash[28843]: audit 2024-03-04T15:50:56.199469+0000 mon.a (mon.0) 799 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:50:57.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:57 smithi117 bash[28843]: audit 2024-03-04T15:50:56.265265+0000 mon.c (mon.2) 50 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:50:57.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:57 smithi117 bash[28843]: audit 2024-03-04T15:50:56.588786+0000 mon.c (mon.2) 51 : audit 1 from='client.? 172.21.15.117:0/577912095' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T15:50:57.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:57 smithi117 bash[28843]: audit 2024-03-04T15:50:56.589269+0000 mon.a (mon.0) 800 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T15:50:57.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:57 smithi117 bash[28843]: audit 2024-03-04T15:50:56.591703+0000 mon.a (mon.0) 801 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-04T15:50:57.947 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-04T15:50:57.947 DEBUG:teuthology.orchestra.run.smithi117:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-03-04T15:50:57.947 DEBUG:teuthology.orchestra.run.smithi117:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-03-04T15:50:57.971 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-03-04T15:50:58.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:58 smithi117 bash[21480]: audit 2024-03-04T15:50:56.296628+0000 mgr.y (mgr.24428) 35 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:50:58.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:58 smithi117 bash[28843]: audit 2024-03-04T15:50:56.296628+0000 mgr.y (mgr.24428) 35 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:50:58.320 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:50:58 smithi117 bash[57922]: ts=2024-03-04T15:50:58.094Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-03-04T15:50:58.320 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:50:58 smithi117 bash[57922]: ts=2024-03-04T15:50:58.094Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-03-04T15:50:58.320 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:50:58 smithi117 bash[57922]: ts=2024-03-04T15:50:58.096Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.117 port=9094 2024-03-04T15:50:58.321 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:50:58 smithi117 bash[57922]: ts=2024-03-04T15:50:58.097Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-03-04T15:50:58.321 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:50:58 smithi117 bash[57922]: ts=2024-03-04T15:50:58.127Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-04T15:50:58.321 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:50:58 smithi117 bash[57922]: ts=2024-03-04T15:50:58.127Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-04T15:50:58.321 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:50:58 smithi117 bash[57922]: ts=2024-03-04T15:50:58.130Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-03-04T15:50:58.321 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:50:58 smithi117 bash[57922]: ts=2024-03-04T15:50:58.130Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-03-04T15:50:58.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:58 smithi151 bash[23756]: audit 2024-03-04T15:50:56.296628+0000 mgr.y (mgr.24428) 35 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:50:59.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:50:59 smithi151 bash[23756]: cluster 2024-03-04T15:50:57.235133+0000 mgr.y (mgr.24428) 36 : cluster 0 pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:50:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:50:59 smithi117 bash[21480]: cluster 2024-03-04T15:50:57.235133+0000 mgr.y (mgr.24428) 36 : cluster 0 pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:50:59.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:50:59 smithi117 bash[28843]: cluster 2024-03-04T15:50:57.235133+0000 mgr.y (mgr.24428) 36 : cluster 0 pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:00.570 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:51:00 smithi117 bash[57922]: ts=2024-03-04T15:51:00.097Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000410078s 2024-03-04T15:51:01.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:01 smithi151 bash[23756]: cluster 2024-03-04T15:50:59.235850+0000 mgr.y (mgr.24428) 37 : cluster 0 pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:01.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:01 smithi151 bash[23756]: audit 2024-03-04T15:51:00.353578+0000 mon.a (mon.0) 802 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:01.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:01 smithi117 bash[21480]: cluster 2024-03-04T15:50:59.235850+0000 mgr.y (mgr.24428) 37 : cluster 0 pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:01.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:01 smithi117 bash[21480]: audit 2024-03-04T15:51:00.353578+0000 mon.a (mon.0) 802 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:01.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:01 smithi117 bash[28843]: cluster 2024-03-04T15:50:59.235850+0000 mgr.y (mgr.24428) 37 : cluster 0 pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:01.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:01 smithi117 bash[28843]: audit 2024-03-04T15:51:00.353578+0000 mon.a (mon.0) 802 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:02.297 INFO:teuthology.orchestra.run.smithi151.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.b/config 2024-03-04T15:51:03.295 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:03 smithi151 bash[23756]: cluster 2024-03-04T15:51:01.236553+0000 mgr.y (mgr.24428) 38 : cluster 0 pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:03.295 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:03 smithi151 bash[23756]: audit 2024-03-04T15:51:02.008270+0000 mon.a (mon.0) 803 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:03.295 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:03 smithi151 bash[23756]: audit 2024-03-04T15:51:02.023991+0000 mon.a (mon.0) 804 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:03.295 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:03 smithi151 bash[23756]: audit 2024-03-04T15:51:02.864869+0000 mon.a (mon.0) 805 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:03.295 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:03 smithi151 bash[23756]: audit 2024-03-04T15:51:02.875017+0000 mon.a (mon.0) 806 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:03.295 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:03 smithi151 bash[23756]: audit 2024-03-04T15:51:02.877764+0000 mon.c (mon.2) 52 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:51:03.295 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:03 smithi151 bash[23756]: audit 2024-03-04T15:51:02.879067+0000 mon.c (mon.2) 53 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:51:03.295 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:03 smithi151 bash[23756]: audit 2024-03-04T15:51:02.890220+0000 mon.a (mon.0) 807 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:03.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[21480]: cluster 2024-03-04T15:51:01.236553+0000 mgr.y (mgr.24428) 38 : cluster 0 pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:03.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[21480]: audit 2024-03-04T15:51:02.008270+0000 mon.a (mon.0) 803 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:03.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[21480]: audit 2024-03-04T15:51:02.023991+0000 mon.a (mon.0) 804 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:03.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[21480]: audit 2024-03-04T15:51:02.864869+0000 mon.a (mon.0) 805 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:03.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[21480]: audit 2024-03-04T15:51:02.875017+0000 mon.a (mon.0) 806 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:03.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[21480]: audit 2024-03-04T15:51:02.877764+0000 mon.c (mon.2) 52 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:51:03.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[21480]: audit 2024-03-04T15:51:02.879067+0000 mon.c (mon.2) 53 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:51:03.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[21480]: audit 2024-03-04T15:51:02.890220+0000 mon.a (mon.0) 807 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:03.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[28843]: cluster 2024-03-04T15:51:01.236553+0000 mgr.y (mgr.24428) 38 : cluster 0 pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:03.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[28843]: audit 2024-03-04T15:51:02.008270+0000 mon.a (mon.0) 803 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:03.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[28843]: audit 2024-03-04T15:51:02.023991+0000 mon.a (mon.0) 804 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:03.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[28843]: audit 2024-03-04T15:51:02.864869+0000 mon.a (mon.0) 805 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:03.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[28843]: audit 2024-03-04T15:51:02.875017+0000 mon.a (mon.0) 806 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:03.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[28843]: audit 2024-03-04T15:51:02.877764+0000 mon.c (mon.2) 52 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:51:03.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[28843]: audit 2024-03-04T15:51:02.879067+0000 mon.c (mon.2) 53 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:51:03.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[28843]: audit 2024-03-04T15:51:02.890220+0000 mon.a (mon.0) 807 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:03.819 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[21734]: [04/Mar/2024:15:51:03] ENGINE Bus STOPPING 2024-03-04T15:51:03.820 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[21734]: [04/Mar/2024:15:51:03] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-04T15:51:03.820 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[21734]: [04/Mar/2024:15:51:03] ENGINE Bus STOPPED 2024-03-04T15:51:03.820 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[21734]: [04/Mar/2024:15:51:03] ENGINE Bus STARTING 2024-03-04T15:51:03.820 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[21734]: [04/Mar/2024:15:51:03] ENGINE Serving on http://:::9283 2024-03-04T15:51:03.820 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:51:03 smithi117 bash[21734]: [04/Mar/2024:15:51:03] ENGINE Bus STARTED 2024-03-04T15:51:03.931 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:03 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:04.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:04 smithi117 bash[21480]: cephadm 2024-03-04T15:51:02.927245+0000 mgr.y (mgr.24428) 39 : cephadm 1 Regenerating cephadm self-signed grafana TLS certificates 2024-03-04T15:51:04.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:04 smithi117 bash[21480]: audit 2024-03-04T15:51:03.024263+0000 mon.a (mon.0) 808 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:04.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:04 smithi117 bash[21480]: audit 2024-03-04T15:51:03.036939+0000 mon.a (mon.0) 809 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:04.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:04 smithi117 bash[21480]: audit 2024-03-04T15:51:03.334923+0000 mon.c (mon.2) 54 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-04T15:51:04.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:04 smithi117 bash[21480]: audit 2024-03-04T15:51:03.344556+0000 mon.a (mon.0) 810 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:04.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:04 smithi117 bash[28843]: cephadm 2024-03-04T15:51:02.927245+0000 mgr.y (mgr.24428) 39 : cephadm 1 Regenerating cephadm self-signed grafana TLS certificates 2024-03-04T15:51:04.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:04 smithi117 bash[28843]: audit 2024-03-04T15:51:03.024263+0000 mon.a (mon.0) 808 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:04.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:04 smithi117 bash[28843]: audit 2024-03-04T15:51:03.036939+0000 mon.a (mon.0) 809 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:04.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:04 smithi117 bash[28843]: audit 2024-03-04T15:51:03.334923+0000 mon.c (mon.2) 54 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-04T15:51:04.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:04 smithi117 bash[28843]: audit 2024-03-04T15:51:03.344556+0000 mon.a (mon.0) 810 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:04.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:04 smithi151 bash[23756]: cephadm 2024-03-04T15:51:02.927245+0000 mgr.y (mgr.24428) 39 : cephadm 1 Regenerating cephadm self-signed grafana TLS certificates 2024-03-04T15:51:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:04 smithi151 bash[23756]: audit 2024-03-04T15:51:03.024263+0000 mon.a (mon.0) 808 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:04 smithi151 bash[23756]: audit 2024-03-04T15:51:03.036939+0000 mon.a (mon.0) 809 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:04 smithi151 bash[23756]: audit 2024-03-04T15:51:03.334923+0000 mon.c (mon.2) 54 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-04T15:51:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:04 smithi151 bash[23756]: audit 2024-03-04T15:51:03.344556+0000 mon.a (mon.0) 810 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:05.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:05 smithi117 bash[21480]: cluster 2024-03-04T15:51:03.334031+0000 mgr.y (mgr.24428) 40 : cluster 0 pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:05.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:05 smithi117 bash[21480]: audit 2024-03-04T15:51:03.336245+0000 mgr.y (mgr.24428) 41 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-04T15:51:05.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:05 smithi117 bash[21480]: cephadm 2024-03-04T15:51:03.355800+0000 mgr.y (mgr.24428) 42 : cephadm 1 Deploying daemon grafana.a on smithi151 2024-03-04T15:51:05.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:05 smithi117 bash[28843]: cluster 2024-03-04T15:51:03.334031+0000 mgr.y (mgr.24428) 40 : cluster 0 pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:05.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:05 smithi117 bash[28843]: audit 2024-03-04T15:51:03.336245+0000 mgr.y (mgr.24428) 41 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-04T15:51:05.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:05 smithi117 bash[28843]: cephadm 2024-03-04T15:51:03.355800+0000 mgr.y (mgr.24428) 42 : cephadm 1 Deploying daemon grafana.a on smithi151 2024-03-04T15:51:05.338 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:05 smithi151 bash[23756]: cluster 2024-03-04T15:51:03.334031+0000 mgr.y (mgr.24428) 40 : cluster 0 pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:05.338 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:05 smithi151 bash[23756]: audit 2024-03-04T15:51:03.336245+0000 mgr.y (mgr.24428) 41 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-03-04T15:51:05.338 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:05 smithi151 bash[23756]: cephadm 2024-03-04T15:51:03.355800+0000 mgr.y (mgr.24428) 42 : cephadm 1 Deploying daemon grafana.a on smithi151 2024-03-04T15:51:05.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90fae6f700 1 -- 172.21.15.151:0/2999869548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90f4106f60 msgr2=0x7f90f4107340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:05.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90fae6f700 1 --2- 172.21.15.151:0/2999869548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90f4106f60 0x7f90f4107340 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f90e40099c0 tx=0x7f90e4009cd0 comp rx=0 tx=0).stop 2024-03-04T15:51:05.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90fae6f700 1 -- 172.21.15.151:0/2999869548 shutdown_connections 2024-03-04T15:51:05.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90fae6f700 1 --2- 172.21.15.151:0/2999869548 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90f41010d0 0x7f90f410ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:05.524 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90fae6f700 1 --2- 172.21.15.151:0/2999869548 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90f4106f60 0x7f90f4107340 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:05.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90fae6f700 1 --2- 172.21.15.151:0/2999869548 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f4100710 0x7f90f4100b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:05.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90fae6f700 1 -- 172.21.15.151:0/2999869548 >> 172.21.15.151:0/2999869548 conn(0x7f90f4075ff0 msgr2=0x7f90f4076400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:05.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90fae6f700 1 -- 172.21.15.151:0/2999869548 shutdown_connections 2024-03-04T15:51:05.525 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90fae6f700 1 -- 172.21.15.151:0/2999869548 wait complete. 2024-03-04T15:51:05.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90fae6f700 1 Processor -- start 2024-03-04T15:51:05.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90fae6f700 1 -- start start 2024-03-04T15:51:05.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90fae6f700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90f4100710 0x7f90f419d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:05.526 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90fae6f700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f41010d0 0x7f90f419dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:05.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90fae6f700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90f4106f60 0x7f90f41978c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:05.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90fae6f700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f41115c0 con 0x7f90f41010d0 2024-03-04T15:51:05.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90fae6f700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90f4111440 con 0x7f90f4100710 2024-03-04T15:51:05.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90fae6f700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f90f4111740 con 0x7f90f4106f60 2024-03-04T15:51:05.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90f8c0b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90f4100710 0x7f90f419d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:05.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90f8c0b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90f4100710 0x7f90f419d6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.151:3300/0 says I am v2:172.21.15.151:58334/0 (socket says 172.21.15.151:58334) 2024-03-04T15:51:05.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90f8c0b700 1 -- 172.21.15.151:0/3921255251 learned_addr learned my addr 172.21.15.151:0/3921255251 (peer_addr_for_me v2:172.21.15.151:0/0) 2024-03-04T15:51:05.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90f3fff700 1 --2- 172.21.15.151:0/3921255251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f41010d0 0x7f90f419dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:05.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90f940c700 1 --2- 172.21.15.151:0/3921255251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90f4106f60 0x7f90f41978c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:05.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90f8c0b700 1 -- 172.21.15.151:0/3921255251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90f4106f60 msgr2=0x7f90f41978c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:05.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90f8c0b700 1 --2- 172.21.15.151:0/3921255251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90f4106f60 0x7f90f41978c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:05.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90f8c0b700 1 -- 172.21.15.151:0/3921255251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f41010d0 msgr2=0x7f90f419dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:05.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90f8c0b700 1 --2- 172.21.15.151:0/3921255251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f41010d0 0x7f90f419dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:05.527 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90f8c0b700 1 -- 172.21.15.151:0/3921255251 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90f4198020 con 0x7f90f4100710 2024-03-04T15:51:05.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90f3fff700 1 --2- 172.21.15.151:0/3921255251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f41010d0 0x7f90f419dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T15:51:05.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90f8c0b700 1 --2- 172.21.15.151:0/3921255251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90f4100710 0x7f90f419d6b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f90e40059d0 tx=0x7f90e40047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:05.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90f1ffb700 1 -- 172.21.15.151:0/3921255251 <== mon.1 v2:172.21.15.151:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4027070 con 0x7f90f4100710 2024-03-04T15:51:05.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90f1ffb700 1 -- 172.21.15.151:0/3921255251 <== mon.1 v2:172.21.15.151:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f90e4003f50 con 0x7f90f4100710 2024-03-04T15:51:05.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90fae6f700 1 -- 172.21.15.151:0/3921255251 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90f41982b0 con 0x7f90f4100710 2024-03-04T15:51:05.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90f1ffb700 1 -- 172.21.15.151:0/3921255251 <== mon.1 v2:172.21.15.151:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90e4022c60 con 0x7f90f4100710 2024-03-04T15:51:05.528 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.520+0000 7f90fae6f700 1 -- 172.21.15.151:0/3921255251 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90f41a4580 con 0x7f90f4100710 2024-03-04T15:51:05.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.524+0000 7f90fae6f700 1 -- 172.21.15.151:0/3921255251 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90f404faa0 con 0x7f90f4100710 2024-03-04T15:51:05.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.524+0000 7f90f1ffb700 1 -- 172.21.15.151:0/3921255251 <== mon.1 v2:172.21.15.151:3300/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f90e402a070 con 0x7f90f4100710 2024-03-04T15:51:05.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.524+0000 7f90f1ffb700 1 --2- 172.21.15.151:0/3921255251 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f90d806d780 0x7f90d806fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:05.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.524+0000 7f90f1ffb700 1 -- 172.21.15.151:0/3921255251 <== mon.1 v2:172.21.15.151:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7f90e408e820 con 0x7f90f4100710 2024-03-04T15:51:05.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.524+0000 7f90f3fff700 1 --2- 172.21.15.151:0/3921255251 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f90d806d780 0x7f90d806fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:05.529 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.524+0000 7f90f3fff700 1 --2- 172.21.15.151:0/3921255251 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f90d806d780 0x7f90d806fc40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f90e0005fd0 tx=0x7f90e0005e30 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:05.533 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.528+0000 7f90f1ffb700 1 -- 172.21.15.151:0/3921255251 <== mon.1 v2:172.21.15.151:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f90e4013070 con 0x7f90f4100710 2024-03-04T15:51:05.825 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.820+0000 7f90fae6f700 1 -- 172.21.15.151:0/3921255251 --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f90f402d830 con 0x7f90f4100710 2024-03-04T15:51:05.832 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.828+0000 7f90f1ffb700 1 -- 172.21.15.151:0/3921255251 <== mon.1 v2:172.21.15.151:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v18) v1 ==== 170+0+59 (secure 0 0 0) 0x7f90e405c0f0 con 0x7f90f4100710 2024-03-04T15:51:05.832 INFO:teuthology.orchestra.run.smithi151.stdout:[client.1] 2024-03-04T15:51:05.832 INFO:teuthology.orchestra.run.smithi151.stdout: key = AQDp7eVl7fFcMRAAZInrjtFvw2mCEs330OIROg== 2024-03-04T15:51:05.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.832+0000 7f90fae6f700 1 -- 172.21.15.151:0/3921255251 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f90d806d780 msgr2=0x7f90d806fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:05.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.832+0000 7f90fae6f700 1 --2- 172.21.15.151:0/3921255251 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f90d806d780 0x7f90d806fc40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f90e0005fd0 tx=0x7f90e0005e30 comp rx=0 tx=0).stop 2024-03-04T15:51:05.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.832+0000 7f90fae6f700 1 -- 172.21.15.151:0/3921255251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90f4100710 msgr2=0x7f90f419d6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:05.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.832+0000 7f90fae6f700 1 --2- 172.21.15.151:0/3921255251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90f4100710 0x7f90f419d6b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f90e40059d0 tx=0x7f90e40047b0 comp rx=0 tx=0).stop 2024-03-04T15:51:05.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.832+0000 7f90fae6f700 1 -- 172.21.15.151:0/3921255251 shutdown_connections 2024-03-04T15:51:05.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.832+0000 7f90fae6f700 1 --2- 172.21.15.151:0/3921255251 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f90f4106f60 0x7f90f41978c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:05.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.832+0000 7f90fae6f700 1 --2- 172.21.15.151:0/3921255251 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f90f4100710 0x7f90f419d6b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:05.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.832+0000 7f90fae6f700 1 --2- 172.21.15.151:0/3921255251 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f90d806d780 0x7f90d806fc40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:05.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.832+0000 7f90fae6f700 1 --2- 172.21.15.151:0/3921255251 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f90f41010d0 0x7f90f419dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:05.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.832+0000 7f90fae6f700 1 -- 172.21.15.151:0/3921255251 >> 172.21.15.151:0/3921255251 conn(0x7f90f4075ff0 msgr2=0x7f90f40ff450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:05.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.832+0000 7f90fae6f700 1 -- 172.21.15.151:0/3921255251 shutdown_connections 2024-03-04T15:51:05.835 INFO:teuthology.orchestra.run.smithi151.stderr:2024-03-04T15:51:05.832+0000 7f90fae6f700 1 -- 172.21.15.151:0/3921255251 wait complete. 2024-03-04T15:51:06.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:06 smithi151 bash[23756]: audit 2024-03-04T15:51:05.389028+0000 mon.c (mon.2) 55 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:51:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:06 smithi151 bash[23756]: audit 2024-03-04T15:51:05.825992+0000 mon.b (mon.1) 32 : audit 1 from='client.? 172.21.15.151:0/3921255251' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T15:51:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:06 smithi151 bash[23756]: audit 2024-03-04T15:51:05.827863+0000 mon.a (mon.0) 811 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T15:51:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:06 smithi151 bash[23756]: audit 2024-03-04T15:51:05.831252+0000 mon.a (mon.0) 812 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-04T15:51:06.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:06 smithi117 bash[21480]: audit 2024-03-04T15:51:05.389028+0000 mon.c (mon.2) 55 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:51:06.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:06 smithi117 bash[21480]: audit 2024-03-04T15:51:05.825992+0000 mon.b (mon.1) 32 : audit 1 from='client.? 172.21.15.151:0/3921255251' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T15:51:06.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:06 smithi117 bash[21480]: audit 2024-03-04T15:51:05.827863+0000 mon.a (mon.0) 811 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T15:51:06.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:06 smithi117 bash[21480]: audit 2024-03-04T15:51:05.831252+0000 mon.a (mon.0) 812 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-04T15:51:06.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:06 smithi117 bash[28843]: audit 2024-03-04T15:51:05.389028+0000 mon.c (mon.2) 55 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:51:06.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:06 smithi117 bash[28843]: audit 2024-03-04T15:51:05.825992+0000 mon.b (mon.1) 32 : audit 1 from='client.? 172.21.15.151:0/3921255251' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T15:51:06.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:06 smithi117 bash[28843]: audit 2024-03-04T15:51:05.827863+0000 mon.a (mon.0) 811 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-03-04T15:51:06.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:06 smithi117 bash[28843]: audit 2024-03-04T15:51:05.831252+0000 mon.a (mon.0) 812 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-03-04T15:51:06.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:51:06 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:51:06.744 DEBUG:teuthology.orchestra.run.smithi151:> set -ex 2024-03-04T15:51:06.744 DEBUG:teuthology.orchestra.run.smithi151:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-03-04T15:51:06.744 DEBUG:teuthology.orchestra.run.smithi151:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-03-04T15:51:06.767 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-03-04T15:51:06.767 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-03-04T15:51:06.768 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph mgr dump --format=json 2024-03-04T15:51:07.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:07 smithi117 bash[21480]: cluster 2024-03-04T15:51:05.334719+0000 mgr.y (mgr.24428) 43 : cluster 0 pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 846 B/s rd, 0 op/s 2024-03-04T15:51:07.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:07 smithi117 bash[28843]: cluster 2024-03-04T15:51:05.334719+0000 mgr.y (mgr.24428) 43 : cluster 0 pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 846 B/s rd, 0 op/s 2024-03-04T15:51:07.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:07 smithi151 bash[23756]: cluster 2024-03-04T15:51:05.334719+0000 mgr.y (mgr.24428) 43 : cluster 0 pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 846 B/s rd, 0 op/s 2024-03-04T15:51:08.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:08 smithi117 bash[21480]: audit 2024-03-04T15:51:06.305014+0000 mgr.y (mgr.24428) 44 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:51:08.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:08 smithi117 bash[28843]: audit 2024-03-04T15:51:06.305014+0000 mgr.y (mgr.24428) 44 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:51:08.320 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:51:08 smithi117 bash[57922]: ts=2024-03-04T15:51:08.100Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003250609s 2024-03-04T15:51:08.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:08 smithi151 bash[23756]: audit 2024-03-04T15:51:06.305014+0000 mgr.y (mgr.24428) 44 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:51:09.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:09 smithi117 bash[21480]: cluster 2024-03-04T15:51:07.336147+0000 mgr.y (mgr.24428) 45 : cluster 0 pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:09.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:09 smithi117 bash[28843]: cluster 2024-03-04T15:51:07.336147+0000 mgr.y (mgr.24428) 45 : cluster 0 pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:09.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:09 smithi151 bash[23756]: cluster 2024-03-04T15:51:07.336147+0000 mgr.y (mgr.24428) 45 : cluster 0 pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:11.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:11 smithi117 bash[21480]: cluster 2024-03-04T15:51:09.336826+0000 mgr.y (mgr.24428) 46 : cluster 0 pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 846 B/s rd, 0 op/s 2024-03-04T15:51:11.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:11 smithi117 bash[28843]: cluster 2024-03-04T15:51:09.336826+0000 mgr.y (mgr.24428) 46 : cluster 0 pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 846 B/s rd, 0 op/s 2024-03-04T15:51:11.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:11 smithi151 bash[23756]: cluster 2024-03-04T15:51:09.336826+0000 mgr.y (mgr.24428) 46 : cluster 0 pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 846 B/s rd, 0 op/s 2024-03-04T15:51:11.545 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:51:13.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:13 smithi117 bash[21480]: cluster 2024-03-04T15:51:11.338391+0000 mgr.y (mgr.24428) 47 : cluster 0 pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:13.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:13 smithi117 bash[28843]: cluster 2024-03-04T15:51:11.338391+0000 mgr.y (mgr.24428) 47 : cluster 0 pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:13.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:13 smithi151 bash[23756]: cluster 2024-03-04T15:51:11.338391+0000 mgr.y (mgr.24428) 47 : cluster 0 pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:14.261 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca31911700 1 -- 172.21.15.117:0/1458337462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca2c106550 msgr2=0x7fca2c106930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:14.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca31911700 1 --2- 172.21.15.117:0/1458337462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca2c106550 0x7fca2c106930 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009990 tx=0x7fca1c009ca0 comp rx=0 tx=0).stop 2024-03-04T15:51:14.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca31911700 1 -- 172.21.15.117:0/1458337462 shutdown_connections 2024-03-04T15:51:14.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca31911700 1 --2- 172.21.15.117:0/1458337462 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca2c106550 0x7fca2c106930 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:14.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca31911700 1 --2- 172.21.15.117:0/1458337462 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca2c101100 0x7fca2c10d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:14.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca31911700 1 --2- 172.21.15.117:0/1458337462 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca2c100740 0x7fca2c100bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:14.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca31911700 1 -- 172.21.15.117:0/1458337462 >> 172.21.15.117:0/1458337462 conn(0x7fca2c077e50 msgr2=0x7fca2c0fe1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:14.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca31911700 1 -- 172.21.15.117:0/1458337462 shutdown_connections 2024-03-04T15:51:14.262 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca31911700 1 -- 172.21.15.117:0/1458337462 wait complete. 2024-03-04T15:51:14.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca31911700 1 Processor -- start 2024-03-04T15:51:14.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca31911700 1 -- start start 2024-03-04T15:51:14.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca31911700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca2c100740 0x7fca2c19f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:14.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca31911700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca2c101100 0x7fca2c19fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:14.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca31911700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca2c106550 0x7fca2c199ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:14.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca31911700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c110550 con 0x7fca2c101100 2024-03-04T15:51:14.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca31911700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c1103d0 con 0x7fca2c106550 2024-03-04T15:51:14.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca31911700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c1106d0 con 0x7fca2c100740 2024-03-04T15:51:14.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca2a7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca2c101100 0x7fca2c19fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:14.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca2affd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca2c100740 0x7fca2c19f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:14.264 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca2a7fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca2c101100 0x7fca2c19fe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49218/0 (socket says 172.21.15.117:49218) 2024-03-04T15:51:14.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca2a7fc700 1 -- 172.21.15.117:0/2716829927 learned_addr learned my addr 172.21.15.117:0/2716829927 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:51:14.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca2affd700 1 --2- 172.21.15.117:0/2716829927 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca2c100740 0x7fca2c19f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:37652/0 (socket says 172.21.15.117:37652) 2024-03-04T15:51:14.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca2b7fe700 1 --2- 172.21.15.117:0/2716829927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca2c106550 0x7fca2c199ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:14.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca2a7fc700 1 -- 172.21.15.117:0/2716829927 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca2c100740 msgr2=0x7fca2c19f910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:14.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca2a7fc700 1 --2- 172.21.15.117:0/2716829927 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca2c100740 0x7fca2c19f910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:14.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca2a7fc700 1 -- 172.21.15.117:0/2716829927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca2c106550 msgr2=0x7fca2c199ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:14.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca2a7fc700 1 --2- 172.21.15.117:0/2716829927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca2c106550 0x7fca2c199ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:14.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.259+0000 7fca2a7fc700 1 -- 172.21.15.117:0/2716829927 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c19a230 con 0x7fca2c101100 2024-03-04T15:51:14.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.263+0000 7fca2a7fc700 1 --2- 172.21.15.117:0/2716829927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca2c101100 0x7fca2c19fe50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fca1800e9a0 tx=0x7fca1800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:14.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.263+0000 7fca3090f700 1 -- 172.21.15.117:0/2716829927 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1800cbd0 con 0x7fca2c101100 2024-03-04T15:51:14.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.263+0000 7fca3090f700 1 -- 172.21.15.117:0/2716829927 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fca1800cd70 con 0x7fca2c101100 2024-03-04T15:51:14.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.263+0000 7fca31911700 1 -- 172.21.15.117:0/2716829927 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2c19a520 con 0x7fca2c101100 2024-03-04T15:51:14.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.263+0000 7fca3090f700 1 -- 172.21.15.117:0/2716829927 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18018680 con 0x7fca2c101100 2024-03-04T15:51:14.265 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.263+0000 7fca31911700 1 -- 172.21.15.117:0/2716829927 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca2c1a67e0 con 0x7fca2c101100 2024-03-04T15:51:14.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.263+0000 7fca31911700 1 -- 172.21.15.117:0/2716829927 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca2c04faa0 con 0x7fca2c101100 2024-03-04T15:51:14.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.263+0000 7fca3090f700 1 -- 172.21.15.117:0/2716829927 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7fca18010430 con 0x7fca2c101100 2024-03-04T15:51:14.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.263+0000 7fca3090f700 1 --2- 172.21.15.117:0/2716829927 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fca1006d780 0x7fca1006fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:14.266 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.263+0000 7fca3090f700 1 -- 172.21.15.117:0/2716829927 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7fca1808dc30 con 0x7fca2c101100 2024-03-04T15:51:14.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.263+0000 7fca2affd700 1 --2- 172.21.15.117:0/2716829927 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fca1006d780 0x7fca1006fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:14.267 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.263+0000 7fca2affd700 1 --2- 172.21.15.117:0/2716829927 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fca1006d780 0x7fca1006fc40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fca1c006010 tx=0x7fca1c00b280 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:14.270 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.267+0000 7fca3090f700 1 -- 172.21.15.117:0/2716829927 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fca1805b480 con 0x7fca2c101100 2024-03-04T15:51:14.532 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.531+0000 7fca31911700 1 -- 172.21.15.117:0/2716829927 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7fca2c1a6a10 con 0x7fca2c101100 2024-03-04T15:51:14.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.539+0000 7fca3090f700 1 -- 172.21.15.117:0/2716829927 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) v1 ==== 74+0+174823 (secure 0 0 0) 0x7fca1805eed0 con 0x7fca2c101100 2024-03-04T15:51:14.544 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-04T15:51:14.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.543+0000 7fca31911700 1 -- 172.21.15.117:0/2716829927 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fca1006d780 msgr2=0x7fca1006fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:14.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.543+0000 7fca31911700 1 --2- 172.21.15.117:0/2716829927 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fca1006d780 0x7fca1006fc40 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fca1c006010 tx=0x7fca1c00b280 comp rx=0 tx=0).stop 2024-03-04T15:51:14.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.543+0000 7fca31911700 1 -- 172.21.15.117:0/2716829927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca2c101100 msgr2=0x7fca2c19fe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:14.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.543+0000 7fca31911700 1 --2- 172.21.15.117:0/2716829927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca2c101100 0x7fca2c19fe50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fca1800e9a0 tx=0x7fca1800ecb0 comp rx=0 tx=0).stop 2024-03-04T15:51:14.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.543+0000 7fca31911700 1 -- 172.21.15.117:0/2716829927 shutdown_connections 2024-03-04T15:51:14.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.543+0000 7fca31911700 1 --2- 172.21.15.117:0/2716829927 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca2c106550 0x7fca2c199ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:14.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.543+0000 7fca31911700 1 --2- 172.21.15.117:0/2716829927 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca2c100740 0x7fca2c19f910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:14.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.543+0000 7fca31911700 1 --2- 172.21.15.117:0/2716829927 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fca1006d780 0x7fca1006fc40 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:14.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.543+0000 7fca31911700 1 --2- 172.21.15.117:0/2716829927 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca2c101100 0x7fca2c19fe50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:14.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.543+0000 7fca31911700 1 -- 172.21.15.117:0/2716829927 >> 172.21.15.117:0/2716829927 conn(0x7fca2c077e50 msgr2=0x7fca2c1044e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:14.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.543+0000 7fca31911700 1 -- 172.21.15.117:0/2716829927 shutdown_connections 2024-03-04T15:51:14.546 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:14.543+0000 7fca31911700 1 -- 172.21.15.117:0/2716829927 wait complete. 2024-03-04T15:51:15.238 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":21,"flags":0,"active_gid":24428,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6800","nonce":2727720514},{"type":"v1","addr":"172.21.15.117:6801","nonce":2727720514}]},"active_addr":"172.21.15.117:6801/2727720514","active_change":"2024-03-04T15:50:35.196233+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24469,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.0.0","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.117:8443/","prometheus":"http://172.21.15.117:9283/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":75,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":3357908652}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":3476782081}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":3949151892}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.117:0","nonce":914064032}]}]} 2024-03-04T15:51:15.242 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-03-04T15:51:15.242 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-03-04T15:51:15.242 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-04T15:51:15.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:15 smithi151 bash[23756]: cluster 2024-03-04T15:51:13.338877+0000 mgr.y (mgr.24428) 48 : cluster 0 pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:15.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:15 smithi151 bash[23756]: audit 2024-03-04T15:51:14.534592+0000 mon.a (mon.0) 813 : audit 0 from='client.? 172.21.15.117:0/2716829927' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-04T15:51:15.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:15 smithi117 bash[21480]: cluster 2024-03-04T15:51:13.338877+0000 mgr.y (mgr.24428) 48 : cluster 0 pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:15.578 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:15 smithi117 bash[21480]: audit 2024-03-04T15:51:14.534592+0000 mon.a (mon.0) 813 : audit 0 from='client.? 172.21.15.117:0/2716829927' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-04T15:51:15.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:15 smithi117 bash[28843]: cluster 2024-03-04T15:51:13.338877+0000 mgr.y (mgr.24428) 48 : cluster 0 pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:15.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:15 smithi117 bash[28843]: audit 2024-03-04T15:51:14.534592+0000 mon.a (mon.0) 813 : audit 0 from='client.? 172.21.15.117:0/2716829927' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-03-04T15:51:16.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:51:16 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:51:17.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:17 smithi151 bash[23756]: cluster 2024-03-04T15:51:15.339501+0000 mgr.y (mgr.24428) 49 : cluster 0 pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:17.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:17 smithi117 bash[21480]: cluster 2024-03-04T15:51:15.339501+0000 mgr.y (mgr.24428) 49 : cluster 0 pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:17 smithi117 bash[28843]: cluster 2024-03-04T15:51:15.339501+0000 mgr.y (mgr.24428) 49 : cluster 0 pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:18.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:18 smithi151 bash[23756]: audit 2024-03-04T15:51:16.313009+0000 mgr.y (mgr.24428) 50 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:51:18.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:18 smithi117 bash[21480]: audit 2024-03-04T15:51:16.313009+0000 mgr.y (mgr.24428) 50 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:51:18.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:18 smithi117 bash[28843]: audit 2024-03-04T15:51:16.313009+0000 mgr.y (mgr.24428) 50 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:51:19.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:19 smithi151 bash[23756]: cluster 2024-03-04T15:51:17.340954+0000 mgr.y (mgr.24428) 51 : cluster 0 pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:19.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:19 smithi117 bash[21480]: cluster 2024-03-04T15:51:17.340954+0000 mgr.y (mgr.24428) 51 : cluster 0 pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:19.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:19 smithi117 bash[28843]: cluster 2024-03-04T15:51:17.340954+0000 mgr.y (mgr.24428) 51 : cluster 0 pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:20.025 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:51:21.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:21 smithi151 bash[23756]: cluster 2024-03-04T15:51:19.341686+0000 mgr.y (mgr.24428) 52 : cluster 0 pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:21.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:21 smithi151 bash[23756]: audit 2024-03-04T15:51:20.418716+0000 mon.c (mon.2) 56 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:51:21.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:21 smithi117 bash[21480]: cluster 2024-03-04T15:51:19.341686+0000 mgr.y (mgr.24428) 52 : cluster 0 pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:21.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:21 smithi117 bash[21480]: audit 2024-03-04T15:51:20.418716+0000 mon.c (mon.2) 56 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:51:21.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:21 smithi117 bash[28843]: cluster 2024-03-04T15:51:19.341686+0000 mgr.y (mgr.24428) 52 : cluster 0 pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:21.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:21 smithi117 bash[28843]: audit 2024-03-04T15:51:20.418716+0000 mon.c (mon.2) 56 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:51:22.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b8b3bd700 1 -- 172.21.15.117:0/2757742488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b84108410 msgr2=0x7f7b84108890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:22.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b8b3bd700 1 --2- 172.21.15.117:0/2757742488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b84108410 0x7f7b84108890 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7b74009990 tx=0x7f7b74009ca0 comp rx=0 tx=0).stop 2024-03-04T15:51:22.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b8b3bd700 1 -- 172.21.15.117:0/2757742488 shutdown_connections 2024-03-04T15:51:22.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b8b3bd700 1 --2- 172.21.15.117:0/2757742488 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b84108dd0 0x7f7b841152e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:22.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b8b3bd700 1 --2- 172.21.15.117:0/2757742488 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b8410e4a0 0x7f7b8410e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:22.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b8b3bd700 1 --2- 172.21.15.117:0/2757742488 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b84108410 0x7f7b84108890 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:22.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b8b3bd700 1 -- 172.21.15.117:0/2757742488 >> 172.21.15.117:0/2757742488 conn(0x7f7b840755b0 msgr2=0x7f7b840779d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:22.506 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b8b3bd700 1 -- 172.21.15.117:0/2757742488 shutdown_connections 2024-03-04T15:51:22.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b8b3bd700 1 -- 172.21.15.117:0/2757742488 wait complete. 2024-03-04T15:51:22.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b8b3bd700 1 Processor -- start 2024-03-04T15:51:22.507 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b8b3bd700 1 -- start start 2024-03-04T15:51:22.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b8b3bd700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b84108410 0x7f7b8419e100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:22.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b8b3bd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b84108dd0 0x7f7b8419e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:22.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b8b3bd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b8410e4a0 0x7f7b841a29d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:22.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b8b3bd700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b84117ba0 con 0x7f7b84108dd0 2024-03-04T15:51:22.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b8b3bd700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b84117a20 con 0x7f7b84108410 2024-03-04T15:51:22.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b8b3bd700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b84117d20 con 0x7f7b8410e4a0 2024-03-04T15:51:22.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b88958700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b84108dd0 0x7f7b8419e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:22.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b88958700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b84108dd0 0x7f7b8419e640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:44052/0 (socket says 172.21.15.117:44052) 2024-03-04T15:51:22.510 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b88958700 1 -- 172.21.15.117:0/4004013348 learned_addr learned my addr 172.21.15.117:0/4004013348 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:51:22.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b8995a700 1 --2- 172.21.15.117:0/4004013348 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b8410e4a0 0x7f7b841a29d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:22.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b89159700 1 --2- 172.21.15.117:0/4004013348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b84108410 0x7f7b8419e100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:22.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b88958700 1 -- 172.21.15.117:0/4004013348 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b8410e4a0 msgr2=0x7f7b841a29d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:22.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b88958700 1 --2- 172.21.15.117:0/4004013348 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b8410e4a0 0x7f7b841a29d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:22.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b88958700 1 -- 172.21.15.117:0/4004013348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b84108410 msgr2=0x7f7b8419e100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:22.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b88958700 1 --2- 172.21.15.117:0/4004013348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b84108410 0x7f7b8419e100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:22.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b88958700 1 -- 172.21.15.117:0/4004013348 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b841a3110 con 0x7f7b84108dd0 2024-03-04T15:51:22.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.503+0000 7f7b8995a700 1 --2- 172.21.15.117:0/4004013348 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b8410e4a0 0x7f7b841a29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T15:51:22.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.507+0000 7f7b88958700 1 --2- 172.21.15.117:0/4004013348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b84108dd0 0x7f7b8419e640 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7b74000c00 tx=0x7f7b740047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:22.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.507+0000 7f7b7a7fc700 1 -- 172.21.15.117:0/4004013348 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b74027070 con 0x7f7b84108dd0 2024-03-04T15:51:22.511 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.507+0000 7f7b7a7fc700 1 -- 172.21.15.117:0/4004013348 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f7b74022470 con 0x7f7b84108dd0 2024-03-04T15:51:22.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.507+0000 7f7b7a7fc700 1 -- 172.21.15.117:0/4004013348 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b74017650 con 0x7f7b84108dd0 2024-03-04T15:51:22.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.507+0000 7f7b8b3bd700 1 -- 172.21.15.117:0/4004013348 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b841a33a0 con 0x7f7b84108dd0 2024-03-04T15:51:22.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.507+0000 7f7b8b3bd700 1 -- 172.21.15.117:0/4004013348 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b841a3800 con 0x7f7b84108dd0 2024-03-04T15:51:22.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.507+0000 7f7b8b3bd700 1 -- 172.21.15.117:0/4004013348 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b8404faa0 con 0x7f7b84108dd0 2024-03-04T15:51:22.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.507+0000 7f7b7a7fc700 1 -- 172.21.15.117:0/4004013348 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f7b74009d10 con 0x7f7b84108dd0 2024-03-04T15:51:22.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.507+0000 7f7b7a7fc700 1 --2- 172.21.15.117:0/4004013348 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f7b7006d780 0x7f7b7006fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:22.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.507+0000 7f7b7a7fc700 1 -- 172.21.15.117:0/4004013348 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7f7b7408e7f0 con 0x7f7b84108dd0 2024-03-04T15:51:22.512 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.507+0000 7f7b89159700 1 --2- 172.21.15.117:0/4004013348 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f7b7006d780 0x7f7b7006fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:22.513 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.507+0000 7f7b89159700 1 --2- 172.21.15.117:0/4004013348 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f7b7006d780 0x7f7b7006fc40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009680 tx=0x7f7b80009260 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:22.515 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.515+0000 7f7b7a7fc700 1 -- 172.21.15.117:0/4004013348 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7b841a3800 con 0x7f7b84108dd0 2024-03-04T15:51:22.718 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.715+0000 7f7b8b3bd700 1 -- 172.21.15.117:0/4004013348 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7b841aac80 con 0x7f7b84108dd0 2024-03-04T15:51:22.720 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.719+0000 7f7b7a7fc700 1 -- 172.21.15.117:0/4004013348 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+23286 (secure 0 0 0) 0x7f7b74013070 con 0x7f7b84108dd0 2024-03-04T15:51:22.721 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-04T15:51:22.721 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":75,"fsid":"e40b5504-da3c-11ee-95c6-87774f69a715","created":"2024-03-04T15:39:15.691654+0000","modified":"2024-03-04T15:50:35.195886+0000","last_up_change":"2024-03-04T15:48:54.249045+0000","last_in_change":"2024-03-04T15:48:27.158859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T15:44:31.324972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-03-04T15:49:32.345716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-03-04T15:49:34.290955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-03-04T15:49:36.207393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"datapool","create_time":"2024-03-04T15:49:36.958754+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-03-04T15:49:38.730661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf1f40a0-bf1a-4681-84de-fbf6fee19de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1332942120},{"type":"v1","addr":"172.21.15.117:6803","nonce":1332942120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1332942120},{"type":"v1","addr":"172.21.15.117:6805","nonce":1332942120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1332942120},{"type":"v1","addr":"172.21.15.117:6809","nonce":1332942120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1332942120},{"type":"v1","addr":"172.21.15.117:6807","nonce":1332942120}]},"public_addr":"172.21.15.117:6803/1332942120","cluster_addr":"172.21.15.117:6805/1332942120","heartbeat_back_addr":"172.21.15.117:6809/1332942120","heartbeat_front_addr":"172.21.15.117:6807/1332942120","state":["exists","up"]},{"osd":1,"uuid":"9e7ba793-aa0e-4435-bfd6-1bf6d28da6ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":444026972},{"type":"v1","addr":"172.21.15.117:6811","nonce":444026972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":444026972},{"type":"v1","addr":"172.21.15.117:6813","nonce":444026972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":444026972},{"type":"v1","addr":"172.21.15.117:6817","nonce":444026972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":444026972},{"type":"v1","addr":"172.21.15.117:6815","nonce":444026972}]},"public_addr":"172.21.15.117:6811/444026972","cluster_addr":"172.21.15.117:6813/444026972","heartbeat_back_addr":"172.21.15.117:6817/444026972","heartbeat_front_addr":"172.21.15.117:6815/444026972","state":["exists","up"]},{"osd":2,"uuid":"49710dbf-78b9-4983-bc61-2a5d0527e417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4102123427},{"type":"v1","addr":"172.21.15.117:6819","nonce":4102123427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":4102123427},{"type":"v1","addr":"172.21.15.117:6821","nonce":4102123427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":4102123427},{"type":"v1","addr":"172.21.15.117:6825","nonce":4102123427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4102123427},{"type":"v1","addr":"172.21.15.117:6823","nonce":4102123427}]},"public_addr":"172.21.15.117:6819/4102123427","cluster_addr":"172.21.15.117:6821/4102123427","heartbeat_back_addr":"172.21.15.117:6825/4102123427","heartbeat_front_addr":"172.21.15.117:6823/4102123427","state":["exists","up"]},{"osd":3,"uuid":"1e1fa658-d1a9-4129-ac7e-18f317b3b318","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3320151079},{"type":"v1","addr":"172.21.15.117:6827","nonce":3320151079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3320151079},{"type":"v1","addr":"172.21.15.117:6829","nonce":3320151079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3320151079},{"type":"v1","addr":"172.21.15.117:6833","nonce":3320151079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3320151079},{"type":"v1","addr":"172.21.15.117:6831","nonce":3320151079}]},"public_addr":"172.21.15.117:6827/3320151079","cluster_addr":"172.21.15.117:6829/3320151079","heartbeat_back_addr":"172.21.15.117:6833/3320151079","heartbeat_front_addr":"172.21.15.117:6831/3320151079","state":["exists","up"]},{"osd":4,"uuid":"35746d23-4625-45a9-81cd-c05f7ea5a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":1293012079},{"type":"v1","addr":"172.21.15.151:6801","nonce":1293012079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":1293012079},{"type":"v1","addr":"172.21.15.151:6803","nonce":1293012079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":1293012079},{"type":"v1","addr":"172.21.15.151:6807","nonce":1293012079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":1293012079},{"type":"v1","addr":"172.21.15.151:6805","nonce":1293012079}]},"public_addr":"172.21.15.151:6801/1293012079","cluster_addr":"172.21.15.151:6803/1293012079","heartbeat_back_addr":"172.21.15.151:6807/1293012079","heartbeat_front_addr":"172.21.15.151:6805/1293012079","state":["exists","up"]},{"osd":5,"uuid":"9660e52c-72e0-4f9c-8cf5-bf5868080e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3277640682},{"type":"v1","addr":"172.21.15.151:6809","nonce":3277640682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3277640682},{"type":"v1","addr":"172.21.15.151:6811","nonce":3277640682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3277640682},{"type":"v1","addr":"172.21.15.151:6815","nonce":3277640682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3277640682},{"type":"v1","addr":"172.21.15.151:6813","nonce":3277640682}]},"public_addr":"172.21.15.151:6809/3277640682","cluster_addr":"172.21.15.151:6811/3277640682","heartbeat_back_addr":"172.21.15.151:6815/3277640682","heartbeat_front_addr":"172.21.15.151:6813/3277640682","state":["exists","up"]},{"osd":6,"uuid":"706cd130-5cc0-41f3-9e78-85922c29a01c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":3501440615},{"type":"v1","addr":"172.21.15.151:6817","nonce":3501440615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":3501440615},{"type":"v1","addr":"172.21.15.151:6819","nonce":3501440615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":3501440615},{"type":"v1","addr":"172.21.15.151:6823","nonce":3501440615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":3501440615},{"type":"v1","addr":"172.21.15.151:6821","nonce":3501440615}]},"public_addr":"172.21.15.151:6817/3501440615","cluster_addr":"172.21.15.151:6819/3501440615","heartbeat_back_addr":"172.21.15.151:6823/3501440615","heartbeat_front_addr":"172.21.15.151:6821/3501440615","state":["exists","up"]},{"osd":7,"uuid":"0f49ea27-0d78-4e68-8d06-b3c565e37a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3965450917},{"type":"v1","addr":"172.21.15.151:6825","nonce":3965450917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3965450917},{"type":"v1","addr":"172.21.15.151:6827","nonce":3965450917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3965450917},{"type":"v1","addr":"172.21.15.151:6831","nonce":3965450917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3965450917},{"type":"v1","addr":"172.21.15.151:6829","nonce":3965450917}]},"public_addr":"172.21.15.151:6825/3965450917","cluster_addr":"172.21.15.151:6827/3965450917","heartbeat_back_addr":"172.21.15.151:6831/3965450917","heartbeat_front_addr":"172.21.15.151:6829/3965450917","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:42:47.269066+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:43:37.699556+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:44:28.025013+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:45:20.549377+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:46:11.142621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:47:04.620036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:47:58.871589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:48:52.475968+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1f","mappings":[{"from":7,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/2488053720":"2024-03-05T15:39:47.643279+0000","172.21.15.117:6801/427955642":"2024-03-05T15:39:47.643279+0000","172.21.15.117:0/2167394439":"2024-03-05T15:39:47.643279+0000","172.21.15.117:6800/427955642":"2024-03-05T15:39:47.643279+0000","172.21.15.117:6800/1454790014":"2024-03-05T15:40:22.691924+0000","172.21.15.117:0/2490993128":"2024-03-05T15:50:35.195755+0000","172.21.15.117:0/163360369":"2024-03-05T15:40:22.691924+0000","172.21.15.117:0/4187340781":"2024-03-05T15:39:47.643279+0000","172.21.15.117:0/546245806":"2024-03-05T15:40:22.691924+0000","172.21.15.117:0/2279801001":"2024-03-05T15:50:35.195755+0000","172.21.15.117:0/2351166229":"2024-03-05T15:50:35.195755+0000","172.21.15.117:0/923237697":"2024-03-05T15:40:22.691924+0000","172.21.15.117:6800/2865911351":"2024-03-05T15:50:35.195755+0000","172.21.15.117:6801/1454790014":"2024-03-05T15:40:22.691924+0000","172.21.15.117:6801/2865911351":"2024-03-05T15:50:35.195755+0000","172.21.15.117:0/1913859421":"2024-03-05T15:50:35.195755+0000","172.21.15.117:0/1102112426":"2024-03-05T15:50:35.195755+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T15:51:22.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.719+0000 7f7b8b3bd700 1 -- 172.21.15.117:0/4004013348 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f7b7006d780 msgr2=0x7f7b7006fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:22.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.719+0000 7f7b8b3bd700 1 --2- 172.21.15.117:0/4004013348 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f7b7006d780 0x7f7b7006fc40 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7b80009680 tx=0x7f7b80009260 comp rx=0 tx=0).stop 2024-03-04T15:51:22.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.719+0000 7f7b8b3bd700 1 -- 172.21.15.117:0/4004013348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b84108dd0 msgr2=0x7f7b8419e640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:22.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.719+0000 7f7b8b3bd700 1 --2- 172.21.15.117:0/4004013348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b84108dd0 0x7f7b8419e640 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f7b74000c00 tx=0x7f7b740047b0 comp rx=0 tx=0).stop 2024-03-04T15:51:22.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.719+0000 7f7b8b3bd700 1 -- 172.21.15.117:0/4004013348 shutdown_connections 2024-03-04T15:51:22.723 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.719+0000 7f7b8b3bd700 1 --2- 172.21.15.117:0/4004013348 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7b8410e4a0 0x7f7b841a29d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:22.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.719+0000 7f7b8b3bd700 1 --2- 172.21.15.117:0/4004013348 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7b84108410 0x7f7b8419e100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:22.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.719+0000 7f7b8b3bd700 1 --2- 172.21.15.117:0/4004013348 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f7b7006d780 0x7f7b7006fc40 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:22.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.719+0000 7f7b8b3bd700 1 --2- 172.21.15.117:0/4004013348 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7b84108dd0 0x7f7b8419e640 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:22.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.723+0000 7f7b8b3bd700 1 -- 172.21.15.117:0/4004013348 >> 172.21.15.117:0/4004013348 conn(0x7f7b840755b0 msgr2=0x7f7b84075990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:22.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.723+0000 7f7b8b3bd700 1 -- 172.21.15.117:0/4004013348 shutdown_connections 2024-03-04T15:51:22.724 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:22.723+0000 7f7b8b3bd700 1 -- 172.21.15.117:0/4004013348 wait complete. 2024-03-04T15:51:23.417 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:23 smithi117 bash[21480]: cluster 2024-03-04T15:51:21.343107+0000 mgr.y (mgr.24428) 53 : cluster 0 pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:23.417 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:23 smithi117 bash[21480]: audit 2024-03-04T15:51:22.720547+0000 mon.a (mon.0) 814 : audit 0 from='client.? 172.21.15.117:0/4004013348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T15:51:23.417 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:23 smithi117 bash[28843]: cluster 2024-03-04T15:51:21.343107+0000 mgr.y (mgr.24428) 53 : cluster 0 pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:23.417 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:23 smithi117 bash[28843]: audit 2024-03-04T15:51:22.720547+0000 mon.a (mon.0) 814 : audit 0 from='client.? 172.21.15.117:0/4004013348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T15:51:23.418 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-03-04T15:51:23.419 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph osd dump --format=json 2024-03-04T15:51:23.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:23 smithi151 bash[23756]: cluster 2024-03-04T15:51:21.343107+0000 mgr.y (mgr.24428) 53 : cluster 0 pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:23.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:23 smithi151 bash[23756]: audit 2024-03-04T15:51:22.720547+0000 mon.a (mon.0) 814 : audit 0 from='client.? 172.21.15.117:0/4004013348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T15:51:25.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:25 smithi151 bash[23756]: cluster 2024-03-04T15:51:23.343876+0000 mgr.y (mgr.24428) 54 : cluster 0 pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:25.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:25 smithi117 bash[21480]: cluster 2024-03-04T15:51:23.343876+0000 mgr.y (mgr.24428) 54 : cluster 0 pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:25.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:25 smithi117 bash[28843]: cluster 2024-03-04T15:51:23.343876+0000 mgr.y (mgr.24428) 54 : cluster 0 pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:26.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:51:26 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:51:27.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:27 smithi151 bash[23756]: cluster 2024-03-04T15:51:25.344625+0000 mgr.y (mgr.24428) 55 : cluster 0 pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:27.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:27 smithi117 bash[21480]: cluster 2024-03-04T15:51:25.344625+0000 mgr.y (mgr.24428) 55 : cluster 0 pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:27.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:27 smithi117 bash[28843]: cluster 2024-03-04T15:51:25.344625+0000 mgr.y (mgr.24428) 55 : cluster 0 pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:28.210 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:51:28.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:28 smithi151 bash[23756]: audit 2024-03-04T15:51:26.323416+0000 mgr.y (mgr.24428) 56 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:51:28.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:28 smithi117 bash[28843]: audit 2024-03-04T15:51:26.323416+0000 mgr.y (mgr.24428) 56 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:51:28.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:28 smithi117 bash[21480]: audit 2024-03-04T15:51:26.323416+0000 mgr.y (mgr.24428) 56 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:51:29.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:29 smithi117 bash[28843]: cluster 2024-03-04T15:51:27.346059+0000 mgr.y (mgr.24428) 57 : cluster 0 pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:29.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:29 smithi117 bash[21480]: cluster 2024-03-04T15:51:27.346059+0000 mgr.y (mgr.24428) 57 : cluster 0 pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:29.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:29 smithi151 bash[23756]: cluster 2024-03-04T15:51:27.346059+0000 mgr.y (mgr.24428) 57 : cluster 0 pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:30.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fc000ffa700 1 -- 172.21.15.117:0/4180109937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbffc101ec0 msgr2=0x7fbffc102340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:30.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fc000ffa700 1 --2- 172.21.15.117:0/4180109937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbffc101ec0 0x7fbffc102340 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fbfec0098f0 tx=0x7fbfec009c00 comp rx=0 tx=0).stop 2024-03-04T15:51:30.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fc000ffa700 1 -- 172.21.15.117:0/4180109937 shutdown_connections 2024-03-04T15:51:30.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fc000ffa700 1 --2- 172.21.15.117:0/4180109937 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbffc102880 0x7fbffc10ec70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:30.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fc000ffa700 1 --2- 172.21.15.117:0/4180109937 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbffc107f30 0x7fbffc108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:30.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fc000ffa700 1 --2- 172.21.15.117:0/4180109937 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbffc101ec0 0x7fbffc102340 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:30.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fc000ffa700 1 -- 172.21.15.117:0/4180109937 >> 172.21.15.117:0/4180109937 conn(0x7fbffc078eb0 msgr2=0x7fbffc0792c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:30.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fc000ffa700 1 -- 172.21.15.117:0/4180109937 shutdown_connections 2024-03-04T15:51:30.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fc000ffa700 1 -- 172.21.15.117:0/4180109937 wait complete. 2024-03-04T15:51:30.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fc000ffa700 1 Processor -- start 2024-03-04T15:51:30.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fc000ffa700 1 -- start start 2024-03-04T15:51:30.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fc000ffa700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbffc101ec0 0x7fbffc19d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:30.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fc000ffa700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbffc102880 0x7fbffc19dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:30.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fc000ffa700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbffc107f30 0x7fbffc197a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:30.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fc000ffa700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc1116e0 con 0x7fbffc101ec0 2024-03-04T15:51:30.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fc000ffa700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc111560 con 0x7fbffc102880 2024-03-04T15:51:30.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fc000ffa700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbffc111860 con 0x7fbffc107f30 2024-03-04T15:51:30.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fbffa59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbffc101ec0 0x7fbffc19d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:30.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fbffad9d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbffc107f30 0x7fbffc197a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:30.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fbffa59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbffc101ec0 0x7fbffc19d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:49844/0 (socket says 172.21.15.117:49844) 2024-03-04T15:51:30.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fbffa59c700 1 -- 172.21.15.117:0/334984664 learned_addr learned my addr 172.21.15.117:0/334984664 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:51:30.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fbffa59c700 1 -- 172.21.15.117:0/334984664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbffc107f30 msgr2=0x7fbffc197a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:30.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fbffa59c700 1 --2- 172.21.15.117:0/334984664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbffc107f30 0x7fbffc197a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:30.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fbff9d9b700 1 --2- 172.21.15.117:0/334984664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbffc102880 0x7fbffc19dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:30.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fbffa59c700 1 -- 172.21.15.117:0/334984664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbffc102880 msgr2=0x7fbffc19dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:30.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fbffa59c700 1 --2- 172.21.15.117:0/334984664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbffc102880 0x7fbffc19dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:30.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fbffa59c700 1 -- 172.21.15.117:0/334984664 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbffc198160 con 0x7fbffc101ec0 2024-03-04T15:51:30.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fbffad9d700 1 --2- 172.21.15.117:0/334984664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbffc107f30 0x7fbffc197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:51:30.732 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fbffa59c700 1 --2- 172.21.15.117:0/334984664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbffc101ec0 0x7fbffc19d840 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbff000e9c0 tx=0x7fbff000ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:30.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fbfeb7fe700 1 -- 172.21.15.117:0/334984664 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff000cbd0 con 0x7fbffc101ec0 2024-03-04T15:51:30.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fc000ffa700 1 -- 172.21.15.117:0/334984664 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbffc198450 con 0x7fbffc101ec0 2024-03-04T15:51:30.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fbfeb7fe700 1 -- 172.21.15.117:0/334984664 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fbff000cd70 con 0x7fbffc101ec0 2024-03-04T15:51:30.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fbfeb7fe700 1 -- 172.21.15.117:0/334984664 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbff00054a0 con 0x7fbffc101ec0 2024-03-04T15:51:30.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.727+0000 7fc000ffa700 1 -- 172.21.15.117:0/334984664 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbffc1a4710 con 0x7fbffc101ec0 2024-03-04T15:51:30.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.731+0000 7fc000ffa700 1 -- 172.21.15.117:0/334984664 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbffc04faa0 con 0x7fbffc101ec0 2024-03-04T15:51:30.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.731+0000 7fbfeb7fe700 1 -- 172.21.15.117:0/334984664 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7fbff0004750 con 0x7fbffc101ec0 2024-03-04T15:51:30.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.731+0000 7fbfeb7fe700 1 --2- 172.21.15.117:0/334984664 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fbfe406d780 0x7fbfe406fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:30.738 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.731+0000 7fbfeb7fe700 1 -- 172.21.15.117:0/334984664 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7fbff008dd10 con 0x7fbffc101ec0 2024-03-04T15:51:30.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.731+0000 7fbff9d9b700 1 --2- 172.21.15.117:0/334984664 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fbfe406d780 0x7fbfe406fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:30.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.731+0000 7fbff9d9b700 1 --2- 172.21.15.117:0/334984664 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fbfe406d780 0x7fbfe406fc40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009230 tx=0x7fbfec005b40 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:30.739 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.735+0000 7fbfeb7fe700 1 -- 172.21.15.117:0/334984664 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fbff005b560 con 0x7fbffc101ec0 2024-03-04T15:51:30.941 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.939+0000 7fc000ffa700 1 -- 172.21.15.117:0/334984664 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbffc198f30 con 0x7fbffc101ec0 2024-03-04T15:51:30.945 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.943+0000 7fbfeb7fe700 1 -- 172.21.15.117:0/334984664 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+23286 (secure 0 0 0) 0x7fbff005efb0 con 0x7fbffc101ec0 2024-03-04T15:51:30.945 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-04T15:51:30.945 INFO:teuthology.orchestra.run.smithi117.stdout:{"epoch":75,"fsid":"e40b5504-da3c-11ee-95c6-87774f69a715","created":"2024-03-04T15:39:15.691654+0000","modified":"2024-03-04T15:50:35.195886+0000","last_up_change":"2024-03-04T15:48:54.249045+0000","last_in_change":"2024-03-04T15:48:27.158859+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-03-04T15:44:31.324972+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":7.8899998664855957,"score_stable":7.8899998664855957,"optimal_score":0.37999999523162842,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-03-04T15:49:32.345716+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"62","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.log","create_time":"2024-03-04T15:49:34.290955+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"64","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.control","create_time":"2024-03-04T15:49:36.207393+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"datapool","create_time":"2024-03-04T15:49:36.958754+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"71","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":71,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.6500000953674316,"score_stable":2.6500000953674316,"optimal_score":0.87999999523162842,"raw_score_acting":2.3299999237060547,"raw_score_stable":2.3299999237060547,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-03-04T15:49:38.730661+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"68","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"cf1f40a0-bf1a-4681-84de-fbf6fee19de7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6802","nonce":1332942120},{"type":"v1","addr":"172.21.15.117:6803","nonce":1332942120}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6804","nonce":1332942120},{"type":"v1","addr":"172.21.15.117:6805","nonce":1332942120}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6808","nonce":1332942120},{"type":"v1","addr":"172.21.15.117:6809","nonce":1332942120}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6806","nonce":1332942120},{"type":"v1","addr":"172.21.15.117:6807","nonce":1332942120}]},"public_addr":"172.21.15.117:6803/1332942120","cluster_addr":"172.21.15.117:6805/1332942120","heartbeat_back_addr":"172.21.15.117:6809/1332942120","heartbeat_front_addr":"172.21.15.117:6807/1332942120","state":["exists","up"]},{"osd":1,"uuid":"9e7ba793-aa0e-4435-bfd6-1bf6d28da6ff","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6810","nonce":444026972},{"type":"v1","addr":"172.21.15.117:6811","nonce":444026972}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6812","nonce":444026972},{"type":"v1","addr":"172.21.15.117:6813","nonce":444026972}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6816","nonce":444026972},{"type":"v1","addr":"172.21.15.117:6817","nonce":444026972}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6814","nonce":444026972},{"type":"v1","addr":"172.21.15.117:6815","nonce":444026972}]},"public_addr":"172.21.15.117:6811/444026972","cluster_addr":"172.21.15.117:6813/444026972","heartbeat_back_addr":"172.21.15.117:6817/444026972","heartbeat_front_addr":"172.21.15.117:6815/444026972","state":["exists","up"]},{"osd":2,"uuid":"49710dbf-78b9-4983-bc61-2a5d0527e417","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6818","nonce":4102123427},{"type":"v1","addr":"172.21.15.117:6819","nonce":4102123427}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6820","nonce":4102123427},{"type":"v1","addr":"172.21.15.117:6821","nonce":4102123427}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6824","nonce":4102123427},{"type":"v1","addr":"172.21.15.117:6825","nonce":4102123427}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6822","nonce":4102123427},{"type":"v1","addr":"172.21.15.117:6823","nonce":4102123427}]},"public_addr":"172.21.15.117:6819/4102123427","cluster_addr":"172.21.15.117:6821/4102123427","heartbeat_back_addr":"172.21.15.117:6825/4102123427","heartbeat_front_addr":"172.21.15.117:6823/4102123427","state":["exists","up"]},{"osd":3,"uuid":"1e1fa658-d1a9-4129-ac7e-18f317b3b318","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6826","nonce":3320151079},{"type":"v1","addr":"172.21.15.117:6827","nonce":3320151079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6828","nonce":3320151079},{"type":"v1","addr":"172.21.15.117:6829","nonce":3320151079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6832","nonce":3320151079},{"type":"v1","addr":"172.21.15.117:6833","nonce":3320151079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.117:6830","nonce":3320151079},{"type":"v1","addr":"172.21.15.117:6831","nonce":3320151079}]},"public_addr":"172.21.15.117:6827/3320151079","cluster_addr":"172.21.15.117:6829/3320151079","heartbeat_back_addr":"172.21.15.117:6833/3320151079","heartbeat_front_addr":"172.21.15.117:6831/3320151079","state":["exists","up"]},{"osd":4,"uuid":"35746d23-4625-45a9-81cd-c05f7ea5a98c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6800","nonce":1293012079},{"type":"v1","addr":"172.21.15.151:6801","nonce":1293012079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6802","nonce":1293012079},{"type":"v1","addr":"172.21.15.151:6803","nonce":1293012079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6806","nonce":1293012079},{"type":"v1","addr":"172.21.15.151:6807","nonce":1293012079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6804","nonce":1293012079},{"type":"v1","addr":"172.21.15.151:6805","nonce":1293012079}]},"public_addr":"172.21.15.151:6801/1293012079","cluster_addr":"172.21.15.151:6803/1293012079","heartbeat_back_addr":"172.21.15.151:6807/1293012079","heartbeat_front_addr":"172.21.15.151:6805/1293012079","state":["exists","up"]},{"osd":5,"uuid":"9660e52c-72e0-4f9c-8cf5-bf5868080e5b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":39,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6808","nonce":3277640682},{"type":"v1","addr":"172.21.15.151:6809","nonce":3277640682}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6810","nonce":3277640682},{"type":"v1","addr":"172.21.15.151:6811","nonce":3277640682}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6814","nonce":3277640682},{"type":"v1","addr":"172.21.15.151:6815","nonce":3277640682}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6812","nonce":3277640682},{"type":"v1","addr":"172.21.15.151:6813","nonce":3277640682}]},"public_addr":"172.21.15.151:6809/3277640682","cluster_addr":"172.21.15.151:6811/3277640682","heartbeat_back_addr":"172.21.15.151:6815/3277640682","heartbeat_front_addr":"172.21.15.151:6813/3277640682","state":["exists","up"]},{"osd":6,"uuid":"706cd130-5cc0-41f3-9e78-85922c29a01c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":64,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6816","nonce":3501440615},{"type":"v1","addr":"172.21.15.151:6817","nonce":3501440615}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6818","nonce":3501440615},{"type":"v1","addr":"172.21.15.151:6819","nonce":3501440615}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6822","nonce":3501440615},{"type":"v1","addr":"172.21.15.151:6823","nonce":3501440615}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6820","nonce":3501440615},{"type":"v1","addr":"172.21.15.151:6821","nonce":3501440615}]},"public_addr":"172.21.15.151:6817/3501440615","cluster_addr":"172.21.15.151:6819/3501440615","heartbeat_back_addr":"172.21.15.151:6823/3501440615","heartbeat_front_addr":"172.21.15.151:6821/3501440615","state":["exists","up"]},{"osd":7,"uuid":"0f49ea27-0d78-4e68-8d06-b3c565e37a58","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6824","nonce":3965450917},{"type":"v1","addr":"172.21.15.151:6825","nonce":3965450917}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6826","nonce":3965450917},{"type":"v1","addr":"172.21.15.151:6827","nonce":3965450917}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6830","nonce":3965450917},{"type":"v1","addr":"172.21.15.151:6831","nonce":3965450917}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.151:6828","nonce":3965450917},{"type":"v1","addr":"172.21.15.151:6829","nonce":3965450917}]},"public_addr":"172.21.15.151:6825/3965450917","cluster_addr":"172.21.15.151:6827/3965450917","heartbeat_back_addr":"172.21.15.151:6831/3965450917","heartbeat_front_addr":"172.21.15.151:6829/3965450917","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:42:47.269066+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:43:37.699556+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:44:28.025013+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:45:20.549377+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:46:11.142621+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:47:04.620036+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:47:58.871589+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-03-04T15:48:52.475968+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.1f","mappings":[{"from":7,"to":2}]},{"pgid":"4.2","mappings":[{"from":1,"to":2}]},{"pgid":"4.12","mappings":[{"from":1,"to":5}]},{"pgid":"4.1b","mappings":[{"from":1,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.117:0/2488053720":"2024-03-05T15:39:47.643279+0000","172.21.15.117:6801/427955642":"2024-03-05T15:39:47.643279+0000","172.21.15.117:0/2167394439":"2024-03-05T15:39:47.643279+0000","172.21.15.117:6800/427955642":"2024-03-05T15:39:47.643279+0000","172.21.15.117:6800/1454790014":"2024-03-05T15:40:22.691924+0000","172.21.15.117:0/2490993128":"2024-03-05T15:50:35.195755+0000","172.21.15.117:0/163360369":"2024-03-05T15:40:22.691924+0000","172.21.15.117:0/4187340781":"2024-03-05T15:39:47.643279+0000","172.21.15.117:0/546245806":"2024-03-05T15:40:22.691924+0000","172.21.15.117:0/2279801001":"2024-03-05T15:50:35.195755+0000","172.21.15.117:0/2351166229":"2024-03-05T15:50:35.195755+0000","172.21.15.117:0/923237697":"2024-03-05T15:40:22.691924+0000","172.21.15.117:6800/2865911351":"2024-03-05T15:50:35.195755+0000","172.21.15.117:6801/1454790014":"2024-03-05T15:40:22.691924+0000","172.21.15.117:6801/2865911351":"2024-03-05T15:50:35.195755+0000","172.21.15.117:0/1913859421":"2024-03-05T15:50:35.195755+0000","172.21.15.117:0/1102112426":"2024-03-05T15:50:35.195755+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-03-04T15:51:30.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.947+0000 7fc000ffa700 1 -- 172.21.15.117:0/334984664 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fbfe406d780 msgr2=0x7fbfe406fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:30.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.947+0000 7fc000ffa700 1 --2- 172.21.15.117:0/334984664 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fbfe406d780 0x7fbfe406fc40 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbfec009230 tx=0x7fbfec005b40 comp rx=0 tx=0).stop 2024-03-04T15:51:30.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.947+0000 7fc000ffa700 1 -- 172.21.15.117:0/334984664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbffc101ec0 msgr2=0x7fbffc19d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:30.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.947+0000 7fc000ffa700 1 --2- 172.21.15.117:0/334984664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbffc101ec0 0x7fbffc19d840 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fbff000e9c0 tx=0x7fbff000ecd0 comp rx=0 tx=0).stop 2024-03-04T15:51:30.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.947+0000 7fc000ffa700 1 -- 172.21.15.117:0/334984664 shutdown_connections 2024-03-04T15:51:30.947 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.947+0000 7fc000ffa700 1 --2- 172.21.15.117:0/334984664 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fbfe406d780 0x7fbfe406fc40 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:30.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.947+0000 7fc000ffa700 1 --2- 172.21.15.117:0/334984664 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fbffc101ec0 0x7fbffc19d840 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:30.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.947+0000 7fc000ffa700 1 --2- 172.21.15.117:0/334984664 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fbffc107f30 0x7fbffc197a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:30.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.947+0000 7fc000ffa700 1 --2- 172.21.15.117:0/334984664 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fbffc102880 0x7fbffc19dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:30.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.947+0000 7fc000ffa700 1 -- 172.21.15.117:0/334984664 >> 172.21.15.117:0/334984664 conn(0x7fbffc078eb0 msgr2=0x7fbffc100c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:30.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.947+0000 7fc000ffa700 1 -- 172.21.15.117:0/334984664 shutdown_connections 2024-03-04T15:51:30.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:30.947+0000 7fc000ffa700 1 -- 172.21.15.117:0/334984664 wait complete. 2024-03-04T15:51:31.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:31 smithi117 bash[21480]: cluster 2024-03-04T15:51:29.346619+0000 mgr.y (mgr.24428) 58 : cluster 0 pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:31.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:31 smithi117 bash[21480]: audit 2024-03-04T15:51:30.943222+0000 mon.a (mon.0) 815 : audit 0 from='client.? 172.21.15.117:0/334984664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T15:51:31.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:31 smithi117 bash[28843]: cluster 2024-03-04T15:51:29.346619+0000 mgr.y (mgr.24428) 58 : cluster 0 pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:31.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:31 smithi117 bash[28843]: audit 2024-03-04T15:51:30.943222+0000 mon.a (mon.0) 815 : audit 0 from='client.? 172.21.15.117:0/334984664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T15:51:31.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:31 smithi151 bash[23756]: cluster 2024-03-04T15:51:29.346619+0000 mgr.y (mgr.24428) 58 : cluster 0 pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:31.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:31 smithi151 bash[23756]: audit 2024-03-04T15:51:30.943222+0000 mon.a (mon.0) 815 : audit 0 from='client.? 172.21.15.117:0/334984664' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-03-04T15:51:31.587 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2024-03-04T15:51:31.587 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2024-03-04T15:51:31.588 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2024-03-04T15:51:31.588 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2024-03-04T15:51:31.588 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2024-03-04T15:51:31.589 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2024-03-04T15:51:31.589 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2024-03-04T15:51:31.590 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2024-03-04T15:51:33.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:33 smithi151 bash[23756]: cluster 2024-03-04T15:51:31.347958+0000 mgr.y (mgr.24428) 59 : cluster 0 pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:33.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:33 smithi117 bash[21480]: cluster 2024-03-04T15:51:31.347958+0000 mgr.y (mgr.24428) 59 : cluster 0 pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:33.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:33 smithi117 bash[28843]: cluster 2024-03-04T15:51:31.347958+0000 mgr.y (mgr.24428) 59 : cluster 0 pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:34.995 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:34 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:34.995 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 15:51:34 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:34.996 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 15:51:34 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:34.996 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 15:51:34 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:34.996 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:51:34 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:34.997 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:51:34 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:34.997 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 15:51:34 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:34.997 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:51:34 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:34.997 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:51:34 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:35.431 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 15:51:35 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:35.432 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 15:51:35 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:35.432 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 15:51:35 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:35.432 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:51:35 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:35.436 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:34 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:35.436 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:34 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:35.436 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:34 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:35.436 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:35 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:35.436 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:35 smithi151 systemd[1]: Started Ceph grafana.a for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T15:51:35.436 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 15:51:35 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:35.437 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 15:51:35 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:35.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:35 smithi151 bash[23756]: cluster 2024-03-04T15:51:33.348792+0000 mgr.y (mgr.24428) 60 : cluster 0 pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:35.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:35 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:35.437 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:51:35 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:35.438 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 15:51:35 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T15:51:35.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:35 smithi117 bash[21480]: cluster 2024-03-04T15:51:33.348792+0000 mgr.y (mgr.24428) 60 : cluster 0 pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:35.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:35 smithi117 bash[28843]: cluster 2024-03-04T15:51:33.348792+0000 mgr.y (mgr.24428) 60 : cluster 0 pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:36.481 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:51:36.483 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:51:36.483 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:51:36.483 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:51:36.485 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:51:36.486 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:51:36.489 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:51:36.490 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:51:36.510 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:51:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:51:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:51:36.511 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:36 smithi117 bash[21480]: audit 2024-03-04T15:51:35.321385+0000 mon.a (mon.0) 816 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:36.512 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:36 smithi117 bash[21480]: audit 2024-03-04T15:51:35.337038+0000 mon.a (mon.0) 817 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:36.512 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:36 smithi117 bash[21480]: cluster 2024-03-04T15:51:35.349516+0000 mgr.y (mgr.24428) 61 : cluster 0 pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:36.512 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:36 smithi117 bash[21480]: audit 2024-03-04T15:51:35.372182+0000 mon.a (mon.0) 818 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:36.512 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:36 smithi117 bash[21480]: audit 2024-03-04T15:51:35.384431+0000 mon.a (mon.0) 819 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:36.512 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:36 smithi117 bash[21480]: audit 2024-03-04T15:51:35.448050+0000 mon.c (mon.2) 57 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:51:36.512 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:36 smithi117 bash[21480]: audit 2024-03-04T15:51:35.457285+0000 mon.c (mon.2) 58 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:51:36.512 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:36 smithi117 bash[21480]: audit 2024-03-04T15:51:35.941205+0000 mon.a (mon.0) 820 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:36.512 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:36 smithi117 bash[21480]: audit 2024-03-04T15:51:35.953680+0000 mon.a (mon.0) 821 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:36.513 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:36 smithi117 bash[28843]: audit 2024-03-04T15:51:35.321385+0000 mon.a (mon.0) 816 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:36.513 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:36 smithi117 bash[28843]: audit 2024-03-04T15:51:35.337038+0000 mon.a (mon.0) 817 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:36.513 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:36 smithi117 bash[28843]: cluster 2024-03-04T15:51:35.349516+0000 mgr.y (mgr.24428) 61 : cluster 0 pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:36.513 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:36 smithi117 bash[28843]: audit 2024-03-04T15:51:35.372182+0000 mon.a (mon.0) 818 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:36.513 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:36 smithi117 bash[28843]: audit 2024-03-04T15:51:35.384431+0000 mon.a (mon.0) 819 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:36.514 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:36 smithi117 bash[28843]: audit 2024-03-04T15:51:35.448050+0000 mon.c (mon.2) 57 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:51:36.514 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:36 smithi117 bash[28843]: audit 2024-03-04T15:51:35.457285+0000 mon.c (mon.2) 58 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:51:36.514 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:36 smithi117 bash[28843]: audit 2024-03-04T15:51:35.941205+0000 mon.a (mon.0) 820 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:36.514 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:36 smithi117 bash[28843]: audit 2024-03-04T15:51:35.953680+0000 mon.a (mon.0) 821 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:36.617 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:51:36.618 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[23756]: audit 2024-03-04T15:51:35.321385+0000 mon.a (mon.0) 816 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:36.618 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[23756]: audit 2024-03-04T15:51:35.337038+0000 mon.a (mon.0) 817 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:36.618 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[23756]: cluster 2024-03-04T15:51:35.349516+0000 mgr.y (mgr.24428) 61 : cluster 0 pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:36.618 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[23756]: audit 2024-03-04T15:51:35.372182+0000 mon.a (mon.0) 818 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:36.618 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[23756]: audit 2024-03-04T15:51:35.384431+0000 mon.a (mon.0) 819 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:36.618 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[23756]: audit 2024-03-04T15:51:35.448050+0000 mon.c (mon.2) 57 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:51:36.618 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[23756]: audit 2024-03-04T15:51:35.457285+0000 mon.c (mon.2) 58 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:51:36.618 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[23756]: audit 2024-03-04T15:51:35.941205+0000 mon.a (mon.0) 820 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:36.618 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[23756]: audit 2024-03-04T15:51:35.953680+0000 mon.a (mon.0) 821 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:36.880 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=settings t=2024-03-04T15:51:36.616674574Z level=info msg="Starting Grafana" version=9.4.12 commit=37ee25ba46 branch=HEAD compiled=2023-05-22T15:58:05Z 2024-03-04T15:51:36.880 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=settings t=2024-03-04T15:51:36.616909777Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-03-04T15:51:36.880 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=settings t=2024-03-04T15:51:36.616930619Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-03-04T15:51:36.880 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=settings t=2024-03-04T15:51:36.616938849Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-03-04T15:51:36.881 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=settings t=2024-03-04T15:51:36.616950427Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-03-04T15:51:36.881 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=settings t=2024-03-04T15:51:36.616961602Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-03-04T15:51:36.881 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=settings t=2024-03-04T15:51:36.616972729Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-03-04T15:51:36.881 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=settings t=2024-03-04T15:51:36.616994912Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-03-04T15:51:36.881 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=settings t=2024-03-04T15:51:36.617006467Z level=info msg="Path Home" path=/usr/share/grafana 2024-03-04T15:51:36.881 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=settings t=2024-03-04T15:51:36.617018908Z level=info msg="Path Data" path=/var/lib/grafana 2024-03-04T15:51:36.881 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=settings t=2024-03-04T15:51:36.617029997Z level=info msg="Path Logs" path=/var/log/grafana 2024-03-04T15:51:36.881 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=settings t=2024-03-04T15:51:36.617040648Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-03-04T15:51:36.881 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=settings t=2024-03-04T15:51:36.617051506Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-03-04T15:51:36.881 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=settings t=2024-03-04T15:51:36.617062927Z level=info msg="App mode production" 2024-03-04T15:51:36.881 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=sqlstore t=2024-03-04T15:51:36.617147946Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-03-04T15:51:36.882 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=sqlstore t=2024-03-04T15:51:36.617178293Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-03-04T15:51:36.882 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:36.61823582Z level=info msg="Starting DB migrations" 2024-03-04T15:51:36.882 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:36.618335704Z level=info msg="Executing migration" id="create migration_log table" 2024-03-04T15:51:36.882 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:36.651687448Z level=info msg="Executing migration" id="create user table" 2024-03-04T15:51:36.882 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:36.693816123Z level=info msg="Executing migration" id="add unique index user.login" 2024-03-04T15:51:36.882 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:36.735825094Z level=info msg="Executing migration" id="add unique index user.email" 2024-03-04T15:51:36.882 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:36.761356632Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-03-04T15:51:36.882 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:36.786629163Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-03-04T15:51:36.882 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:36.812019261Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-03-04T15:51:36.882 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:36.85403154Z level=info msg="Executing migration" id="create user table v2" 2024-03-04T15:51:36.882 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:36.879559712Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-03-04T15:51:37.133 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:36.921610514Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-03-04T15:51:37.133 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:36.963826571Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-03-04T15:51:37.133 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:36 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:36.989221003Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-03-04T15:51:37.133 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.01455886Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-03-04T15:51:37.133 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.039850375Z level=info msg="Executing migration" id="Update user table charset" 2024-03-04T15:51:37.133 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.065197833Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-03-04T15:51:37.133 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.107441339Z level=info msg="Executing migration" id="Add missing user data" 2024-03-04T15:51:37.133 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.132865866Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-03-04T15:51:37.402 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[23756]: audit 2024-03-04T15:51:36.333058+0000 mgr.y (mgr.24428) 62 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:51:37.402 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.158160891Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-03-04T15:51:37.402 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.183465599Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-03-04T15:51:37.403 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.225497722Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-03-04T15:51:37.403 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.267687425Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-03-04T15:51:37.403 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.309818699Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-03-04T15:51:37.403 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.351763019Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-03-04T15:51:37.403 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.37712573Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-03-04T15:51:37.403 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.402455185Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-03-04T15:51:37.625 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:37 smithi117 bash[21480]: audit 2024-03-04T15:51:36.333058+0000 mgr.y (mgr.24428) 62 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:51:37.626 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:37 smithi117 bash[28843]: audit 2024-03-04T15:51:36.333058+0000 mgr.y (mgr.24428) 62 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:51:37.680 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.427520452Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-03-04T15:51:37.680 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.45283946Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-03-04T15:51:37.681 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.478398717Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-03-04T15:51:37.681 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.503703702Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-03-04T15:51:37.681 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.529015951Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-03-04T15:51:37.681 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.571049259Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-03-04T15:51:37.681 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.613193353Z level=info msg="Executing migration" id="create temp_user v2" 2024-03-04T15:51:37.681 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.63844515Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-03-04T15:51:37.681 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.680521038Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-03-04T15:51:37.931 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.714179218Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-03-04T15:51:37.932 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.73953129Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-03-04T15:51:37.932 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.764865942Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-03-04T15:51:37.932 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.790045138Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-03-04T15:51:37.932 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.832065207Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-03-04T15:51:37.932 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.882869952Z level=info msg="Executing migration" id="create star table" 2024-03-04T15:51:37.932 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.908136751Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-03-04T15:51:38.212 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.950342967Z level=info msg="Executing migration" id="create org table v1" 2024-03-04T15:51:38.212 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:37 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:37.967258794Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-03-04T15:51:38.212 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.026028883Z level=info msg="Executing migration" id="create org_user table v1" 2024-03-04T15:51:38.212 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.068257231Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-03-04T15:51:38.212 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.093606558Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-03-04T15:51:38.212 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.118967618Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-03-04T15:51:38.212 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.144277527Z level=info msg="Executing migration" id="Update org table charset" 2024-03-04T15:51:38.212 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.186407342Z level=info msg="Executing migration" id="Update org_user table charset" 2024-03-04T15:51:38.213 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.211731307Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-03-04T15:51:38.472 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[23756]: cluster 2024-03-04T15:51:37.350818+0000 mgr.y (mgr.24428) 63 : cluster 0 pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:38.473 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.253901278Z level=info msg="Executing migration" id="create dashboard table" 2024-03-04T15:51:38.473 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.30430612Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-03-04T15:51:38.473 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.37289221Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-03-04T15:51:38.473 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.396943202Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-03-04T15:51:38.473 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.422159053Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-03-04T15:51:38.473 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.447489556Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-03-04T15:51:38.473 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.472654941Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-03-04T15:51:38.709 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:38 smithi117 bash[21480]: cluster 2024-03-04T15:51:37.350818+0000 mgr.y (mgr.24428) 63 : cluster 0 pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:38.710 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:38 smithi117 bash[28843]: cluster 2024-03-04T15:51:37.350818+0000 mgr.y (mgr.24428) 63 : cluster 0 pgmap v34: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:38.742 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.498053543Z level=info msg="Executing migration" id="create dashboard v2" 2024-03-04T15:51:38.743 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.523363255Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-03-04T15:51:38.743 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.548741831Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-03-04T15:51:38.743 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.591060423Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-03-04T15:51:38.743 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.616261211Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-03-04T15:51:38.743 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.650004028Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-03-04T15:51:38.743 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.691953876Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-03-04T15:51:38.743 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.71735724Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-03-04T15:51:38.743 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.742612143Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-03-04T15:51:39.003 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.78481731Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-03-04T15:51:39.004 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.810138799Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-03-04T15:51:39.004 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.85215465Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-03-04T15:51:39.004 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.877465822Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-03-04T15:51:39.004 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.902834327Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-03-04T15:51:39.004 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:38 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:38.953265635Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-03-04T15:51:39.004 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.003754022Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-03-04T15:51:39.282 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.037996486Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-03-04T15:51:39.282 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.087799645Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-03-04T15:51:39.282 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.121595109Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-03-04T15:51:39.282 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.147047156Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-03-04T15:51:39.282 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.172457167Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-03-04T15:51:39.282 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.197710301Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-03-04T15:51:39.282 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.239630777Z level=info msg="Executing migration" id="Update dashboard title length" 2024-03-04T15:51:39.282 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.28193898Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-03-04T15:51:39.560 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.307364806Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-03-04T15:51:39.561 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.349651956Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-03-04T15:51:39.561 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.375112737Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-03-04T15:51:39.561 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.400454652Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-03-04T15:51:39.561 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.442502453Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-03-04T15:51:39.561 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.47614879Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-03-04T15:51:39.561 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.509810358Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-03-04T15:51:39.561 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.560217418Z level=info msg="Executing migration" id="Add check_sum column" 2024-03-04T15:51:39.813 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.593917259Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-03-04T15:51:39.813 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.644370575Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-03-04T15:51:39.813 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.694790238Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-03-04T15:51:39.813 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.745290729Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-03-04T15:51:39.813 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.778930225Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-03-04T15:51:39.813 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.812662053Z level=info msg="Executing migration" id="create data_source table" 2024-03-04T15:51:40.082 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.86301042Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-03-04T15:51:40.082 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.913485611Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-03-04T15:51:40.082 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.955454177Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-03-04T15:51:40.083 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:39 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:39.980760534Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-03-04T15:51:40.083 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.006149069Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-03-04T15:51:40.083 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.031521232Z level=info msg="Executing migration" id="create data_source table v2" 2024-03-04T15:51:40.083 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.056900858Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-03-04T15:51:40.083 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.082227358Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-03-04T15:51:40.335 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.124580456Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-03-04T15:51:40.335 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.149998891Z level=info msg="Executing migration" id="Add column with_credentials" 2024-03-04T15:51:40.335 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.175150571Z level=info msg="Executing migration" id="Add secure json data column" 2024-03-04T15:51:40.335 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.200498384Z level=info msg="Executing migration" id="Update data_source table charset" 2024-03-04T15:51:40.335 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.242489149Z level=info msg="Executing migration" id="Update initial version to 1" 2024-03-04T15:51:40.335 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.284721969Z level=info msg="Executing migration" id="Add read_only data column" 2024-03-04T15:51:40.335 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.335197408Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-03-04T15:51:40.587 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[23756]: cluster 2024-03-04T15:51:39.351488+0000 mgr.y (mgr.24428) 64 : cluster 0 pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:40.588 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[23756]: audit 2024-03-04T15:51:40.374757+0000 mon.a (mon.0) 822 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:40.588 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.385616789Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-03-04T15:51:40.588 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.436344728Z level=info msg="Executing migration" id="Add uid column" 2024-03-04T15:51:40.588 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.486748792Z level=info msg="Executing migration" id="Update uid value" 2024-03-04T15:51:40.588 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.520333957Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-03-04T15:51:40.588 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.554040304Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-03-04T15:51:40.588 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.587747518Z level=info msg="Executing migration" id="create api_key table" 2024-03-04T15:51:40.679 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:40 smithi117 bash[21480]: cluster 2024-03-04T15:51:39.351488+0000 mgr.y (mgr.24428) 64 : cluster 0 pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:40.679 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:40 smithi117 bash[21480]: audit 2024-03-04T15:51:40.374757+0000 mon.a (mon.0) 822 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:40.679 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:40 smithi117 bash[28843]: cluster 2024-03-04T15:51:39.351488+0000 mgr.y (mgr.24428) 64 : cluster 0 pgmap v35: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:40.679 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:40 smithi117 bash[28843]: audit 2024-03-04T15:51:40.374757+0000 mon.a (mon.0) 822 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:40.915 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.621380271Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-03-04T15:51:40.915 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.671648521Z level=info msg="Executing migration" id="add index api_key.key" 2024-03-04T15:51:40.915 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.705492555Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-03-04T15:51:40.916 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.755989188Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-03-04T15:51:40.916 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.781296489Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-03-04T15:51:40.916 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.915386112Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-03-04T15:51:41.168 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.957661291Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-03-04T15:51:41.168 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:40 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:40.99152676Z level=info msg="Executing migration" id="create api_key table v2" 2024-03-04T15:51:41.168 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.016833132Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-03-04T15:51:41.168 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.05891464Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-03-04T15:51:41.169 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.118069366Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-03-04T15:51:41.169 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.143492859Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-03-04T15:51:41.169 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.168478744Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-03-04T15:51:41.431 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.193787954Z level=info msg="Executing migration" id="Update api_key table charset" 2024-03-04T15:51:41.432 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.23595843Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-03-04T15:51:41.432 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.278047325Z level=info msg="Executing migration" id="Add service account foreign key" 2024-03-04T15:51:41.432 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.320207388Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-03-04T15:51:41.432 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.370915926Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-03-04T15:51:41.432 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.404843967Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-03-04T15:51:41.716 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.455319831Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-03-04T15:51:41.716 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.488930244Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-03-04T15:51:41.716 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.522425266Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-03-04T15:51:41.716 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.572976362Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-03-04T15:51:41.717 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.623559505Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-03-04T15:51:41.717 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.665750088Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-03-04T15:51:41.994 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.716697208Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-03-04T15:51:41.994 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.758877667Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-03-04T15:51:41.994 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.801276862Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-03-04T15:51:41.994 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.885058191Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-03-04T15:51:41.994 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.927212028Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-03-04T15:51:41.995 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.952558359Z level=info msg="Executing migration" id="create quota table v1" 2024-03-04T15:51:41.995 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:41 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:41.994597521Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-03-04T15:51:42.273 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.036750927Z level=info msg="Executing migration" id="Update quota table charset" 2024-03-04T15:51:42.273 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.062013962Z level=info msg="Executing migration" id="create plugin_setting table" 2024-03-04T15:51:42.273 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.087471437Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-03-04T15:51:42.273 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.137981768Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-03-04T15:51:42.273 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.163365549Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-03-04T15:51:42.273 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.205378987Z level=info msg="Executing migration" id="create session table" 2024-03-04T15:51:42.273 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.230856372Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-03-04T15:51:42.273 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.272915989Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-03-04T15:51:42.535 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.315085218Z level=info msg="Executing migration" id="create playlist table v2" 2024-03-04T15:51:42.535 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.340354Z level=info msg="Executing migration" id="create playlist item table v2" 2024-03-04T15:51:42.535 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.382493686Z level=info msg="Executing migration" id="Update playlist table charset" 2024-03-04T15:51:42.535 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.407723266Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-03-04T15:51:42.535 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.450122151Z level=info msg="Executing migration" id="drop preferences table v2" 2024-03-04T15:51:42.535 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.467177792Z level=info msg="Executing migration" id="drop preferences table v3" 2024-03-04T15:51:42.535 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.484032912Z level=info msg="Executing migration" id="create preferences table v3" 2024-03-04T15:51:42.535 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.501089892Z level=info msg="Executing migration" id="Update preferences table charset" 2024-03-04T15:51:42.535 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.518018475Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-03-04T15:51:42.535 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.534923408Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-03-04T15:51:42.788 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.577009534Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-03-04T15:51:42.788 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.602326006Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-03-04T15:51:42.788 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.644403333Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-03-04T15:51:42.788 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.686422807Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-03-04T15:51:42.788 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.711748582Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-03-04T15:51:42.788 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.737202798Z level=info msg="Executing migration" id="create alert table v1" 2024-03-04T15:51:42.788 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.762478757Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-03-04T15:51:42.788 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.787732353Z level=info msg="Executing migration" id="add index alert state" 2024-03-04T15:51:43.049 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[23756]: cluster 2024-03-04T15:51:41.352259+0000 mgr.y (mgr.24428) 65 : cluster 0 pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:43.049 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[23756]: audit 2024-03-04T15:51:41.997228+0000 mon.a (mon.0) 823 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:43.049 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[23756]: audit 2024-03-04T15:51:42.010413+0000 mon.a (mon.0) 824 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:43.049 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.812988559Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-03-04T15:51:43.049 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.855206697Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-03-04T15:51:43.049 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.880511759Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-03-04T15:51:43.050 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.905831367Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-03-04T15:51:43.050 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.94770627Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-03-04T15:51:43.050 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:42 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:42.973093837Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-03-04T15:51:43.050 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.015376164Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-03-04T15:51:43.050 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.049040449Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-03-04T15:51:43.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:42 smithi117 bash[28843]: cluster 2024-03-04T15:51:41.352259+0000 mgr.y (mgr.24428) 65 : cluster 0 pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:43.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:42 smithi117 bash[28843]: audit 2024-03-04T15:51:41.997228+0000 mon.a (mon.0) 823 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:43.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:42 smithi117 bash[28843]: audit 2024-03-04T15:51:42.010413+0000 mon.a (mon.0) 824 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:43.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:42 smithi117 bash[21480]: cluster 2024-03-04T15:51:41.352259+0000 mgr.y (mgr.24428) 65 : cluster 0 pgmap v36: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:43.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:42 smithi117 bash[21480]: audit 2024-03-04T15:51:41.997228+0000 mon.a (mon.0) 823 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:43.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:42 smithi117 bash[21480]: audit 2024-03-04T15:51:42.010413+0000 mon.a (mon.0) 824 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:43.335 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.082700935Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-03-04T15:51:43.335 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.133116541Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-03-04T15:51:43.335 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.175448981Z level=info msg="Executing migration" id="Add column is_default" 2024-03-04T15:51:43.335 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.200750797Z level=info msg="Executing migration" id="Add column frequency" 2024-03-04T15:51:43.335 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.226071815Z level=info msg="Executing migration" id="Add column send_reminder" 2024-03-04T15:51:43.336 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.268110631Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-03-04T15:51:43.336 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.293418163Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-03-04T15:51:43.336 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.335474601Z level=info msg="Executing migration" id="Update alert table charset" 2024-03-04T15:51:43.614 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.36077854Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-03-04T15:51:43.614 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.386000497Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-03-04T15:51:43.614 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.411472547Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-03-04T15:51:43.615 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.436806416Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-03-04T15:51:43.615 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.478880967Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-03-04T15:51:43.615 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.504220542Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-03-04T15:51:43.615 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.521288773Z level=info msg="Executing migration" id="Add for to alert table" 2024-03-04T15:51:43.615 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.538228316Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-03-04T15:51:43.615 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.555163122Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-03-04T15:51:43.615 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.572044949Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-03-04T15:51:43.615 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.614237182Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-03-04T15:51:43.867 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.639545334Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-03-04T15:51:43.867 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.681536676Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-03-04T15:51:43.867 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.706847909Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-03-04T15:51:43.868 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.732131536Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-03-04T15:51:43.868 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.774286185Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-03-04T15:51:43.868 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.799722799Z level=info msg="Executing migration" id="create annotation table v5" 2024-03-04T15:51:43.868 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.842111639Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-03-04T15:51:43.868 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.867405185Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-03-04T15:51:44.145 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.892777926Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-03-04T15:51:44.145 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.934726129Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-03-04T15:51:44.145 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:43 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:43.98536176Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-03-04T15:51:44.145 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.035797491Z level=info msg="Executing migration" id="Update annotation table charset" 2024-03-04T15:51:44.145 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.069463335Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-03-04T15:51:44.146 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.103199593Z level=info msg="Executing migration" id="Drop category_id index" 2024-03-04T15:51:44.146 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.14519893Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-03-04T15:51:44.398 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.187261135Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-03-04T15:51:44.398 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.229271085Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-03-04T15:51:44.398 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.271417399Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-03-04T15:51:44.398 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.313483755Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-03-04T15:51:44.398 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.347166438Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-03-04T15:51:44.398 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.372482208Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-03-04T15:51:44.398 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.397868249Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-03-04T15:51:44.651 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.423185186Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-03-04T15:51:44.651 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.465297151Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-03-04T15:51:44.651 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.490584365Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-03-04T15:51:44.652 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.515822459Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-03-04T15:51:44.652 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.541267289Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-03-04T15:51:44.652 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.566656208Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-03-04T15:51:44.652 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.608887157Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-03-04T15:51:44.652 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.651035645Z level=info msg="Executing migration" id="Add epoch_end column" 2024-03-04T15:51:44.903 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.693139303Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-03-04T15:51:44.904 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.735263128Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-03-04T15:51:44.904 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.768864582Z level=info msg="Executing migration" id="Move region to single row" 2024-03-04T15:51:44.904 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.802564248Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-03-04T15:51:44.904 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.852851645Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-03-04T15:51:44.904 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.903427832Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-03-04T15:51:45.156 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[23756]: cluster 2024-03-04T15:51:43.352721+0000 mgr.y (mgr.24428) 66 : cluster 0 pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:45.156 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.953824422Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-03-04T15:51:45.156 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:44 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:44.987510017Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-03-04T15:51:45.156 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.038061185Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-03-04T15:51:45.156 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.071670366Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-03-04T15:51:45.156 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.105437998Z level=info msg="Executing migration" id="create test_data table" 2024-03-04T15:51:45.200 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:44 smithi117 bash[21480]: cluster 2024-03-04T15:51:43.352721+0000 mgr.y (mgr.24428) 66 : cluster 0 pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:45.200 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:44 smithi117 bash[28843]: cluster 2024-03-04T15:51:43.352721+0000 mgr.y (mgr.24428) 66 : cluster 0 pgmap v37: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:45.418 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.155921071Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-03-04T15:51:45.418 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.206310038Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-03-04T15:51:45.418 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.223377167Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-03-04T15:51:45.418 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.256941375Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-03-04T15:51:45.418 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.273977765Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-03-04T15:51:45.418 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.316038748Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-03-04T15:51:45.418 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.341309603Z level=info msg="Executing migration" id="create team table" 2024-03-04T15:51:45.418 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.366688188Z level=info msg="Executing migration" id="add index team.org_id" 2024-03-04T15:51:45.418 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.392382339Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-03-04T15:51:45.418 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.417615584Z level=info msg="Executing migration" id="create team member table" 2024-03-04T15:51:45.681 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.459691904Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-03-04T15:51:45.682 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.485169535Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-03-04T15:51:45.682 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.510473769Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-03-04T15:51:45.682 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.535712883Z level=info msg="Executing migration" id="Add column email to team table" 2024-03-04T15:51:45.682 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.561054033Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-03-04T15:51:45.682 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.603224025Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-03-04T15:51:45.682 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.653595449Z level=info msg="Executing migration" id="create dashboard acl table" 2024-03-04T15:51:45.990 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.729290717Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-03-04T15:51:45.990 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.779773286Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-03-04T15:51:45.990 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.830208963Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-03-04T15:51:45.990 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.863790661Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-03-04T15:51:45.990 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.897611833Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-03-04T15:51:45.990 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:45 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:45.990013284Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-03-04T15:51:46.277 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:46 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:46.040741287Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-03-04T15:51:46.282 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:46 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:46.066069381Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-03-04T15:51:46.282 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:46 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:46.10811863Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-03-04T15:51:46.282 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:46 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:46.133412187Z level=info msg="Executing migration" id="create tag table" 2024-03-04T15:51:46.282 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:46 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:46.175805267Z level=info msg="Executing migration" id="add index tag.key_value" 2024-03-04T15:51:46.283 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:46 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:46.234842945Z level=info msg="Executing migration" id="create login attempt table" 2024-03-04T15:51:46.283 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:46 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:46.276997131Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-03-04T15:51:46.322 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:51:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:51:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:51:46.566 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:51:46 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:51:46.566 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:46 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:46.319229583Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-03-04T15:51:46.566 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:46 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:46.361245365Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-03-04T15:51:46.566 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:46 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:46.403335525Z level=info msg="Executing migration" id="create login_attempt v2" 2024-03-04T15:51:46.566 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:46 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:46.478666577Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-03-04T15:51:46.566 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:46 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:46.512511332Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-03-04T15:51:46.566 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:46 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:46.562952734Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-03-04T15:51:46.886 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:46 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:46.596604078Z level=info msg="Executing migration" id="create user auth table" 2024-03-04T15:51:46.886 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:46 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:46.646991491Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-03-04T15:51:46.886 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:46 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:46.723704075Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-03-04T15:51:46.887 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:46 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:46.781165246Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-03-04T15:51:47.150 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[23756]: cluster 2024-03-04T15:51:45.353350+0000 mgr.y (mgr.24428) 67 : cluster 0 pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:51:47.150 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:46 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:46.886376936Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-03-04T15:51:47.150 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:46 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:46.992073983Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-03-04T15:51:47.151 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.049352198Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-03-04T15:51:47.151 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.099772538Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-03-04T15:51:47.151 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.125084442Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-03-04T15:51:47.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:47 smithi117 bash[28843]: cluster 2024-03-04T15:51:45.353350+0000 mgr.y (mgr.24428) 67 : cluster 0 pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:51:47.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:47 smithi117 bash[21480]: cluster 2024-03-04T15:51:45.353350+0000 mgr.y (mgr.24428) 67 : cluster 0 pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:51:47.420 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.150387045Z level=info msg="Executing migration" id="create server_lock table" 2024-03-04T15:51:47.420 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.201184558Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-03-04T15:51:47.421 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.226570358Z level=info msg="Executing migration" id="create user auth token table" 2024-03-04T15:51:47.421 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.276883546Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-03-04T15:51:47.421 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.302278879Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-03-04T15:51:47.421 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.327614637Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-03-04T15:51:47.421 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.369755321Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-03-04T15:51:47.421 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.395102823Z level=info msg="Executing migration" id="create cache_data table" 2024-03-04T15:51:47.421 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.420416641Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-03-04T15:51:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f8334518700 1 -- 172.21.15.117:0/1658833048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f832c109350 msgr2=0x7f832c110f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f8334518700 1 --2- 172.21.15.117:0/1658833048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f832c109350 0x7f832c110f20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f831c0099c0 tx=0x7f831c009cd0 comp rx=0 tx=0).stop 2024-03-04T15:51:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f8334518700 1 -- 172.21.15.117:0/1658833048 shutdown_connections 2024-03-04T15:51:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f8334518700 1 --2- 172.21.15.117:0/1658833048 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f832c109350 0x7f832c110f20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f8334518700 1 --2- 172.21.15.117:0/1658833048 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f832c104270 0x7f832c104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f8334518700 1 --2- 172.21.15.117:0/1658833048 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f832c104c20 0x7f832c108c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f8334518700 1 -- 172.21.15.117:0/1658833048 >> 172.21.15.117:0/1658833048 conn(0x7f832c0ffb00 msgr2=0x7f832c101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f8334518700 1 -- 172.21.15.117:0/1658833048 shutdown_connections 2024-03-04T15:51:47.606 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f8334518700 1 -- 172.21.15.117:0/1658833048 wait complete. 2024-03-04T15:51:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f8334518700 1 Processor -- start 2024-03-04T15:51:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f8334518700 1 -- start start 2024-03-04T15:51:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f8334518700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f832c104270 0x7f832c199d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f8334518700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f832c104c20 0x7f832c19a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f8334518700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f832c109350 0x7f832c19e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f8334518700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c113800 con 0x7f832c104270 2024-03-04T15:51:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f8334518700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832c113680 con 0x7f832c109350 2024-03-04T15:51:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f8334518700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f832c113980 con 0x7f832c104c20 2024-03-04T15:51:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f83322b4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f832c104270 0x7f832c199d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f8331ab3700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f832c104c20 0x7f832c19a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:47.608 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f83322b4700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f832c104270 0x7f832c199d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51586/0 (socket says 172.21.15.117:51586) 2024-03-04T15:51:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.603+0000 7f83322b4700 1 -- 172.21.15.117:0/2508641198 learned_addr learned my addr 172.21.15.117:0/2508641198 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:51:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f8332ab5700 1 --2- 172.21.15.117:0/2508641198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f832c109350 0x7f832c19e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f83322b4700 1 -- 172.21.15.117:0/2508641198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f832c104c20 msgr2=0x7f832c19a2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f83322b4700 1 --2- 172.21.15.117:0/2508641198 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f832c104c20 0x7f832c19a2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f83322b4700 1 -- 172.21.15.117:0/2508641198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f832c109350 msgr2=0x7f832c19e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f83322b4700 1 --2- 172.21.15.117:0/2508641198 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f832c109350 0x7f832c19e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f83322b4700 1 -- 172.21.15.117:0/2508641198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c19eda0 con 0x7f832c104270 2024-03-04T15:51:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f83322b4700 1 --2- 172.21.15.117:0/2508641198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f832c104270 0x7f832c199d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f832400ef10 tx=0x7f832400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f832b7fe700 1 -- 172.21.15.117:0/2508641198 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832400ca70 con 0x7f832c104270 2024-03-04T15:51:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f8334518700 1 -- 172.21.15.117:0/2508641198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832c19f030 con 0x7f832c104270 2024-03-04T15:51:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f832b7fe700 1 -- 172.21.15.117:0/2508641198 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f8324019070 con 0x7f832c104270 2024-03-04T15:51:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f832b7fe700 1 -- 172.21.15.117:0/2508641198 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8324014480 con 0x7f832c104270 2024-03-04T15:51:47.609 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f8334518700 1 -- 172.21.15.117:0/2508641198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832c1a6910 con 0x7f832c104270 2024-03-04T15:51:47.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f8334518700 1 -- 172.21.15.117:0/2508641198 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f832c100920 con 0x7f832c104270 2024-03-04T15:51:47.610 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f832b7fe700 1 -- 172.21.15.117:0/2508641198 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f8324012050 con 0x7f832c104270 2024-03-04T15:51:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f832b7fe700 1 --2- 172.21.15.117:0/2508641198 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f831406d780 0x7f831406fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f832b7fe700 1 -- 172.21.15.117:0/2508641198 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7f832408e3d0 con 0x7f832c104270 2024-03-04T15:51:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f8331ab3700 1 --2- 172.21.15.117:0/2508641198 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f831406d780 0x7f831406fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f832b7fe700 1 --2- 172.21.15.117:0/2508641198 >> [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427] conn(0x7f8314077710 0x7f8314079bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f832b7fe700 1 -- 172.21.15.117:0/2508641198 --> [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8324025e50 con 0x7f8314077710 2024-03-04T15:51:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f832b7fe700 1 -- 172.21.15.117:0/2508641198 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f832408e860 con 0x7f832c104270 2024-03-04T15:51:47.611 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.607+0000 7f8332ab5700 1 --2- 172.21.15.117:0/2508641198 >> [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427] conn(0x7f8314077710 0x7f8314079bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:47.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.611+0000 7f8331ab3700 1 --2- 172.21.15.117:0/2508641198 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f831406d780 0x7f831406fc40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f832c108cd0 tx=0x7f832000a300 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:47.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.611+0000 7f8332ab5700 1 --2- 172.21.15.117:0/2508641198 >> [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427] conn(0x7f8314077710 0x7f8314079bb0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:47.612 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.611+0000 7f832b7fe700 1 -- 172.21.15.117:0/2508641198 <== osd.2 v2:172.21.15.117:6818/4102123427 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f8324025e50 con 0x7f8314077710 2024-03-04T15:51:47.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.627+0000 7f8334518700 1 -- 172.21.15.117:0/2508641198 --> [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f832c105750 con 0x7f8314077710 2024-03-04T15:51:47.629 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.627+0000 7f832b7fe700 1 -- 172.21.15.117:0/2508641198 <== osd.2 v2:172.21.15.117:6818/4102123427 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f832c105750 con 0x7f8314077710 2024-03-04T15:51:47.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.627+0000 7f8334518700 1 -- 172.21.15.117:0/2508641198 >> [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427] conn(0x7f8314077710 msgr2=0x7f8314079bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:47.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.627+0000 7f8334518700 1 --2- 172.21.15.117:0/2508641198 >> [v2:172.21.15.117:6818/4102123427,v1:172.21.15.117:6819/4102123427] conn(0x7f8314077710 0x7f8314079bb0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:47.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.627+0000 7f8334518700 1 -- 172.21.15.117:0/2508641198 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f831406d780 msgr2=0x7f831406fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:47.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.627+0000 7f8334518700 1 --2- 172.21.15.117:0/2508641198 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f831406d780 0x7f831406fc40 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f832c108cd0 tx=0x7f832000a300 comp rx=0 tx=0).stop 2024-03-04T15:51:47.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.627+0000 7f8334518700 1 -- 172.21.15.117:0/2508641198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f832c104270 msgr2=0x7f832c199d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:47.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.627+0000 7f8334518700 1 --2- 172.21.15.117:0/2508641198 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f832c104270 0x7f832c199d90 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f832400ef10 tx=0x7f832400c3a0 comp rx=0 tx=0).stop 2024-03-04T15:51:47.630 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.627+0000 7f8332ab5700 1 -- 172.21.15.117:0/2508641198 reap_dead start 2024-03-04T15:51:47.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.627+0000 7f8334518700 1 -- 172.21.15.117:0/2508641198 shutdown_connections 2024-03-04T15:51:47.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.627+0000 7f8334518700 1 -- 172.21.15.117:0/2508641198 >> 172.21.15.117:0/2508641198 conn(0x7f832c0ffb00 msgr2=0x7f832c0fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:47.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.627+0000 7f8334518700 1 -- 172.21.15.117:0/2508641198 shutdown_connections 2024-03-04T15:51:47.631 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.627+0000 7f8334518700 1 -- 172.21.15.117:0/2508641198 wait complete. 2024-03-04T15:51:47.681 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.445759089Z level=info msg="Executing migration" id="create short_url table v1" 2024-03-04T15:51:47.682 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.47110427Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-03-04T15:51:47.682 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.496459987Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-03-04T15:51:47.682 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.521747712Z level=info msg="Executing migration" id="delete alert_definition table" 2024-03-04T15:51:47.682 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.563781438Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-03-04T15:51:47.682 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.589219904Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-03-04T15:51:47.682 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.631294566Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-03-04T15:51:47.682 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.656697047Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-03-04T15:51:47.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bce91b700 1 -- 172.21.15.117:0/4003548021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0bc8104c30 msgr2=0x7f0bc8108c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:47.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bce91b700 1 --2- 172.21.15.117:0/4003548021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0bc8104c30 0x7f0bc8108c80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0bbc0099c0 tx=0x7f0bbc009cd0 comp rx=0 tx=0).stop 2024-03-04T15:51:47.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bce91b700 1 -- 172.21.15.117:0/4003548021 shutdown_connections 2024-03-04T15:51:47.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bce91b700 1 --2- 172.21.15.117:0/4003548021 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bc8109360 0x7f0bc8110f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:47.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bce91b700 1 --2- 172.21.15.117:0/4003548021 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc8104280 0x7f0bc8104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:47.861 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bce91b700 1 --2- 172.21.15.117:0/4003548021 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0bc8104c30 0x7f0bc8108c80 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:47.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bce91b700 1 -- 172.21.15.117:0/4003548021 >> 172.21.15.117:0/4003548021 conn(0x7f0bc80ffb10 msgr2=0x7f0bc8101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:47.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bce91b700 1 -- 172.21.15.117:0/4003548021 shutdown_connections 2024-03-04T15:51:47.862 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bce91b700 1 -- 172.21.15.117:0/4003548021 wait complete. 2024-03-04T15:51:47.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bce91b700 1 Processor -- start 2024-03-04T15:51:47.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bce91b700 1 -- start start 2024-03-04T15:51:47.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bce91b700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bc8104280 0x7f0bc819f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:47.863 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bce91b700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc8104c30 0x7f0bc819fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:47.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bce91b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0bc8109360 0x7f0bc8199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:47.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bce91b700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc81137d0 con 0x7f0bc8104280 2024-03-04T15:51:47.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bce91b700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc8113650 con 0x7f0bc8104c30 2024-03-04T15:51:47.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bce91b700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bc8113950 con 0x7f0bc8109360 2024-03-04T15:51:47.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bc7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bc8104280 0x7f0bc819f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:47.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bcceb8700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0bc8109360 0x7f0bc8199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:47.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bcceb8700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0bc8109360 0x7f0bc8199ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:50116/0 (socket says 172.21.15.117:50116) 2024-03-04T15:51:47.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bcceb8700 1 -- 172.21.15.117:0/41519692 learned_addr learned my addr 172.21.15.117:0/41519692 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:51:47.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bc77fe700 1 --2- 172.21.15.117:0/41519692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc8104c30 0x7f0bc819fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:47.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bcceb8700 1 -- 172.21.15.117:0/41519692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc8104c30 msgr2=0x7f0bc819fe60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:47.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bcceb8700 1 --2- 172.21.15.117:0/41519692 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0bc8104c30 0x7f0bc819fe60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:47.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bcceb8700 1 -- 172.21.15.117:0/41519692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bc8104280 msgr2=0x7f0bc819f920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:47.864 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bcceb8700 1 --2- 172.21.15.117:0/41519692 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f0bc8104280 0x7f0bc819f920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:47.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bcceb8700 1 -- 172.21.15.117:0/41519692 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bc819a240 con 0x7f0bc8109360 2024-03-04T15:51:47.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.859+0000 7f0bcceb8700 1 --2- 172.21.15.117:0/41519692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0bc8109360 0x7f0bc8199ae0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000c370 tx=0x7f0bc000c680 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:47.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.863+0000 7f0bc57fa700 1 -- 172.21.15.117:0/41519692 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc000ce20 con 0x7f0bc8109360 2024-03-04T15:51:47.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.863+0000 7f0bc57fa700 1 -- 172.21.15.117:0/41519692 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f0bc00090d0 con 0x7f0bc8109360 2024-03-04T15:51:47.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.863+0000 7f0bce91b700 1 -- 172.21.15.117:0/41519692 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bc819a4d0 con 0x7f0bc8109360 2024-03-04T15:51:47.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.863+0000 7f0bc57fa700 1 -- 172.21.15.117:0/41519692 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bc00046c0 con 0x7f0bc8109360 2024-03-04T15:51:47.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.863+0000 7f0bce91b700 1 -- 172.21.15.117:0/41519692 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bc81a67f0 con 0x7f0bc8109360 2024-03-04T15:51:47.865 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.863+0000 7f0baeffd700 1 -- 172.21.15.117:0/41519692 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ba8000f80 con 0x7f0bc8109360 2024-03-04T15:51:47.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.863+0000 7f0bc57fa700 1 -- 172.21.15.117:0/41519692 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f0bc001a020 con 0x7f0bc8109360 2024-03-04T15:51:47.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.863+0000 7f0bc57fa700 1 --2- 172.21.15.117:0/41519692 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f0bb006d780 0x7f0bb006fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:47.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.863+0000 7f0bc57fa700 1 -- 172.21.15.117:0/41519692 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7f0bc008dd80 con 0x7f0bc8109360 2024-03-04T15:51:47.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.863+0000 7f0bc7fff700 1 --2- 172.21.15.117:0/41519692 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f0bb006d780 0x7f0bb006fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:47.866 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.863+0000 7f0bc57fa700 1 --2- 172.21.15.117:0/41519692 >> [v2:172.21.15.151:6824/3965450917,v1:172.21.15.151:6825/3965450917] conn(0x7f0bb0077710 0x7f0bb0079bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:47.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.863+0000 7f0bc57fa700 1 -- 172.21.15.117:0/41519692 --> [v2:172.21.15.151:6824/3965450917,v1:172.21.15.151:6825/3965450917] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0bc001de50 con 0x7f0bb0077710 2024-03-04T15:51:47.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.863+0000 7f0bc57fa700 1 -- 172.21.15.117:0/41519692 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0bc008e210 con 0x7f0bc8109360 2024-03-04T15:51:47.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.863+0000 7f0bc7fff700 1 --2- 172.21.15.117:0/41519692 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f0bb006d780 0x7f0bb006fc40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800ac50 tx=0x7f0bb800a380 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:47.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.863+0000 7f0bc77fe700 1 --2- 172.21.15.117:0/41519692 >> [v2:172.21.15.151:6824/3965450917,v1:172.21.15.151:6825/3965450917] conn(0x7f0bb0077710 0x7f0bb0079bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:47.867 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.863+0000 7f0bc77fe700 1 --2- 172.21.15.117:0/41519692 >> [v2:172.21.15.151:6824/3965450917,v1:172.21.15.151:6825/3965450917] conn(0x7f0bb0077710 0x7f0bb0079bb0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:47.868 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.867+0000 7f0bc57fa700 1 -- 172.21.15.117:0/41519692 <== osd.7 v2:172.21.15.151:6824/3965450917 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f0bc001de50 con 0x7f0bb0077710 2024-03-04T15:51:47.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.887+0000 7f0baeffd700 1 -- 172.21.15.117:0/41519692 --> [v2:172.21.15.151:6824/3965450917,v1:172.21.15.151:6825/3965450917] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0ba8002d00 con 0x7f0bb0077710 2024-03-04T15:51:47.887 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.887+0000 7f0bc57fa700 1 -- 172.21.15.117:0/41519692 <== osd.7 v2:172.21.15.151:6824/3965450917 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f0ba8002d00 con 0x7f0bb0077710 2024-03-04T15:51:47.888 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.887+0000 7f0baeffd700 1 -- 172.21.15.117:0/41519692 >> [v2:172.21.15.151:6824/3965450917,v1:172.21.15.151:6825/3965450917] conn(0x7f0bb0077710 msgr2=0x7f0bb0079bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:47.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.887+0000 7f0baeffd700 1 --2- 172.21.15.117:0/41519692 >> [v2:172.21.15.151:6824/3965450917,v1:172.21.15.151:6825/3965450917] conn(0x7f0bb0077710 0x7f0bb0079bb0 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:47.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.887+0000 7f0baeffd700 1 -- 172.21.15.117:0/41519692 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f0bb006d780 msgr2=0x7f0bb006fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:47.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.887+0000 7f0baeffd700 1 --2- 172.21.15.117:0/41519692 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f0bb006d780 0x7f0bb006fc40 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f0bb800ac50 tx=0x7f0bb800a380 comp rx=0 tx=0).stop 2024-03-04T15:51:47.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.887+0000 7f0baeffd700 1 -- 172.21.15.117:0/41519692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0bc8109360 msgr2=0x7f0bc8199ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:47.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.887+0000 7f0baeffd700 1 --2- 172.21.15.117:0/41519692 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0bc8109360 0x7f0bc8199ae0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0bc000c370 tx=0x7f0bc000c680 comp rx=0 tx=0).stop 2024-03-04T15:51:47.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.887+0000 7f0bcceb8700 1 -- 172.21.15.117:0/41519692 reap_dead start 2024-03-04T15:51:47.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.887+0000 7f0baeffd700 1 -- 172.21.15.117:0/41519692 shutdown_connections 2024-03-04T15:51:47.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.887+0000 7f0baeffd700 1 -- 172.21.15.117:0/41519692 >> 172.21.15.117:0/41519692 conn(0x7f0bc80ffb10 msgr2=0x7f0bc8100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:47.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.887+0000 7f0baeffd700 1 -- 172.21.15.117:0/41519692 shutdown_connections 2024-03-04T15:51:47.889 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:47.887+0000 7f0baeffd700 1 -- 172.21.15.117:0/41519692 wait complete. 2024-03-04T15:51:47.977 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.698661969Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-03-04T15:51:47.977 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.723965355Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-03-04T15:51:47.977 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.757822274Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-03-04T15:51:47.977 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.783062897Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-03-04T15:51:47.978 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.808298813Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-03-04T15:51:47.978 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.850333991Z level=info msg="Executing migration" id="drop alert_definition table" 2024-03-04T15:51:47.978 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.892817728Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-03-04T15:51:47.978 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.934958825Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-03-04T15:51:47.978 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:47 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:47.977038217Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-03-04T15:51:48.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.143+0000 7f194cbd3700 1 -- 172.21.15.117:0/2441510372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19481019c0 msgr2=0x7f194810db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:48.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.143+0000 7f194cbd3700 1 --2- 172.21.15.117:0/2441510372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19481019c0 0x7f194810db40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f194000b0d0 tx=0x7f194000b3e0 comp rx=0 tx=0).stop 2024-03-04T15:51:48.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.143+0000 7f194cbd3700 1 -- 172.21.15.117:0/2441510372 shutdown_connections 2024-03-04T15:51:48.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.143+0000 7f194cbd3700 1 --2- 172.21.15.117:0/2441510372 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19481019c0 0x7f194810db40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:48.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.143+0000 7f194cbd3700 1 --2- 172.21.15.117:0/2441510372 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1948106ef0 0x7f19481072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:48.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.143+0000 7f194cbd3700 1 --2- 172.21.15.117:0/2441510372 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1948101000 0x7f1948101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:48.147 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.143+0000 7f194cbd3700 1 -- 172.21.15.117:0/2441510372 >> 172.21.15.117:0/2441510372 conn(0x7f1948077a40 msgr2=0x7f1948077e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:48.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.143+0000 7f194cbd3700 1 -- 172.21.15.117:0/2441510372 shutdown_connections 2024-03-04T15:51:48.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.143+0000 7f194cbd3700 1 -- 172.21.15.117:0/2441510372 wait complete. 2024-03-04T15:51:48.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f194cbd3700 1 Processor -- start 2024-03-04T15:51:48.148 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f194cbd3700 1 -- start start 2024-03-04T15:51:48.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f194cbd3700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1948101000 0x7f194810e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:48.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f194cbd3700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19481019c0 0x7f194810ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:48.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f194cbd3700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1948106ef0 0x7f194810f110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:48.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f194cbd3700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1948111700 con 0x7f1948101000 2024-03-04T15:51:48.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f194cbd3700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1948111580 con 0x7f1948106ef0 2024-03-04T15:51:48.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f194cbd3700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1948111880 con 0x7f19481019c0 2024-03-04T15:51:48.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f194659c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1948101000 0x7f194810e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:48.149 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f1945d9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19481019c0 0x7f194810ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:48.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f1945d9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19481019c0 0x7f194810ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:50130/0 (socket says 172.21.15.117:50130) 2024-03-04T15:51:48.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f194659c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1948101000 0x7f194810e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51622/0 (socket says 172.21.15.117:51622) 2024-03-04T15:51:48.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f1945d9b700 1 -- 172.21.15.117:0/23105779 learned_addr learned my addr 172.21.15.117:0/23105779 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:51:48.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f1945d9b700 1 -- 172.21.15.117:0/23105779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1948106ef0 msgr2=0x7f194810f110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:48.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f1945d9b700 1 --2- 172.21.15.117:0/23105779 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f1948106ef0 0x7f194810f110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:48.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f1945d9b700 1 -- 172.21.15.117:0/23105779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1948101000 msgr2=0x7f194810e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:48.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f1945d9b700 1 --2- 172.21.15.117:0/23105779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1948101000 0x7f194810e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:48.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f1945d9b700 1 -- 172.21.15.117:0/23105779 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19481a5f90 con 0x7f19481019c0 2024-03-04T15:51:48.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f194659c700 1 --2- 172.21.15.117:0/23105779 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f1948101000 0x7f194810e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:51:48.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f1945d9b700 1 --2- 172.21.15.117:0/23105779 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19481019c0 0x7f194810ebd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f193c00eb30 tx=0x7f193c00ee40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:48.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f19377fe700 1 -- 172.21.15.117:0/23105779 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c00cc90 con 0x7f19481019c0 2024-03-04T15:51:48.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f19377fe700 1 -- 172.21.15.117:0/23105779 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f193c00ce30 con 0x7f19481019c0 2024-03-04T15:51:48.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f194cbd3700 1 -- 172.21.15.117:0/23105779 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19481a6280 con 0x7f19481019c0 2024-03-04T15:51:48.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f19377fe700 1 -- 172.21.15.117:0/23105779 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f193c003d80 con 0x7f19481019c0 2024-03-04T15:51:48.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f194cbd3700 1 -- 172.21.15.117:0/23105779 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19481a67c0 con 0x7f19481019c0 2024-03-04T15:51:48.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f194cbd3700 1 -- 172.21.15.117:0/23105779 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f19480624f0 con 0x7f19481019c0 2024-03-04T15:51:48.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f19377fe700 1 -- 172.21.15.117:0/23105779 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f193c003fc0 con 0x7f19481019c0 2024-03-04T15:51:48.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f19377fe700 1 --2- 172.21.15.117:0/23105779 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f193006d660 0x7f193006fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:48.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f19377fe700 1 -- 172.21.15.117:0/23105779 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7f193c08dbf0 con 0x7f19481019c0 2024-03-04T15:51:48.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.147+0000 7f194659c700 1 --2- 172.21.15.117:0/23105779 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f193006d660 0x7f193006fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:48.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.151+0000 7f19377fe700 1 --2- 172.21.15.117:0/23105779 >> [v2:172.21.15.151:6808/3277640682,v1:172.21.15.151:6809/3277640682] conn(0x7f1930077470 0x7f1930079910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:48.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.151+0000 7f19377fe700 1 -- 172.21.15.117:0/23105779 --> [v2:172.21.15.151:6808/3277640682,v1:172.21.15.151:6809/3277640682] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f193c00ce30 con 0x7f1930077470 2024-03-04T15:51:48.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.151+0000 7f19377fe700 1 -- 172.21.15.117:0/23105779 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f193c090270 con 0x7f19481019c0 2024-03-04T15:51:48.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.151+0000 7f194659c700 1 --2- 172.21.15.117:0/23105779 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f193006d660 0x7f193006fb20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1938005950 tx=0x7f193800a300 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:48.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.151+0000 7f1946d9d700 1 --2- 172.21.15.117:0/23105779 >> [v2:172.21.15.151:6808/3277640682,v1:172.21.15.151:6809/3277640682] conn(0x7f1930077470 0x7f1930079910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:48.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.151+0000 7f1946d9d700 1 --2- 172.21.15.117:0/23105779 >> [v2:172.21.15.151:6808/3277640682,v1:172.21.15.151:6809/3277640682] conn(0x7f1930077470 0x7f1930079910 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:48.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.151+0000 7f19377fe700 1 -- 172.21.15.117:0/23105779 <== osd.5 v2:172.21.15.151:6808/3277640682 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f193c00ce30 con 0x7f1930077470 2024-03-04T15:51:48.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.171+0000 7f194cbd3700 1 -- 172.21.15.117:0/23105779 --> [v2:172.21.15.151:6808/3277640682,v1:172.21.15.151:6809/3277640682] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f19480628f0 con 0x7f1930077470 2024-03-04T15:51:48.172 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.171+0000 7f19377fe700 1 -- 172.21.15.117:0/23105779 <== osd.5 v2:172.21.15.151:6808/3277640682 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f19480628f0 con 0x7f1930077470 2024-03-04T15:51:48.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.171+0000 7f194cbd3700 1 -- 172.21.15.117:0/23105779 >> [v2:172.21.15.151:6808/3277640682,v1:172.21.15.151:6809/3277640682] conn(0x7f1930077470 msgr2=0x7f1930079910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:48.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.171+0000 7f194cbd3700 1 --2- 172.21.15.117:0/23105779 >> [v2:172.21.15.151:6808/3277640682,v1:172.21.15.151:6809/3277640682] conn(0x7f1930077470 0x7f1930079910 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:48.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.171+0000 7f194cbd3700 1 -- 172.21.15.117:0/23105779 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f193006d660 msgr2=0x7f193006fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:48.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.171+0000 7f194cbd3700 1 --2- 172.21.15.117:0/23105779 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f193006d660 0x7f193006fb20 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1938005950 tx=0x7f193800a300 comp rx=0 tx=0).stop 2024-03-04T15:51:48.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.171+0000 7f194cbd3700 1 -- 172.21.15.117:0/23105779 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19481019c0 msgr2=0x7f194810ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:48.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.171+0000 7f194cbd3700 1 --2- 172.21.15.117:0/23105779 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f19481019c0 0x7f194810ebd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f193c00eb30 tx=0x7f193c00ee40 comp rx=0 tx=0).stop 2024-03-04T15:51:48.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.171+0000 7f1946d9d700 1 -- 172.21.15.117:0/23105779 reap_dead start 2024-03-04T15:51:48.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.171+0000 7f194cbd3700 1 -- 172.21.15.117:0/23105779 shutdown_connections 2024-03-04T15:51:48.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.171+0000 7f194cbd3700 1 -- 172.21.15.117:0/23105779 >> 172.21.15.117:0/23105779 conn(0x7f1948077a40 msgr2=0x7f19480ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:48.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.171+0000 7f194cbd3700 1 -- 172.21.15.117:0/23105779 shutdown_connections 2024-03-04T15:51:48.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:48.171+0000 7f194cbd3700 1 -- 172.21.15.117:0/23105779 wait complete. 2024-03-04T15:51:48.185 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:48 smithi117 bash[21480]: audit 2024-03-04T15:51:46.344481+0000 mgr.y (mgr.24428) 68 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:51:48.185 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:48 smithi117 bash[28843]: audit 2024-03-04T15:51:46.344481+0000 mgr.y (mgr.24428) 68 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:51:48.263 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[23756]: audit 2024-03-04T15:51:46.344481+0000 mgr.y (mgr.24428) 68 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:51:48.264 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.019180185Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-03-04T15:51:48.264 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.044693601Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-03-04T15:51:48.264 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.086625437Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-03-04T15:51:48.264 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.112046357Z level=info msg="Executing migration" id="create alert_instance table" 2024-03-04T15:51:48.264 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.137382215Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-03-04T15:51:48.264 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.179532077Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-03-04T15:51:48.264 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.221634526Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-03-04T15:51:48.264 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.263669947Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-03-04T15:51:48.525 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.28916658Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-03-04T15:51:48.525 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.322845785Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-03-04T15:51:48.525 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.373185058Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-03-04T15:51:48.525 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.42349138Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-03-04T15:51:48.525 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.474020592Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-03-04T15:51:48.525 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.499479795Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-03-04T15:51:48.525 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.524834949Z level=info msg="Executing migration" id="create alert_rule table" 2024-03-04T15:51:48.778 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.550226844Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-03-04T15:51:48.778 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.583772855Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-03-04T15:51:48.778 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.626156793Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-03-04T15:51:48.778 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.676628977Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-03-04T15:51:48.778 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.702165022Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-03-04T15:51:48.779 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.727448918Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-03-04T15:51:48.779 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.752826889Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-03-04T15:51:48.779 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.778125565Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-03-04T15:51:49.048 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[23756]: cluster 2024-03-04T15:51:47.354726+0000 mgr.y (mgr.24428) 69 : cluster 0 pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:49.048 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.803523602Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-03-04T15:51:49.048 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.845531303Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-03-04T15:51:49.048 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.870904021Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-03-04T15:51:49.048 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.896281725Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-03-04T15:51:49.049 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.938318009Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-03-04T15:51:49.049 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.963690777Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-03-04T15:51:49.049 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:48 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:48.997378053Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-03-04T15:51:49.049 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.047984095Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-03-04T15:51:49.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.147+0000 7f13e23aa700 1 -- 172.21.15.117:0/1347594273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13dc109370 msgr2=0x7f13dc110f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:49.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.147+0000 7f13e23aa700 1 --2- 172.21.15.117:0/1347594273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13dc109370 0x7f13dc110f40 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f13d4009fc0 tx=0x7f13d400b330 comp rx=0 tx=0).stop 2024-03-04T15:51:49.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.147+0000 7f13e23aa700 1 -- 172.21.15.117:0/1347594273 shutdown_connections 2024-03-04T15:51:49.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.147+0000 7f13e23aa700 1 --2- 172.21.15.117:0/1347594273 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13dc109370 0x7f13dc110f40 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:49.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.147+0000 7f13e23aa700 1 --2- 172.21.15.117:0/1347594273 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13dc104290 0x7f13dc104670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:49.150 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.147+0000 7f13e23aa700 1 --2- 172.21.15.117:0/1347594273 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13dc104c40 0x7f13dc108c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:49.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.147+0000 7f13e23aa700 1 -- 172.21.15.117:0/1347594273 >> 172.21.15.117:0/1347594273 conn(0x7f13dc0ffb40 msgr2=0x7f13dc101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:49.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.147+0000 7f13e23aa700 1 -- 172.21.15.117:0/1347594273 shutdown_connections 2024-03-04T15:51:49.151 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.147+0000 7f13e23aa700 1 -- 172.21.15.117:0/1347594273 wait complete. 2024-03-04T15:51:49.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.147+0000 7f13e23aa700 1 Processor -- start 2024-03-04T15:51:49.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13e23aa700 1 -- start start 2024-03-04T15:51:49.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13e23aa700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13dc104290 0x7f13dc199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:49.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13e23aa700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13dc104c40 0x7f13dc19a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:49.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13e23aa700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13dc109370 0x7f13dc19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:49.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13e23aa700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc113820 con 0x7f13dc104290 2024-03-04T15:51:49.152 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13e23aa700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc1136a0 con 0x7f13dc104c40 2024-03-04T15:51:49.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13e23aa700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13dc1139a0 con 0x7f13dc109370 2024-03-04T15:51:49.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13dbfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13dc104290 0x7f13dc199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:49.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13dbfff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13dc104290 0x7f13dc199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51640/0 (socket says 172.21.15.117:51640) 2024-03-04T15:51:49.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13dbfff700 1 -- 172.21.15.117:0/4267397955 learned_addr learned my addr 172.21.15.117:0/4267397955 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:51:49.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13e0947700 1 --2- 172.21.15.117:0/4267397955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13dc109370 0x7f13dc19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:49.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13d37fe700 1 --2- 172.21.15.117:0/4267397955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13dc104c40 0x7f13dc19a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:49.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13dbfff700 1 -- 172.21.15.117:0/4267397955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13dc109370 msgr2=0x7f13dc19e630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:49.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13dbfff700 1 --2- 172.21.15.117:0/4267397955 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f13dc109370 0x7f13dc19e630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:49.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13dbfff700 1 -- 172.21.15.117:0/4267397955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13dc104c40 msgr2=0x7f13dc19a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:49.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13dbfff700 1 --2- 172.21.15.117:0/4267397955 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f13dc104c40 0x7f13dc19a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:49.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13dbfff700 1 -- 172.21.15.117:0/4267397955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13dc19ed70 con 0x7f13dc104290 2024-03-04T15:51:49.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13dbfff700 1 --2- 172.21.15.117:0/4267397955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13dc104290 0x7f13dc199d60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00d950 tx=0x7f13cc00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:49.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13d9ffb700 1 -- 172.21.15.117:0/4267397955 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc016070 con 0x7f13dc104290 2024-03-04T15:51:49.153 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13e23aa700 1 -- 172.21.15.117:0/4267397955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13dc19f060 con 0x7f13dc104290 2024-03-04T15:51:49.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13d9ffb700 1 -- 172.21.15.117:0/4267397955 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f13cc009dc0 con 0x7f13dc104290 2024-03-04T15:51:49.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13d9ffb700 1 -- 172.21.15.117:0/4267397955 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13cc00f3c0 con 0x7f13dc104290 2024-03-04T15:51:49.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13e23aa700 1 -- 172.21.15.117:0/4267397955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13dc1a68e0 con 0x7f13dc104290 2024-03-04T15:51:49.154 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13e23aa700 1 -- 172.21.15.117:0/4267397955 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f13dc1009c0 con 0x7f13dc104290 2024-03-04T15:51:49.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13d9ffb700 1 -- 172.21.15.117:0/4267397955 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f13cc019070 con 0x7f13dc104290 2024-03-04T15:51:49.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.151+0000 7f13d9ffb700 1 --2- 172.21.15.117:0/4267397955 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f13c806d780 0x7f13c806fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:49.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.155+0000 7f13d9ffb700 1 -- 172.21.15.117:0/4267397955 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7f13cc08d9f0 con 0x7f13dc104290 2024-03-04T15:51:49.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.155+0000 7f13d37fe700 1 --2- 172.21.15.117:0/4267397955 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f13c806d780 0x7f13c806fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:49.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.155+0000 7f13d9ffb700 1 --2- 172.21.15.117:0/4267397955 >> [v2:172.21.15.117:6826/3320151079,v1:172.21.15.117:6827/3320151079] conn(0x7f13c8077710 0x7f13c8079bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:49.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.155+0000 7f13d9ffb700 1 -- 172.21.15.117:0/4267397955 --> [v2:172.21.15.117:6826/3320151079,v1:172.21.15.117:6827/3320151079] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f13cc014e80 con 0x7f13c8077710 2024-03-04T15:51:49.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.155+0000 7f13d9ffb700 1 -- 172.21.15.117:0/4267397955 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f13cc08de80 con 0x7f13dc104290 2024-03-04T15:51:49.155 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.155+0000 7f13e0947700 1 --2- 172.21.15.117:0/4267397955 >> [v2:172.21.15.117:6826/3320151079,v1:172.21.15.117:6827/3320151079] conn(0x7f13c8077710 0x7f13c8079bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:49.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.155+0000 7f13d37fe700 1 --2- 172.21.15.117:0/4267397955 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f13c806d780 0x7f13c806fc40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f13dc108cd0 tx=0x7f13c4009310 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:49.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.155+0000 7f13e0947700 1 --2- 172.21.15.117:0/4267397955 >> [v2:172.21.15.117:6826/3320151079,v1:172.21.15.117:6827/3320151079] conn(0x7f13c8077710 0x7f13c8079bb0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:49.156 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.155+0000 7f13d9ffb700 1 -- 172.21.15.117:0/4267397955 <== osd.3 v2:172.21.15.117:6826/3320151079 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f13cc014e80 con 0x7f13c8077710 2024-03-04T15:51:49.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.171+0000 7f13e23aa700 1 -- 172.21.15.117:0/4267397955 --> [v2:172.21.15.117:6826/3320151079,v1:172.21.15.117:6827/3320151079] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f13dc105770 con 0x7f13c8077710 2024-03-04T15:51:49.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.171+0000 7f13d9ffb700 1 -- 172.21.15.117:0/4267397955 <== osd.3 v2:172.21.15.117:6826/3320151079 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f13dc105770 con 0x7f13c8077710 2024-03-04T15:51:49.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.171+0000 7f13e23aa700 1 -- 172.21.15.117:0/4267397955 >> [v2:172.21.15.117:6826/3320151079,v1:172.21.15.117:6827/3320151079] conn(0x7f13c8077710 msgr2=0x7f13c8079bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:49.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.171+0000 7f13e23aa700 1 --2- 172.21.15.117:0/4267397955 >> [v2:172.21.15.117:6826/3320151079,v1:172.21.15.117:6827/3320151079] conn(0x7f13c8077710 0x7f13c8079bb0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:49.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.171+0000 7f13e23aa700 1 -- 172.21.15.117:0/4267397955 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f13c806d780 msgr2=0x7f13c806fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:49.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.171+0000 7f13e23aa700 1 --2- 172.21.15.117:0/4267397955 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f13c806d780 0x7f13c806fc40 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f13dc108cd0 tx=0x7f13c4009310 comp rx=0 tx=0).stop 2024-03-04T15:51:49.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.171+0000 7f13e23aa700 1 -- 172.21.15.117:0/4267397955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13dc104290 msgr2=0x7f13dc199d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:49.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.171+0000 7f13e23aa700 1 --2- 172.21.15.117:0/4267397955 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f13dc104290 0x7f13dc199d60 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f13cc00d950 tx=0x7f13cc00dc60 comp rx=0 tx=0).stop 2024-03-04T15:51:49.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.171+0000 7f13e0947700 1 -- 172.21.15.117:0/4267397955 reap_dead start 2024-03-04T15:51:49.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.171+0000 7f13e23aa700 1 -- 172.21.15.117:0/4267397955 shutdown_connections 2024-03-04T15:51:49.175 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.171+0000 7f13e23aa700 1 -- 172.21.15.117:0/4267397955 >> 172.21.15.117:0/4267397955 conn(0x7f13dc0ffb40 msgr2=0x7f13dc0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:49.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.175+0000 7f13e23aa700 1 -- 172.21.15.117:0/4267397955 shutdown_connections 2024-03-04T15:51:49.176 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:49.175+0000 7f13e23aa700 1 -- 172.21.15.117:0/4267397955 wait complete. 2024-03-04T15:51:49.219 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:49 smithi117 bash[21480]: cluster 2024-03-04T15:51:47.354726+0000 mgr.y (mgr.24428) 69 : cluster 0 pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:49.220 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:49 smithi117 bash[28843]: cluster 2024-03-04T15:51:47.354726+0000 mgr.y (mgr.24428) 69 : cluster 0 pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:49.309 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.081695094Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-03-04T15:51:49.309 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.132041401Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-03-04T15:51:49.309 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.174148308Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-03-04T15:51:49.309 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.2245487Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-03-04T15:51:49.309 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.274968465Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-03-04T15:51:49.309 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.308724191Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-03-04T15:51:49.570 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.342390361Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-03-04T15:51:49.570 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.367966547Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-03-04T15:51:49.570 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.393357652Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-03-04T15:51:49.570 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.418671647Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-03-04T15:51:49.570 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.444021804Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-03-04T15:51:49.570 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.477646325Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-03-04T15:51:49.570 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.569798359Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-03-04T15:51:49.848 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.595252267Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-03-04T15:51:49.848 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.637171587Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-03-04T15:51:49.848 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.679359578Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-03-04T15:51:49.848 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.704712891Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-03-04T15:51:49.848 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.746803888Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-03-04T15:51:49.848 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.797410293Z level=info msg="Executing migration" id="create provenance_type table" 2024-03-04T15:51:49.848 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.847796273Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-03-04T15:51:50.109 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.873215737Z level=info msg="Executing migration" id="create alert_image table" 2024-03-04T15:51:50.110 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.906912797Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-03-04T15:51:50.110 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.932206726Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-03-04T15:51:50.110 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:49 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:49.974249291Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-03-04T15:51:50.110 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.016410097Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-03-04T15:51:50.110 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.058462133Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-03-04T15:51:50.110 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.05882437Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-03-04T15:51:50.110 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.083822572Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-03-04T15:51:50.110 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.109139142Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-03-04T15:51:50.362 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.151544662Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-03-04T15:51:50.362 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.152028068Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-03-04T15:51:50.362 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.152584596Z level=info msg="alerts found to migrate" alerts=0 2024-03-04T15:51:50.363 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.19357851Z level=info msg="Executing migration" id="create library_element table v1" 2024-03-04T15:51:50.363 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.219001869Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-03-04T15:51:50.363 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.261004613Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-03-04T15:51:50.363 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.286478222Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-03-04T15:51:50.363 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.311654198Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-03-04T15:51:50.363 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.337116799Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-03-04T15:51:50.363 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.362271372Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-03-04T15:51:50.597 INFO:teuthology.orchestra.run.smithi117.stdout:167503724602 2024-03-04T15:51:50.597 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.5 2024-03-04T15:51:50.640 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.387649957Z level=info msg="Executing migration" id="create data_keys table" 2024-03-04T15:51:50.640 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.4130177Z level=info msg="Executing migration" id="create secrets table" 2024-03-04T15:51:50.640 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.43831439Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-03-04T15:51:50.640 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.480334099Z level=info msg="Executing migration" id="add name column into data_keys" 2024-03-04T15:51:50.640 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.505626056Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-03-04T15:51:50.640 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.530918946Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-03-04T15:51:50.641 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.572957682Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-03-04T15:51:50.641 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.640106204Z level=info msg="Executing migration" id="create kv_store table v1" 2024-03-04T15:51:50.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.723+0000 7f5b25bde700 1 -- 172.21.15.117:0/493769880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b2010e670 msgr2=0x7f5b20110aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:50.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.723+0000 7f5b25bde700 1 --2- 172.21.15.117:0/493769880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b2010e670 0x7f5b20110aa0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f5b18009f90 tx=0x7f5b1800b330 comp rx=0 tx=0).stop 2024-03-04T15:51:50.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.723+0000 7f5b25bde700 1 -- 172.21.15.117:0/493769880 shutdown_connections 2024-03-04T15:51:50.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.723+0000 7f5b25bde700 1 --2- 172.21.15.117:0/493769880 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b20102d00 0x7f5b2006a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:50.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.723+0000 7f5b25bde700 1 --2- 172.21.15.117:0/493769880 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b2010e670 0x7f5b20110aa0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:50.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.723+0000 7f5b25bde700 1 --2- 172.21.15.117:0/493769880 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b2006a980 0x7f5b2006ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:50.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.723+0000 7f5b25bde700 1 -- 172.21.15.117:0/493769880 >> 172.21.15.117:0/493769880 conn(0x7f5b20077a40 msgr2=0x7f5b20077e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:50.727 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.723+0000 7f5b25bde700 1 -- 172.21.15.117:0/493769880 shutdown_connections 2024-03-04T15:51:50.728 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.723+0000 7f5b25bde700 1 -- 172.21.15.117:0/493769880 wait complete. 2024-03-04T15:51:50.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b25bde700 1 Processor -- start 2024-03-04T15:51:50.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b25bde700 1 -- start start 2024-03-04T15:51:50.729 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b25bde700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2006a980 0x7f5b2019fa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:50.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b25bde700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b20102d00 0x7f5b2019ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:50.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b25bde700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b2010e670 0x7f5b20199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:50.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b25bde700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b201137b0 con 0x7f5b2006a980 2024-03-04T15:51:50.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b25bde700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b20113630 con 0x7f5b2010e670 2024-03-04T15:51:50.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b25bde700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b20113930 con 0x7f5b20102d00 2024-03-04T15:51:50.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b1f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2006a980 0x7f5b2019fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:50.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b1effd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b20102d00 0x7f5b2019ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:50.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b1f7fe700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2006a980 0x7f5b2019fa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51668/0 (socket says 172.21.15.117:51668) 2024-03-04T15:51:50.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b1f7fe700 1 -- 172.21.15.117:0/3344203139 learned_addr learned my addr 172.21.15.117:0/3344203139 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:51:50.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b1ffff700 1 --2- 172.21.15.117:0/3344203139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b2010e670 0x7f5b20199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:50.730 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b1f7fe700 1 -- 172.21.15.117:0/3344203139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b20102d00 msgr2=0x7f5b2019ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:50.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b1f7fe700 1 --2- 172.21.15.117:0/3344203139 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5b20102d00 0x7f5b2019ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:50.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b1f7fe700 1 -- 172.21.15.117:0/3344203139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b2010e670 msgr2=0x7f5b20199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:50.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b1f7fe700 1 --2- 172.21.15.117:0/3344203139 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5b2010e670 0x7f5b20199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:50.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b1f7fe700 1 -- 172.21.15.117:0/3344203139 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b2019a400 con 0x7f5b2006a980 2024-03-04T15:51:50.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b1f7fe700 1 --2- 172.21.15.117:0/3344203139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2006a980 0x7f5b2019fa50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800ec10 tx=0x7f5b0800ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:50.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b1cff9700 1 -- 172.21.15.117:0/3344203139 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b0800cd40 con 0x7f5b2006a980 2024-03-04T15:51:50.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b25bde700 1 -- 172.21.15.117:0/3344203139 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b2019a6f0 con 0x7f5b2006a980 2024-03-04T15:51:50.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b25bde700 1 -- 172.21.15.117:0/3344203139 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b201a69d0 con 0x7f5b2006a980 2024-03-04T15:51:50.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b1cff9700 1 -- 172.21.15.117:0/3344203139 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f5b080044a0 con 0x7f5b2006a980 2024-03-04T15:51:50.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b1cff9700 1 -- 172.21.15.117:0/3344203139 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b08010430 con 0x7f5b2006a980 2024-03-04T15:51:50.731 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b25bde700 1 -- 172.21.15.117:0/3344203139 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f5b200624b0 con 0x7f5b2006a980 2024-03-04T15:51:50.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.727+0000 7f5b1cff9700 1 -- 172.21.15.117:0/3344203139 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f5b08003f50 con 0x7f5b2006a980 2024-03-04T15:51:50.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.731+0000 7f5b1cff9700 1 --2- 172.21.15.117:0/3344203139 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f5b0c06d780 0x7f5b0c06fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:50.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.731+0000 7f5b1cff9700 1 -- 172.21.15.117:0/3344203139 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7f5b0808e1b0 con 0x7f5b2006a980 2024-03-04T15:51:50.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.731+0000 7f5b1effd700 1 --2- 172.21.15.117:0/3344203139 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f5b0c06d780 0x7f5b0c06fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:50.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.731+0000 7f5b1cff9700 1 --2- 172.21.15.117:0/3344203139 >> [v2:172.21.15.117:6810/444026972,v1:172.21.15.117:6811/444026972] conn(0x7f5b0c077710 0x7f5b0c079bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:50.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.731+0000 7f5b1cff9700 1 -- 172.21.15.117:0/3344203139 --> [v2:172.21.15.117:6810/444026972,v1:172.21.15.117:6811/444026972] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f5b08022e80 con 0x7f5b0c077710 2024-03-04T15:51:50.733 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.731+0000 7f5b1cff9700 1 -- 172.21.15.117:0/3344203139 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5b0808e640 con 0x7f5b2006a980 2024-03-04T15:51:50.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.731+0000 7f5b1ffff700 1 --2- 172.21.15.117:0/3344203139 >> [v2:172.21.15.117:6810/444026972,v1:172.21.15.117:6811/444026972] conn(0x7f5b0c077710 0x7f5b0c079bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:50.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.731+0000 7f5b1effd700 1 --2- 172.21.15.117:0/3344203139 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f5b0c06d780 0x7f5b0c06fc40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5b2019b270 tx=0x7f5b1000a300 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:50.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.731+0000 7f5b1ffff700 1 --2- 172.21.15.117:0/3344203139 >> [v2:172.21.15.117:6810/444026972,v1:172.21.15.117:6811/444026972] conn(0x7f5b0c077710 0x7f5b0c079bb0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:50.734 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.731+0000 7f5b1cff9700 1 -- 172.21.15.117:0/3344203139 <== osd.1 v2:172.21.15.117:6810/444026972 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f5b08022e80 con 0x7f5b0c077710 2024-03-04T15:51:50.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.747+0000 7f5b25bde700 1 -- 172.21.15.117:0/3344203139 --> [v2:172.21.15.117:6810/444026972,v1:172.21.15.117:6811/444026972] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f5b200628b0 con 0x7f5b0c077710 2024-03-04T15:51:50.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.747+0000 7f5b1cff9700 1 -- 172.21.15.117:0/3344203139 <== osd.1 v2:172.21.15.117:6810/444026972 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f5b200628b0 con 0x7f5b0c077710 2024-03-04T15:51:50.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.747+0000 7f5b25bde700 1 -- 172.21.15.117:0/3344203139 >> [v2:172.21.15.117:6810/444026972,v1:172.21.15.117:6811/444026972] conn(0x7f5b0c077710 msgr2=0x7f5b0c079bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:50.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.747+0000 7f5b25bde700 1 --2- 172.21.15.117:0/3344203139 >> [v2:172.21.15.117:6810/444026972,v1:172.21.15.117:6811/444026972] conn(0x7f5b0c077710 0x7f5b0c079bb0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:50.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.747+0000 7f5b25bde700 1 -- 172.21.15.117:0/3344203139 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f5b0c06d780 msgr2=0x7f5b0c06fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:50.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.747+0000 7f5b25bde700 1 --2- 172.21.15.117:0/3344203139 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f5b0c06d780 0x7f5b0c06fc40 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5b2019b270 tx=0x7f5b1000a300 comp rx=0 tx=0).stop 2024-03-04T15:51:50.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.747+0000 7f5b25bde700 1 -- 172.21.15.117:0/3344203139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2006a980 msgr2=0x7f5b2019fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:50.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.747+0000 7f5b25bde700 1 --2- 172.21.15.117:0/3344203139 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5b2006a980 0x7f5b2019fa50 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f5b0800ec10 tx=0x7f5b0800ef20 comp rx=0 tx=0).stop 2024-03-04T15:51:50.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.747+0000 7f5b1ffff700 1 -- 172.21.15.117:0/3344203139 reap_dead start 2024-03-04T15:51:50.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.747+0000 7f5b25bde700 1 -- 172.21.15.117:0/3344203139 shutdown_connections 2024-03-04T15:51:50.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.747+0000 7f5b25bde700 1 -- 172.21.15.117:0/3344203139 >> 172.21.15.117:0/3344203139 conn(0x7f5b20077a40 msgr2=0x7f5b20110170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:50.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.751+0000 7f5b25bde700 1 -- 172.21.15.117:0/3344203139 shutdown_connections 2024-03-04T15:51:50.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.751+0000 7f5b25bde700 1 -- 172.21.15.117:0/3344203139 wait complete. 2024-03-04T15:51:50.902 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.690591006Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-03-04T15:51:50.902 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.715831277Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-03-04T15:51:50.902 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.757927138Z level=info msg="Executing migration" id="create permission table" 2024-03-04T15:51:50.902 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.783336977Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-03-04T15:51:50.902 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.808704626Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-03-04T15:51:50.903 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.850747861Z level=info msg="Executing migration" id="create role table" 2024-03-04T15:51:50.903 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.876043632Z level=info msg="Executing migration" id="add column display_name" 2024-03-04T15:51:50.903 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.901643595Z level=info msg="Executing migration" id="add column group_name" 2024-03-04T15:51:50.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbed2a700 1 -- 172.21.15.117:0/2171002727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db8100d40 msgr2=0x7f3db81011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:50.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbed2a700 1 --2- 172.21.15.117:0/2171002727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db8100d40 0x7f3db81011c0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00b0a0 tx=0x7f3dac00b3b0 comp rx=0 tx=0).stop 2024-03-04T15:51:50.925 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbed2a700 1 -- 172.21.15.117:0/2171002727 shutdown_connections 2024-03-04T15:51:50.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbed2a700 1 --2- 172.21.15.117:0/2171002727 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db8101700 0x7f3db810db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:50.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbed2a700 1 --2- 172.21.15.117:0/2171002727 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3db8106dc0 0x7f3db81071a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:50.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbed2a700 1 --2- 172.21.15.117:0/2171002727 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db8100d40 0x7f3db81011c0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:50.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbed2a700 1 -- 172.21.15.117:0/2171002727 >> 172.21.15.117:0/2171002727 conn(0x7f3db8076d80 msgr2=0x7f3db8077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:50.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbed2a700 1 -- 172.21.15.117:0/2171002727 shutdown_connections 2024-03-04T15:51:50.926 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbed2a700 1 -- 172.21.15.117:0/2171002727 wait complete. 2024-03-04T15:51:50.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbed2a700 1 Processor -- start 2024-03-04T15:51:50.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbed2a700 1 -- start start 2024-03-04T15:51:50.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbed2a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3db8100d40 0x7f3db819d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:50.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbed2a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db8101700 0x7f3db819dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:50.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbed2a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db8106dc0 0x7f3db8197990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:50.928 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbed2a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db8111640 con 0x7f3db8101700 2024-03-04T15:51:50.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbed2a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3db81114c0 con 0x7f3db8100d40 2024-03-04T15:51:50.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbed2a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3db81117c0 con 0x7f3db8106dc0 2024-03-04T15:51:50.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3db7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db8101700 0x7f3db819dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:50.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3db7fff700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db8101700 0x7f3db819dd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51684/0 (socket says 172.21.15.117:51684) 2024-03-04T15:51:50.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbd2c7700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db8106dc0 0x7f3db8197990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:50.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3db7fff700 1 -- 172.21.15.117:0/3242528767 learned_addr learned my addr 172.21.15.117:0/3242528767 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:51:50.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbcac6700 1 --2- 172.21.15.117:0/3242528767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3db8100d40 0x7f3db819d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:50.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3db7fff700 1 -- 172.21.15.117:0/3242528767 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db8106dc0 msgr2=0x7f3db8197990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:50.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3db7fff700 1 --2- 172.21.15.117:0/3242528767 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db8106dc0 0x7f3db8197990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:50.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3db7fff700 1 -- 172.21.15.117:0/3242528767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3db8100d40 msgr2=0x7f3db819d7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:50.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3db7fff700 1 --2- 172.21.15.117:0/3242528767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3db8100d40 0x7f3db819d7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:50.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3db7fff700 1 -- 172.21.15.117:0/3242528767 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db81980f0 con 0x7f3db8101700 2024-03-04T15:51:50.929 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbcac6700 1 --2- 172.21.15.117:0/3242528767 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3db8100d40 0x7f3db819d7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T15:51:50.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.923+0000 7f3dbd2c7700 1 --2- 172.21.15.117:0/3242528767 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3db8106dc0 0x7f3db8197990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:51:50.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.927+0000 7f3db7fff700 1 --2- 172.21.15.117:0/3242528767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db8101700 0x7f3db819dd10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00b070 tx=0x7f3dac007a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:50.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.927+0000 7f3db5ffb700 1 -- 172.21.15.117:0/3242528767 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac003c00 con 0x7f3db8101700 2024-03-04T15:51:50.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.927+0000 7f3db5ffb700 1 -- 172.21.15.117:0/3242528767 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f3dac022070 con 0x7f3db8101700 2024-03-04T15:51:50.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.927+0000 7f3dbed2a700 1 -- 172.21.15.117:0/3242528767 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db8198380 con 0x7f3db8101700 2024-03-04T15:51:50.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.927+0000 7f3dbed2a700 1 -- 172.21.15.117:0/3242528767 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3db81a46a0 con 0x7f3db8101700 2024-03-04T15:51:50.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.927+0000 7f3db5ffb700 1 -- 172.21.15.117:0/3242528767 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dac01b6c0 con 0x7f3db8101700 2024-03-04T15:51:50.930 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.927+0000 7f3dbed2a700 1 -- 172.21.15.117:0/3242528767 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3da4000f80 con 0x7f3db8101700 2024-03-04T15:51:50.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.927+0000 7f3db5ffb700 1 -- 172.21.15.117:0/3242528767 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f3dac01d470 con 0x7f3db8101700 2024-03-04T15:51:50.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.927+0000 7f3db5ffb700 1 --2- 172.21.15.117:0/3242528767 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f3da006d780 0x7f3da006fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:50.931 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.927+0000 7f3db5ffb700 1 -- 172.21.15.117:0/3242528767 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7f3dac00f020 con 0x7f3db8101700 2024-03-04T15:51:50.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.927+0000 7f3dbcac6700 1 --2- 172.21.15.117:0/3242528767 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f3da006d780 0x7f3da006fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:50.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.927+0000 7f3db5ffb700 1 --2- 172.21.15.117:0/3242528767 >> [v2:172.21.15.151:6816/3501440615,v1:172.21.15.151:6817/3501440615] conn(0x7f3da00776f0 0x7f3da0079b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:50.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.927+0000 7f3db5ffb700 1 -- 172.21.15.117:0/3242528767 --> [v2:172.21.15.151:6816/3501440615,v1:172.21.15.151:6817/3501440615] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f3da007a260 con 0x7f3da00776f0 2024-03-04T15:51:50.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.927+0000 7f3db5ffb700 1 -- 172.21.15.117:0/3242528767 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3dac08f1c0 con 0x7f3db8101700 2024-03-04T15:51:50.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.927+0000 7f3dbcac6700 1 --2- 172.21.15.117:0/3242528767 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f3da006d780 0x7f3da006fc40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3db8198ad0 tx=0x7f3da8008040 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:50.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.927+0000 7f3dbd2c7700 1 --2- 172.21.15.117:0/3242528767 >> [v2:172.21.15.151:6816/3501440615,v1:172.21.15.151:6817/3501440615] conn(0x7f3da00776f0 0x7f3da0079b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:50.932 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.927+0000 7f3dbd2c7700 1 --2- 172.21.15.117:0/3242528767 >> [v2:172.21.15.151:6816/3501440615,v1:172.21.15.151:6817/3501440615] conn(0x7f3da00776f0 0x7f3da0079b90 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:50.933 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.931+0000 7f3db5ffb700 1 -- 172.21.15.117:0/3242528767 <== osd.6 v2:172.21.15.151:6816/3501440615 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7f3da007a260 con 0x7f3da00776f0 2024-03-04T15:51:50.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.947+0000 7f3dbed2a700 1 -- 172.21.15.117:0/3242528767 --> [v2:172.21.15.151:6816/3501440615,v1:172.21.15.151:6817/3501440615] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3da4002d00 con 0x7f3da00776f0 2024-03-04T15:51:50.948 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.947+0000 7f3db5ffb700 1 -- 172.21.15.117:0/3242528767 <== osd.6 v2:172.21.15.151:6816/3501440615 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3da4002d00 con 0x7f3da00776f0 2024-03-04T15:51:50.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.947+0000 7f3dbed2a700 1 -- 172.21.15.117:0/3242528767 >> [v2:172.21.15.151:6816/3501440615,v1:172.21.15.151:6817/3501440615] conn(0x7f3da00776f0 msgr2=0x7f3da0079b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:50.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.947+0000 7f3dbed2a700 1 --2- 172.21.15.117:0/3242528767 >> [v2:172.21.15.151:6816/3501440615,v1:172.21.15.151:6817/3501440615] conn(0x7f3da00776f0 0x7f3da0079b90 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:50.949 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.947+0000 7f3dbed2a700 1 -- 172.21.15.117:0/3242528767 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f3da006d780 msgr2=0x7f3da006fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:50.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.947+0000 7f3dbed2a700 1 --2- 172.21.15.117:0/3242528767 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f3da006d780 0x7f3da006fc40 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f3db8198ad0 tx=0x7f3da8008040 comp rx=0 tx=0).stop 2024-03-04T15:51:50.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.947+0000 7f3dbed2a700 1 -- 172.21.15.117:0/3242528767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db8101700 msgr2=0x7f3db819dd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:50.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.947+0000 7f3dbed2a700 1 --2- 172.21.15.117:0/3242528767 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3db8101700 0x7f3db819dd10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f3dac00b070 tx=0x7f3dac007a00 comp rx=0 tx=0).stop 2024-03-04T15:51:50.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.947+0000 7f3dbd2c7700 1 -- 172.21.15.117:0/3242528767 reap_dead start 2024-03-04T15:51:50.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.947+0000 7f3dbed2a700 1 -- 172.21.15.117:0/3242528767 shutdown_connections 2024-03-04T15:51:50.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.947+0000 7f3dbed2a700 1 -- 172.21.15.117:0/3242528767 >> 172.21.15.117:0/3242528767 conn(0x7f3db8076d80 msgr2=0x7f3db8104900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:50.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.947+0000 7f3dbed2a700 1 -- 172.21.15.117:0/3242528767 shutdown_connections 2024-03-04T15:51:50.950 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:50.947+0000 7f3dbed2a700 1 -- 172.21.15.117:0/3242528767 wait complete. 2024-03-04T15:51:51.021 INFO:teuthology.orchestra.run.smithi117.stdout:77309411418 2024-03-04T15:51:51.022 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.2 2024-03-04T15:51:51.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[23756]: cluster 2024-03-04T15:51:49.355956+0000 mgr.y (mgr.24428) 70 : cluster 0 pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-04T15:51:51.171 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[23756]: audit 2024-03-04T15:51:50.466487+0000 mon.c (mon.2) 59 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:51:51.171 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.94367886Z level=info msg="Executing migration" id="add index role.org_id" 2024-03-04T15:51:51.172 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:50 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:50.985822542Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-03-04T15:51:51.172 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.027859952Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-03-04T15:51:51.172 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.103821977Z level=info msg="Executing migration" id="create team role table" 2024-03-04T15:51:51.172 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.145904367Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-03-04T15:51:51.172 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.171258531Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-03-04T15:51:51.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:51 smithi117 bash[28843]: cluster 2024-03-04T15:51:49.355956+0000 mgr.y (mgr.24428) 70 : cluster 0 pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-04T15:51:51.175 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:51 smithi117 bash[28843]: audit 2024-03-04T15:51:50.466487+0000 mon.c (mon.2) 59 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:51:51.175 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:51 smithi117 bash[21480]: cluster 2024-03-04T15:51:49.355956+0000 mgr.y (mgr.24428) 70 : cluster 0 pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-04T15:51:51.175 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:51 smithi117 bash[21480]: audit 2024-03-04T15:51:50.466487+0000 mon.c (mon.2) 59 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:51:51.424 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.221691955Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-03-04T15:51:51.424 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.272098349Z level=info msg="Executing migration" id="create user role table" 2024-03-04T15:51:51.424 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.322667955Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-03-04T15:51:51.424 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.348006404Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-03-04T15:51:51.424 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.373314922Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-03-04T15:51:51.424 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.398671708Z level=info msg="Executing migration" id="create builtin role table" 2024-03-04T15:51:51.424 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.423892284Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-03-04T15:51:51.506 INFO:teuthology.orchestra.run.smithi117.stdout:236223201316 2024-03-04T15:51:51.507 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.7 2024-03-04T15:51:51.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.611+0000 7fb051019700 1 -- 172.21.15.117:0/3363019918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb04c06a6a0 msgr2=0x7fb04c06aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:51.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.611+0000 7fb051019700 1 --2- 172.21.15.117:0/3363019918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb04c06a6a0 0x7fb04c06aa80 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb044009990 tx=0x7fb044009ca0 comp rx=0 tx=0).stop 2024-03-04T15:51:51.614 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.611+0000 7fb051019700 1 -- 172.21.15.117:0/3363019918 shutdown_connections 2024-03-04T15:51:51.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.611+0000 7fb051019700 1 --2- 172.21.15.117:0/3363019918 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb04c06a6a0 0x7fb04c06aa80 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:51.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.611+0000 7fb051019700 1 --2- 172.21.15.117:0/3363019918 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb04c107220 0x7fb04c10edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:51.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.611+0000 7fb051019700 1 --2- 172.21.15.117:0/3363019918 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb04c06b050 0x7fb04c106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:51.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.611+0000 7fb051019700 1 -- 172.21.15.117:0/3363019918 >> 172.21.15.117:0/3363019918 conn(0x7fb04c077de0 msgr2=0x7fb04c0781f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:51.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.611+0000 7fb051019700 1 -- 172.21.15.117:0/3363019918 shutdown_connections 2024-03-04T15:51:51.615 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.611+0000 7fb051019700 1 -- 172.21.15.117:0/3363019918 wait complete. 2024-03-04T15:51:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.611+0000 7fb051019700 1 Processor -- start 2024-03-04T15:51:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.611+0000 7fb051019700 1 -- start start 2024-03-04T15:51:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb051019700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb04c06a6a0 0x7fb04c100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb051019700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb04c06b050 0x7fb04c1012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb051019700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb04c107220 0x7fb04c101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb051019700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c111690 con 0x7fb04c06b050 2024-03-04T15:51:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb051019700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c111510 con 0x7fb04c06a6a0 2024-03-04T15:51:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb051019700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb04c111810 con 0x7fb04c107220 2024-03-04T15:51:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb04b59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb04c107220 0x7fb04c101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb04a59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb04c06b050 0x7fb04c1012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:51.617 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb04b59e700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb04c107220 0x7fb04c101810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:50186/0 (socket says 172.21.15.117:50186) 2024-03-04T15:51:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb04b59e700 1 -- 172.21.15.117:0/3053481795 learned_addr learned my addr 172.21.15.117:0/3053481795 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:51:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb04ad9d700 1 --2- 172.21.15.117:0/3053481795 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb04c06a6a0 0x7fb04c100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb04b59e700 1 -- 172.21.15.117:0/3053481795 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb04c06a6a0 msgr2=0x7fb04c100d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb04b59e700 1 --2- 172.21.15.117:0/3053481795 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fb04c06a6a0 0x7fb04c100d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb04b59e700 1 -- 172.21.15.117:0/3053481795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb04c06b050 msgr2=0x7fb04c1012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb04b59e700 1 --2- 172.21.15.117:0/3053481795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb04c06b050 0x7fb04c1012d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb04b59e700 1 -- 172.21.15.117:0/3053481795 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb04c1a6090 con 0x7fb04c107220 2024-03-04T15:51:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb04a59c700 1 --2- 172.21.15.117:0/3053481795 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fb04c06b050 0x7fb04c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:51:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb04b59e700 1 --2- 172.21.15.117:0/3053481795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb04c107220 0x7fb04c101810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00b970 tx=0x7fb03c00bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb043fff700 1 -- 172.21.15.117:0/3053481795 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c019070 con 0x7fb04c107220 2024-03-04T15:51:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb043fff700 1 -- 172.21.15.117:0/3053481795 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fb03c00cdb0 con 0x7fb04c107220 2024-03-04T15:51:51.619 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb051019700 1 -- 172.21.15.117:0/3053481795 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb04c1a6380 con 0x7fb04c107220 2024-03-04T15:51:51.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb043fff700 1 -- 172.21.15.117:0/3053481795 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb03c014dc0 con 0x7fb04c107220 2024-03-04T15:51:51.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb051019700 1 -- 172.21.15.117:0/3053481795 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb04c1a6870 con 0x7fb04c107220 2024-03-04T15:51:51.620 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb051019700 1 -- 172.21.15.117:0/3053481795 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb04c0ff510 con 0x7fb04c107220 2024-03-04T15:51:51.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb043fff700 1 -- 172.21.15.117:0/3053481795 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7fb03c01c070 con 0x7fb04c107220 2024-03-04T15:51:51.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb043fff700 1 --2- 172.21.15.117:0/3053481795 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fb03806d780 0x7fb03806fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:51.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb043fff700 1 -- 172.21.15.117:0/3053481795 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7fb03c08cdb0 con 0x7fb04c107220 2024-03-04T15:51:51.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb04ad9d700 1 --2- 172.21.15.117:0/3053481795 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fb03806d780 0x7fb03806fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:51.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb043fff700 1 --2- 172.21.15.117:0/3053481795 >> [v2:172.21.15.151:6800/1293012079,v1:172.21.15.151:6801/1293012079] conn(0x7fb038077690 0x7fb038079b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:51.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb043fff700 1 -- 172.21.15.117:0/3053481795 --> [v2:172.21.15.151:6800/1293012079,v1:172.21.15.151:6801/1293012079] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb03807a200 con 0x7fb038077690 2024-03-04T15:51:51.621 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb043fff700 1 -- 172.21.15.117:0/3053481795 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb03c08f4b0 con 0x7fb04c107220 2024-03-04T15:51:51.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb04ad9d700 1 --2- 172.21.15.117:0/3053481795 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fb03806d780 0x7fb03806fc40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb044006010 tx=0x7fb04401a040 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:51.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.615+0000 7fb04a59c700 1 --2- 172.21.15.117:0/3053481795 >> [v2:172.21.15.151:6800/1293012079,v1:172.21.15.151:6801/1293012079] conn(0x7fb038077690 0x7fb038079b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:51.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.619+0000 7fb04a59c700 1 --2- 172.21.15.117:0/3053481795 >> [v2:172.21.15.151:6800/1293012079,v1:172.21.15.151:6801/1293012079] conn(0x7fb038077690 0x7fb038079b30 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:51.622 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.619+0000 7fb043fff700 1 -- 172.21.15.117:0/3053481795 <== osd.4 v2:172.21.15.151:6800/1293012079 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fb03807a200 con 0x7fb038077690 2024-03-04T15:51:51.637 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.635+0000 7fb051019700 1 -- 172.21.15.117:0/3053481795 --> [v2:172.21.15.151:6800/1293012079,v1:172.21.15.151:6801/1293012079] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb04c1035b0 con 0x7fb038077690 2024-03-04T15:51:51.638 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.635+0000 7fb043fff700 1 -- 172.21.15.117:0/3053481795 <== osd.4 v2:172.21.15.151:6800/1293012079 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fb04c1035b0 con 0x7fb038077690 2024-03-04T15:51:51.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.635+0000 7fb051019700 1 -- 172.21.15.117:0/3053481795 >> [v2:172.21.15.151:6800/1293012079,v1:172.21.15.151:6801/1293012079] conn(0x7fb038077690 msgr2=0x7fb038079b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:51.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.635+0000 7fb051019700 1 --2- 172.21.15.117:0/3053481795 >> [v2:172.21.15.151:6800/1293012079,v1:172.21.15.151:6801/1293012079] conn(0x7fb038077690 0x7fb038079b30 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:51.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.635+0000 7fb051019700 1 -- 172.21.15.117:0/3053481795 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fb03806d780 msgr2=0x7fb03806fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:51.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.635+0000 7fb051019700 1 --2- 172.21.15.117:0/3053481795 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fb03806d780 0x7fb03806fc40 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb044006010 tx=0x7fb04401a040 comp rx=0 tx=0).stop 2024-03-04T15:51:51.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.635+0000 7fb051019700 1 -- 172.21.15.117:0/3053481795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb04c107220 msgr2=0x7fb04c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:51.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.635+0000 7fb051019700 1 --2- 172.21.15.117:0/3053481795 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fb04c107220 0x7fb04c101810 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb03c00b970 tx=0x7fb03c00bc80 comp rx=0 tx=0).stop 2024-03-04T15:51:51.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.635+0000 7fb04b59e700 1 -- 172.21.15.117:0/3053481795 reap_dead start 2024-03-04T15:51:51.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.639+0000 7fb051019700 1 -- 172.21.15.117:0/3053481795 shutdown_connections 2024-03-04T15:51:51.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.639+0000 7fb051019700 1 -- 172.21.15.117:0/3053481795 >> 172.21.15.117:0/3053481795 conn(0x7fb04c077de0 msgr2=0x7fb04c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:51.639 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.639+0000 7fb051019700 1 -- 172.21.15.117:0/3053481795 shutdown_connections 2024-03-04T15:51:51.640 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:51.639+0000 7fb051019700 1 -- 172.21.15.117:0/3053481795 wait complete. 2024-03-04T15:51:51.677 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.449285897Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-03-04T15:51:51.677 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.47457291Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-03-04T15:51:51.677 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.5166367Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-03-04T15:51:51.677 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.55868072Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-03-04T15:51:51.677 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.600795208Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-03-04T15:51:51.677 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.634708947Z level=info msg="Executing migration" id="add unique index role.uid" 2024-03-04T15:51:51.677 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.676843685Z level=info msg="Executing migration" id="create seed assignment table" 2024-03-04T15:51:51.930 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.719057052Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-03-04T15:51:51.930 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.744290434Z level=info msg="Executing migration" id="add column hidden to role table" 2024-03-04T15:51:51.930 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.786474949Z level=info msg="Executing migration" id="create query_history table v1" 2024-03-04T15:51:51.930 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.811804102Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-03-04T15:51:51.930 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.853822119Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-03-04T15:51:51.930 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.879092511Z level=info msg="Executing migration" id="teams permissions migration" 2024-03-04T15:51:51.930 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.904412284Z level=info msg="Executing migration" id="dashboard permissions" 2024-03-04T15:51:51.930 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.929714398Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-03-04T15:51:52.181 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.955068485Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-03-04T15:51:52.182 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:51 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:51.997079492Z level=info msg="Executing migration" id="alerting notification permissions" 2024-03-04T15:51:52.182 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.039244791Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-03-04T15:51:52.182 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.064635507Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-03-04T15:51:52.182 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.08984038Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-03-04T15:51:52.182 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.11522998Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-03-04T15:51:52.182 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.157313534Z level=info msg="Executing migration" id="create correlation table v1" 2024-03-04T15:51:52.471 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.199528488Z level=info msg="Executing migration" id="add index correlations.uid" 2024-03-04T15:51:52.471 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.249984753Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-03-04T15:51:52.471 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.283700866Z level=info msg="Executing migration" id="add correlation config column" 2024-03-04T15:51:52.472 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.317363789Z level=info msg="Executing migration" id="create entity_events table" 2024-03-04T15:51:52.472 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.351215415Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-03-04T15:51:52.472 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.376464495Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-03-04T15:51:52.472 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.376823592Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-03-04T15:51:52.472 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.403739013Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-03-04T15:51:52.472 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.404111367Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-03-04T15:51:52.472 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.429113446Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-03-04T15:51:52.472 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.471289365Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-03-04T15:51:52.749 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.496669752Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-03-04T15:51:52.750 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.522041902Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-03-04T15:51:52.750 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.564047064Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-03-04T15:51:52.750 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.589392425Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-03-04T15:51:52.750 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.614715105Z level=info msg="Executing migration" id="Drop public config table" 2024-03-04T15:51:52.750 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.656797869Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-03-04T15:51:52.750 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.682193439Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-03-04T15:51:52.750 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.707523581Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-03-04T15:51:52.750 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.749585914Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-03-04T15:51:52.974 INFO:teuthology.orchestra.run.smithi117.stdout:107374182478 2024-03-04T15:51:52.975 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.3 2024-03-04T15:51:53.019 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.774992071Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-03-04T15:51:53.019 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.808581201Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-03-04T15:51:53.019 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.850666515Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-03-04T15:51:53.019 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.876108807Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-03-04T15:51:53.019 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.90973673Z level=info msg="Executing migration" id="add share column" 2024-03-04T15:51:53.019 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.9601842Z level=info msg="Executing migration" id="create default alerting folders" 2024-03-04T15:51:53.019 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:52 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:52.993848186Z level=info msg="Executing migration" id="create file table" 2024-03-04T15:51:53.158 INFO:teuthology.orchestra.run.smithi117.stdout:55834574947 2024-03-04T15:51:53.158 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.1 2024-03-04T15:51:53.170 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:53 smithi117 bash[28843]: cluster 2024-03-04T15:51:51.357520+0000 mgr.y (mgr.24428) 71 : cluster 0 pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:53.170 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:53 smithi117 bash[21480]: cluster 2024-03-04T15:51:51.357520+0000 mgr.y (mgr.24428) 71 : cluster 0 pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:53.272 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[23756]: cluster 2024-03-04T15:51:51.357520+0000 mgr.y (mgr.24428) 71 : cluster 0 pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:53.272 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.044568094Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-03-04T15:51:53.272 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.069952248Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-03-04T15:51:53.272 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.095430911Z level=info msg="Executing migration" id="create file_meta table" 2024-03-04T15:51:53.273 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.120778875Z level=info msg="Executing migration" id="file table idx: path key" 2024-03-04T15:51:53.273 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.146063484Z level=info msg="Executing migration" id="set path collation in file table" 2024-03-04T15:51:53.273 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.188118086Z level=info msg="Executing migration" id="managed permissions migration" 2024-03-04T15:51:53.273 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.230143535Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-03-04T15:51:53.273 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.272215542Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-03-04T15:51:53.282 INFO:teuthology.orchestra.run.smithi117.stdout:201863462960 2024-03-04T15:51:53.282 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.6 2024-03-04T15:51:53.533 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.297631486Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-03-04T15:51:53.534 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.323226307Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-03-04T15:51:53.534 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.381768852Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-03-04T15:51:53.534 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.407031339Z level=info msg="Executing migration" id="update group index for alert rules" 2024-03-04T15:51:53.534 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.432313131Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-03-04T15:51:53.534 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.457589277Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-03-04T15:51:53.534 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.474591916Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-03-04T15:51:53.534 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.508275723Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-03-04T15:51:53.534 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.533615177Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-03-04T15:51:53.794 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.558912823Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-03-04T15:51:53.794 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.625887309Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-03-04T15:51:53.794 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.667923316Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-03-04T15:51:53.794 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.69324805Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-03-04T15:51:53.794 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.726970707Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-03-04T15:51:53.795 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.768938732Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-03-04T15:51:53.795 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.794224955Z level=info msg="Executing migration" id="create folder table" 2024-03-04T15:51:53.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.971+0000 7fd0d3244700 1 -- 172.21.15.117:0/2450452676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0cc107ed0 msgr2=0x7fd0cc1082b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:53.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.971+0000 7fd0d3244700 1 --2- 172.21.15.117:0/2450452676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0cc107ed0 0x7fd0cc1082b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00b0a0 tx=0x7fd0bc00b3b0 comp rx=0 tx=0).stop 2024-03-04T15:51:53.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.971+0000 7fd0d3244700 1 -- 172.21.15.117:0/2450452676 shutdown_connections 2024-03-04T15:51:53.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.971+0000 7fd0d3244700 1 --2- 172.21.15.117:0/2450452676 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0cc102830 0x7fd0cc10ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:53.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.971+0000 7fd0d3244700 1 --2- 172.21.15.117:0/2450452676 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0cc107ed0 0x7fd0cc1082b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:53.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.971+0000 7fd0d3244700 1 --2- 172.21.15.117:0/2450452676 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0cc101e90 0x7fd0cc1022f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:53.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.971+0000 7fd0d3244700 1 -- 172.21.15.117:0/2450452676 >> 172.21.15.117:0/2450452676 conn(0x7fd0cc078f90 msgr2=0x7fd0cc0793a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:53.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.971+0000 7fd0d3244700 1 -- 172.21.15.117:0/2450452676 shutdown_connections 2024-03-04T15:51:53.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.971+0000 7fd0d3244700 1 -- 172.21.15.117:0/2450452676 wait complete. 2024-03-04T15:51:53.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d3244700 1 Processor -- start 2024-03-04T15:51:53.975 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d3244700 1 -- start start 2024-03-04T15:51:53.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d3244700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0cc101e90 0x7fd0cc197cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:53.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d3244700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0cc102830 0x7fd0cc198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:53.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d3244700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0cc107ed0 0x7fd0cc19b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:53.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d3244700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc1116f0 con 0x7fd0cc101e90 2024-03-04T15:51:53.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d3244700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc111570 con 0x7fd0cc102830 2024-03-04T15:51:53.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d3244700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0cc111870 con 0x7fd0cc107ed0 2024-03-04T15:51:53.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d0fe0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0cc101e90 0x7fd0cc197cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:53.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d0fe0700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0cc101e90 0x7fd0cc197cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:51722/0 (socket says 172.21.15.117:51722) 2024-03-04T15:51:53.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d0fe0700 1 -- 172.21.15.117:0/417013096 learned_addr learned my addr 172.21.15.117:0/417013096 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:51:53.977 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d17e1700 1 --2- 172.21.15.117:0/417013096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0cc107ed0 0x7fd0cc19b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:53.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0cbfff700 1 --2- 172.21.15.117:0/417013096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0cc102830 0x7fd0cc198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:53.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d0fe0700 1 -- 172.21.15.117:0/417013096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0cc107ed0 msgr2=0x7fd0cc19b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:53.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d0fe0700 1 --2- 172.21.15.117:0/417013096 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd0cc107ed0 0x7fd0cc19b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:53.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d0fe0700 1 -- 172.21.15.117:0/417013096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0cc102830 msgr2=0x7fd0cc198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:53.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d0fe0700 1 --2- 172.21.15.117:0/417013096 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd0cc102830 0x7fd0cc198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:53.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d0fe0700 1 -- 172.21.15.117:0/417013096 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0cc19bc10 con 0x7fd0cc101e90 2024-03-04T15:51:53.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d0fe0700 1 --2- 172.21.15.117:0/417013096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0cc101e90 0x7fd0cc197cc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00b070 tx=0x7fd0bc007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:53.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0c9ffb700 1 -- 172.21.15.117:0/417013096 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc021020 con 0x7fd0cc101e90 2024-03-04T15:51:53.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d3244700 1 -- 172.21.15.117:0/417013096 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0cc19bea0 con 0x7fd0cc101e90 2024-03-04T15:51:53.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0c9ffb700 1 -- 172.21.15.117:0/417013096 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd0bc004c50 con 0x7fd0cc101e90 2024-03-04T15:51:53.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0c9ffb700 1 -- 172.21.15.117:0/417013096 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc005050 con 0x7fd0cc101e90 2024-03-04T15:51:53.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d3244700 1 -- 172.21.15.117:0/417013096 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0cc1a4870 con 0x7fd0cc101e90 2024-03-04T15:51:53.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0d3244700 1 -- 172.21.15.117:0/417013096 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd0cc0624f0 con 0x7fd0cc101e90 2024-03-04T15:51:53.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0c9ffb700 1 -- 172.21.15.117:0/417013096 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7fd0bc024020 con 0x7fd0cc101e90 2024-03-04T15:51:53.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0c9ffb700 1 --2- 172.21.15.117:0/417013096 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fd0b806d660 0x7fd0b806fb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:53.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0c9ffb700 1 -- 172.21.15.117:0/417013096 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7fd0bc00f020 con 0x7fd0cc101e90 2024-03-04T15:51:53.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.975+0000 7fd0cbfff700 1 --2- 172.21.15.117:0/417013096 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fd0b806d660 0x7fd0b806fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:53.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.979+0000 7fd0c9ffb700 1 --2- 172.21.15.117:0/417013096 >> [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120] conn(0x7fd0b80774f0 0x7fd0b8079990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:51:53.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.979+0000 7fd0c9ffb700 1 -- 172.21.15.117:0/417013096 --> [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd0bc019ec0 con 0x7fd0b80774f0 2024-03-04T15:51:53.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.979+0000 7fd0c9ffb700 1 -- 172.21.15.117:0/417013096 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_get_version_reply(handle=1 version=75) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd0bc095050 con 0x7fd0cc101e90 2024-03-04T15:51:53.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.979+0000 7fd0d17e1700 1 --2- 172.21.15.117:0/417013096 >> [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120] conn(0x7fd0b80774f0 0x7fd0b8079990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:51:53.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.979+0000 7fd0cbfff700 1 --2- 172.21.15.117:0/417013096 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fd0b806d660 0x7fd0b806fb20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4009680 tx=0x7fd0b4009260 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:53.980 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.979+0000 7fd0d17e1700 1 --2- 172.21.15.117:0/417013096 >> [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120] conn(0x7fd0b80774f0 0x7fd0b8079990 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:51:53.981 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.979+0000 7fd0c9ffb700 1 -- 172.21.15.117:0/417013096 <== osd.0 v2:172.21.15.117:6802/1332942120 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+27193 (crc 0 0 0) 0x7fd0bc019ec0 con 0x7fd0b80774f0 2024-03-04T15:51:53.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.995+0000 7fd0d3244700 1 -- 172.21.15.117:0/417013096 --> [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd0cc0628f0 con 0x7fd0b80774f0 2024-03-04T15:51:53.997 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.995+0000 7fd0c9ffb700 1 -- 172.21.15.117:0/417013096 <== osd.0 v2:172.21.15.117:6802/1332942120 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd0cc0628f0 con 0x7fd0b80774f0 2024-03-04T15:51:53.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.995+0000 7fd0d3244700 1 -- 172.21.15.117:0/417013096 >> [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120] conn(0x7fd0b80774f0 msgr2=0x7fd0b8079990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:53.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.995+0000 7fd0d3244700 1 --2- 172.21.15.117:0/417013096 >> [v2:172.21.15.117:6802/1332942120,v1:172.21.15.117:6803/1332942120] conn(0x7fd0b80774f0 0x7fd0b8079990 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:51:53.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.995+0000 7fd0d3244700 1 -- 172.21.15.117:0/417013096 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fd0b806d660 msgr2=0x7fd0b806fb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:53.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.995+0000 7fd0d3244700 1 --2- 172.21.15.117:0/417013096 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fd0b806d660 0x7fd0b806fb20 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fd0b4009680 tx=0x7fd0b4009260 comp rx=0 tx=0).stop 2024-03-04T15:51:53.998 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.995+0000 7fd0d3244700 1 -- 172.21.15.117:0/417013096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0cc101e90 msgr2=0x7fd0cc197cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:51:53.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.995+0000 7fd0d3244700 1 --2- 172.21.15.117:0/417013096 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd0cc101e90 0x7fd0cc197cc0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00b070 tx=0x7fd0bc007a20 comp rx=0 tx=0).stop 2024-03-04T15:51:53.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.995+0000 7fd0d17e1700 1 -- 172.21.15.117:0/417013096 reap_dead start 2024-03-04T15:51:53.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.995+0000 7fd0d3244700 1 -- 172.21.15.117:0/417013096 shutdown_connections 2024-03-04T15:51:53.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.995+0000 7fd0d3244700 1 -- 172.21.15.117:0/417013096 >> 172.21.15.117:0/417013096 conn(0x7fd0cc078f90 msgr2=0x7fd0cc100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:51:53.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.995+0000 7fd0d3244700 1 -- 172.21.15.117:0/417013096 shutdown_connections 2024-03-04T15:51:53.999 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:51:53.995+0000 7fd0d3244700 1 -- 172.21.15.117:0/417013096 wait complete. 2024-03-04T15:51:54.079 INFO:teuthology.orchestra.run.smithi117.stdout:133143986246 2024-03-04T15:51:54.080 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.4 2024-03-04T15:51:54.157 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.836318356Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-03-04T15:51:54.157 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.861689303Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-03-04T15:51:54.157 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.895552125Z level=info msg="Executing migration" id="Update folder title length" 2024-03-04T15:51:54.157 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.920858963Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-03-04T15:51:54.157 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=migrator t=2024-03-04T15:51:53.962961747Z level=info msg="migrations completed" performed=485 skipped=0 duration=17.344661124s 2024-03-04T15:51:54.157 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:53 smithi151 bash[51061]: logger=sqlstore t=2024-03-04T15:51:53.964712004Z level=info msg="Created default organization" 2024-03-04T15:51:54.157 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:54 smithi151 bash[51061]: logger=plugin.loader t=2024-03-04T15:51:54.156488625Z level=info msg="Plugin registered" pluginID=input 2024-03-04T15:51:54.431 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:54 smithi151 bash[51061]: logger=plugin.loader t=2024-03-04T15:51:54.197452173Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-03-04T15:51:54.431 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:54 smithi151 bash[51061]: logger=plugin.loader t=2024-03-04T15:51:54.197482744Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-03-04T15:51:54.432 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:54 smithi151 bash[51061]: logger=secrets t=2024-03-04T15:51:54.197527387Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-03-04T15:51:54.432 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:54 smithi151 bash[51061]: logger=query_data t=2024-03-04T15:51:54.200402406Z level=info msg="Query Service initialization" 2024-03-04T15:51:54.432 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:54 smithi151 bash[51061]: logger=live.push_http t=2024-03-04T15:51:54.203724032Z level=info msg="Live Push Gateway initialization" 2024-03-04T15:51:54.763 INFO:teuthology.orchestra.run.smithi117.stdout:34359738478 2024-03-04T15:51:54.764 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph osd last-stat-seq osd.0 2024-03-04T15:51:55.273 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:55 smithi151 bash[23756]: cluster 2024-03-04T15:51:53.358013+0000 mgr.y (mgr.24428) 72 : cluster 0 pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:55.273 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:55 smithi151 bash[51061]: logger=infra.usagestats.collector t=2024-03-04T15:51:55.272297758Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-03-04T15:51:55.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:55 smithi117 bash[28843]: cluster 2024-03-04T15:51:53.358013+0000 mgr.y (mgr.24428) 72 : cluster 0 pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:55.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:55 smithi117 bash[21480]: cluster 2024-03-04T15:51:53.358013+0000 mgr.y (mgr.24428) 72 : cluster 0 pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:55.542 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:55 smithi151 bash[51061]: logger=provisioning.datasources t=2024-03-04T15:51:55.273553952Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-03-04T15:51:55.542 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:55 smithi151 bash[51061]: logger=provisioning.datasources t=2024-03-04T15:51:55.31440479Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-03-04T15:51:55.542 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:55 smithi151 bash[51061]: logger=provisioning.alerting t=2024-03-04T15:51:55.348414498Z level=info msg="starting to provision alerting" 2024-03-04T15:51:55.542 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:55 smithi151 bash[51061]: logger=provisioning.alerting t=2024-03-04T15:51:55.348440203Z level=info msg="finished to provision alerting" 2024-03-04T15:51:55.542 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:55 smithi151 bash[51061]: logger=grafanaStorageLogger t=2024-03-04T15:51:55.348687161Z level=info msg="storage starting" 2024-03-04T15:51:55.542 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:55 smithi151 bash[51061]: logger=ngalert.state.manager t=2024-03-04T15:51:55.349414863Z level=info msg="Warming state cache for startup" 2024-03-04T15:51:55.542 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:55 smithi151 bash[51061]: logger=http.server t=2024-03-04T15:51:55.351544011Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-03-04T15:51:55.542 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:55 smithi151 bash[51061]: logger=sqlstore.transactions t=2024-03-04T15:51:55.445511445Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-03-04T15:51:55.543 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:55 smithi151 bash[51061]: logger=ngalert.state.manager t=2024-03-04T15:51:55.541658926Z level=info msg="State cache has been initialized" states=0 duration=192.241544ms 2024-03-04T15:51:55.543 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:55 smithi151 bash[51061]: logger=ngalert.multiorg.alertmanager t=2024-03-04T15:51:55.541765728Z level=info msg="starting MultiOrg Alertmanager" 2024-03-04T15:51:55.543 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 15:51:55 smithi151 bash[51061]: logger=ticker t=2024-03-04T15:51:55.541817224Z level=info msg=starting first_tick=2024-03-04T15:52:00Z 2024-03-04T15:51:56.569 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:51:56 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:51:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:51:56.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:51:56 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:51:57.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:56 smithi117 bash[21480]: cluster 2024-03-04T15:51:55.358636+0000 mgr.y (mgr.24428) 73 : cluster 0 pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:57.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:56 smithi117 bash[21480]: audit 2024-03-04T15:51:55.779219+0000 mon.a (mon.0) 825 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:57.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:56 smithi117 bash[21480]: audit 2024-03-04T15:51:55.792439+0000 mon.a (mon.0) 826 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:57.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:56 smithi117 bash[21480]: audit 2024-03-04T15:51:56.368421+0000 mon.c (mon.2) 60 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:51:57.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:56 smithi117 bash[21480]: audit 2024-03-04T15:51:56.370104+0000 mon.c (mon.2) 61 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:51:57.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:56 smithi117 bash[21480]: audit 2024-03-04T15:51:56.381862+0000 mon.a (mon.0) 827 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:57.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:56 smithi117 bash[28843]: cluster 2024-03-04T15:51:55.358636+0000 mgr.y (mgr.24428) 73 : cluster 0 pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:57.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:56 smithi117 bash[28843]: audit 2024-03-04T15:51:55.779219+0000 mon.a (mon.0) 825 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:57.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:56 smithi117 bash[28843]: audit 2024-03-04T15:51:55.792439+0000 mon.a (mon.0) 826 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:57.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:56 smithi117 bash[28843]: audit 2024-03-04T15:51:56.368421+0000 mon.c (mon.2) 60 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:51:57.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:56 smithi117 bash[28843]: audit 2024-03-04T15:51:56.370104+0000 mon.c (mon.2) 61 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:51:57.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:56 smithi117 bash[28843]: audit 2024-03-04T15:51:56.381862+0000 mon.a (mon.0) 827 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:57.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:56 smithi151 bash[23756]: cluster 2024-03-04T15:51:55.358636+0000 mgr.y (mgr.24428) 73 : cluster 0 pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:51:57.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:56 smithi151 bash[23756]: audit 2024-03-04T15:51:55.779219+0000 mon.a (mon.0) 825 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:57.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:56 smithi151 bash[23756]: audit 2024-03-04T15:51:55.792439+0000 mon.a (mon.0) 826 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:57.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:56 smithi151 bash[23756]: audit 2024-03-04T15:51:56.368421+0000 mon.c (mon.2) 60 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:51:57.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:56 smithi151 bash[23756]: audit 2024-03-04T15:51:56.370104+0000 mon.c (mon.2) 61 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:51:57.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:56 smithi151 bash[23756]: audit 2024-03-04T15:51:56.381862+0000 mon.a (mon.0) 827 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:51:57.516 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:51:57.517 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:51:57.517 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:51:57.518 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:51:57.521 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:51:57.521 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:51:57.792 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:57 smithi117 bash[21480]: audit 2024-03-04T15:51:56.348657+0000 mgr.y (mgr.24428) 74 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:51:57.792 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:57 smithi117 bash[21480]: cephadm 2024-03-04T15:51:56.460662+0000 mgr.y (mgr.24428) 75 : cephadm 1 Reconfiguring alertmanager.a (dependencies changed)... 2024-03-04T15:51:57.792 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:57 smithi117 bash[21480]: cephadm 2024-03-04T15:51:56.470356+0000 mgr.y (mgr.24428) 76 : cephadm 1 Reconfiguring daemon alertmanager.a on smithi117 2024-03-04T15:51:58.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:57 smithi117 bash[28843]: audit 2024-03-04T15:51:56.348657+0000 mgr.y (mgr.24428) 74 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:51:58.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:57 smithi117 bash[28843]: cephadm 2024-03-04T15:51:56.460662+0000 mgr.y (mgr.24428) 75 : cephadm 1 Reconfiguring alertmanager.a (dependencies changed)... 2024-03-04T15:51:58.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:57 smithi117 bash[28843]: cephadm 2024-03-04T15:51:56.470356+0000 mgr.y (mgr.24428) 76 : cephadm 1 Reconfiguring daemon alertmanager.a on smithi117 2024-03-04T15:51:58.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:57 smithi151 bash[23756]: audit 2024-03-04T15:51:56.348657+0000 mgr.y (mgr.24428) 74 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:51:58.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:57 smithi151 bash[23756]: cephadm 2024-03-04T15:51:56.460662+0000 mgr.y (mgr.24428) 75 : cephadm 1 Reconfiguring alertmanager.a (dependencies changed)... 2024-03-04T15:51:58.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:57 smithi151 bash[23756]: cephadm 2024-03-04T15:51:56.470356+0000 mgr.y (mgr.24428) 76 : cephadm 1 Reconfiguring daemon alertmanager.a on smithi117 2024-03-04T15:51:58.538 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:51:58.540 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:51:59.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:51:58 smithi117 bash[28843]: cluster 2024-03-04T15:51:57.360116+0000 mgr.y (mgr.24428) 77 : cluster 0 pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:59.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:51:58 smithi117 bash[21480]: cluster 2024-03-04T15:51:57.360116+0000 mgr.y (mgr.24428) 77 : cluster 0 pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:51:59.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:51:58 smithi151 bash[23756]: cluster 2024-03-04T15:51:57.360116+0000 mgr.y (mgr.24428) 77 : cluster 0 pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:01.107 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:00 smithi117 bash[28843]: cluster 2024-03-04T15:51:59.360807+0000 mgr.y (mgr.24428) 78 : cluster 0 pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:01.108 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:00 smithi117 bash[21480]: cluster 2024-03-04T15:51:59.360807+0000 mgr.y (mgr.24428) 78 : cluster 0 pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:01.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:00 smithi151 bash[23756]: cluster 2024-03-04T15:51:59.360807+0000 mgr.y (mgr.24428) 78 : cluster 0 pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:03.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:02 smithi117 bash[28843]: cluster 2024-03-04T15:52:01.362175+0000 mgr.y (mgr.24428) 79 : cluster 0 pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T15:52:03.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:02 smithi117 bash[21480]: cluster 2024-03-04T15:52:01.362175+0000 mgr.y (mgr.24428) 79 : cluster 0 pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T15:52:03.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:02 smithi151 bash[23756]: cluster 2024-03-04T15:52:01.362175+0000 mgr.y (mgr.24428) 79 : cluster 0 pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T15:52:05.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:04 smithi117 bash[21480]: cluster 2024-03-04T15:52:03.362627+0000 mgr.y (mgr.24428) 80 : cluster 0 pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:05.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:04 smithi117 bash[28843]: cluster 2024-03-04T15:52:03.362627+0000 mgr.y (mgr.24428) 80 : cluster 0 pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:05.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:04 smithi151 bash[23756]: cluster 2024-03-04T15:52:03.362627+0000 mgr.y (mgr.24428) 80 : cluster 0 pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:05.950 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:05 smithi117 bash[21480]: audit 2024-03-04T15:52:05.497363+0000 mon.c (mon.2) 62 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:52:05.951 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:05 smithi117 bash[28843]: audit 2024-03-04T15:52:05.497363+0000 mon.c (mon.2) 62 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:52:06.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:05 smithi151 bash[23756]: audit 2024-03-04T15:52:05.497363+0000 mon.c (mon.2) 62 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:52:06.319 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:06 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:52:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:52:06.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:52:06 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:52:07.125 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:06 smithi117 bash[21480]: cluster 2024-03-04T15:52:05.363213+0000 mgr.y (mgr.24428) 81 : cluster 0 pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:07.125 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:06 smithi117 bash[28843]: cluster 2024-03-04T15:52:05.363213+0000 mgr.y (mgr.24428) 81 : cluster 0 pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:07.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:06 smithi151 bash[23756]: cluster 2024-03-04T15:52:05.363213+0000 mgr.y (mgr.24428) 81 : cluster 0 pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:08.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:07 smithi151 bash[23756]: audit 2024-03-04T15:52:06.350150+0000 mgr.y (mgr.24428) 82 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:52:08.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:07 smithi117 bash[28843]: audit 2024-03-04T15:52:06.350150+0000 mgr.y (mgr.24428) 82 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:52:08.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:07 smithi117 bash[21480]: audit 2024-03-04T15:52:06.350150+0000 mgr.y (mgr.24428) 82 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:52:09.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.054+0000 7f3a67619700 1 -- 172.21.15.117:0/16714168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a60104c30 msgr2=0x7f3a60108c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:09.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.054+0000 7f3a67619700 1 --2- 172.21.15.117:0/16714168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a60104c30 0x7f3a60108c80 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3a5c0099c0 tx=0x7f3a5c009cd0 comp rx=0 tx=0).stop 2024-03-04T15:52:09.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.054+0000 7f3a67619700 1 -- 172.21.15.117:0/16714168 shutdown_connections 2024-03-04T15:52:09.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.054+0000 7f3a67619700 1 --2- 172.21.15.117:0/16714168 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a60109360 0x7f3a60110f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:09.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.054+0000 7f3a67619700 1 --2- 172.21.15.117:0/16714168 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a60104280 0x7f3a60104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:09.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.054+0000 7f3a67619700 1 --2- 172.21.15.117:0/16714168 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a60104c30 0x7f3a60108c80 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:09.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.054+0000 7f3a67619700 1 -- 172.21.15.117:0/16714168 >> 172.21.15.117:0/16714168 conn(0x7f3a600ffb10 msgr2=0x7f3a60101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:09.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.054+0000 7f3a67619700 1 -- 172.21.15.117:0/16714168 shutdown_connections 2024-03-04T15:52:09.058 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.054+0000 7f3a67619700 1 -- 172.21.15.117:0/16714168 wait complete. 2024-03-04T15:52:09.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.054+0000 7f3a67619700 1 Processor -- start 2024-03-04T15:52:09.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.054+0000 7f3a67619700 1 -- start start 2024-03-04T15:52:09.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.054+0000 7f3a67619700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a60104280 0x7f3a60199d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:09.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.054+0000 7f3a67619700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a60104c30 0x7f3a6019a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:09.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.054+0000 7f3a67619700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a60109360 0x7f3a6019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:09.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.054+0000 7f3a67619700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a601137d0 con 0x7f3a60104280 2024-03-04T15:52:09.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.054+0000 7f3a67619700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a60113650 con 0x7f3a60104c30 2024-03-04T15:52:09.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.054+0000 7f3a67619700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a60113950 con 0x7f3a60109360 2024-03-04T15:52:09.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a653b5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a60104280 0x7f3a60199d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:09.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a653b5700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a60104280 0x7f3a60199d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50546/0 (socket says 172.21.15.117:50546) 2024-03-04T15:52:09.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a653b5700 1 -- 172.21.15.117:0/1542725212 learned_addr learned my addr 172.21.15.117:0/1542725212 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:52:09.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a65bb6700 1 --2- 172.21.15.117:0/1542725212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a60109360 0x7f3a6019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:09.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a64bb4700 1 --2- 172.21.15.117:0/1542725212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a60104c30 0x7f3a6019a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:09.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a65bb6700 1 -- 172.21.15.117:0/1542725212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a60104c30 msgr2=0x7f3a6019a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:09.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a65bb6700 1 --2- 172.21.15.117:0/1542725212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a60104c30 0x7f3a6019a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:09.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a65bb6700 1 -- 172.21.15.117:0/1542725212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a60104280 msgr2=0x7f3a60199d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:09.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a65bb6700 1 --2- 172.21.15.117:0/1542725212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a60104280 0x7f3a60199d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:09.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a65bb6700 1 -- 172.21.15.117:0/1542725212 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a6019ed60 con 0x7f3a60109360 2024-03-04T15:52:09.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a653b5700 1 --2- 172.21.15.117:0/1542725212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a60104280 0x7f3a60199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:52:09.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a65bb6700 1 --2- 172.21.15.117:0/1542725212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a60109360 0x7f3a6019e620 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800efc0 tx=0x7f3a5800c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:09.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a567fc700 1 -- 172.21.15.117:0/1542725212 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a580092e0 con 0x7f3a60109360 2024-03-04T15:52:09.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a567fc700 1 -- 172.21.15.117:0/1542725212 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f3a58004bb0 con 0x7f3a60109360 2024-03-04T15:52:09.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a567fc700 1 -- 172.21.15.117:0/1542725212 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a58004020 con 0x7f3a60109360 2024-03-04T15:52:09.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a67619700 1 -- 172.21.15.117:0/1542725212 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6019eff0 con 0x7f3a60109360 2024-03-04T15:52:09.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a67619700 1 -- 172.21.15.117:0/1542725212 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a601a68d0 con 0x7f3a60109360 2024-03-04T15:52:09.062 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a67619700 1 -- 172.21.15.117:0/1542725212 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a60105760 con 0x7f3a60109360 2024-03-04T15:52:09.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a567fc700 1 -- 172.21.15.117:0/1542725212 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f3a5801b030 con 0x7f3a60109360 2024-03-04T15:52:09.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a567fc700 1 --2- 172.21.15.117:0/1542725212 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f3a4c06d780 0x7f3a4c06fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:09.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a567fc700 1 -- 172.21.15.117:0/1542725212 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7f3a5808d560 con 0x7f3a60109360 2024-03-04T15:52:09.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a653b5700 1 --2- 172.21.15.117:0/1542725212 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f3a4c06d780 0x7f3a4c06fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:09.063 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.058+0000 7f3a653b5700 1 --2- 172.21.15.117:0/1542725212 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f3a4c06d780 0x7f3a4c06fc40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3a50005950 tx=0x7f3a5000a400 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:09.067 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.066+0000 7f3a567fc700 1 -- 172.21.15.117:0/1542725212 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f3a58022020 con 0x7f3a60109360 2024-03-04T15:52:09.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:08 smithi151 bash[23756]: cluster 2024-03-04T15:52:07.364688+0000 mgr.y (mgr.24428) 83 : cluster 0 pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:09.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.274+0000 7f3a67619700 1 -- 172.21.15.117:0/1542725212 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f3a6019b0b0 con 0x7f3a60109360 2024-03-04T15:52:09.278 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.274+0000 7f3a567fc700 1 -- 172.21.15.117:0/1542725212 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3a58010040 con 0x7f3a60109360 2024-03-04T15:52:09.279 INFO:teuthology.orchestra.run.smithi117.stdout:107374182482 2024-03-04T15:52:09.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.278+0000 7f3a67619700 1 -- 172.21.15.117:0/1542725212 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f3a4c06d780 msgr2=0x7f3a4c06fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:09.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.278+0000 7f3a67619700 1 --2- 172.21.15.117:0/1542725212 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f3a4c06d780 0x7f3a4c06fc40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f3a50005950 tx=0x7f3a5000a400 comp rx=0 tx=0).stop 2024-03-04T15:52:09.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.278+0000 7f3a67619700 1 -- 172.21.15.117:0/1542725212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a60109360 msgr2=0x7f3a6019e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:09.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.278+0000 7f3a67619700 1 --2- 172.21.15.117:0/1542725212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a60109360 0x7f3a6019e620 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f3a5800efc0 tx=0x7f3a5800c3a0 comp rx=0 tx=0).stop 2024-03-04T15:52:09.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.278+0000 7f3a67619700 1 -- 172.21.15.117:0/1542725212 shutdown_connections 2024-03-04T15:52:09.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.278+0000 7f3a67619700 1 --2- 172.21.15.117:0/1542725212 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f3a4c06d780 0x7f3a4c06fc40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:09.281 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.278+0000 7f3a67619700 1 --2- 172.21.15.117:0/1542725212 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3a60104280 0x7f3a60199d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:09.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.278+0000 7f3a67619700 1 --2- 172.21.15.117:0/1542725212 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3a60109360 0x7f3a6019e620 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:09.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.278+0000 7f3a67619700 1 --2- 172.21.15.117:0/1542725212 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3a60104c30 0x7f3a6019a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:09.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.278+0000 7f3a67619700 1 -- 172.21.15.117:0/1542725212 >> 172.21.15.117:0/1542725212 conn(0x7f3a600ffb10 msgr2=0x7f3a601000d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:09.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.278+0000 7f3a67619700 1 -- 172.21.15.117:0/1542725212 shutdown_connections 2024-03-04T15:52:09.282 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:09.278+0000 7f3a67619700 1 -- 172.21.15.117:0/1542725212 wait complete. 2024-03-04T15:52:09.298 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:08 smithi117 bash[28843]: cluster 2024-03-04T15:52:07.364688+0000 mgr.y (mgr.24428) 83 : cluster 0 pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:09.298 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:08 smithi117 bash[21480]: cluster 2024-03-04T15:52:07.364688+0000 mgr.y (mgr.24428) 83 : cluster 0 pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:10.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:09 smithi151 bash[23756]: audit 2024-03-04T15:52:09.280746+0000 mon.c (mon.2) 63 : audit 0 from='client.? 172.21.15.117:0/1542725212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-04T15:52:10.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:09 smithi117 bash[21480]: audit 2024-03-04T15:52:09.280746+0000 mon.c (mon.2) 63 : audit 0 from='client.? 172.21.15.117:0/1542725212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-04T15:52:10.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:09 smithi117 bash[28843]: audit 2024-03-04T15:52:09.280746+0000 mon.c (mon.2) 63 : audit 0 from='client.? 172.21.15.117:0/1542725212' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-03-04T15:52:10.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0707987700 1 -- 172.21.15.117:0/532996326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f070006a4b0 msgr2=0x7f070010ecd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:10.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0707987700 1 --2- 172.21.15.117:0/532996326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f070006a4b0 0x7f070010ecd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f06f8009f90 tx=0x7f06f800b330 comp rx=0 tx=0).stop 2024-03-04T15:52:10.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0707987700 1 -- 172.21.15.117:0/532996326 shutdown_connections 2024-03-04T15:52:10.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0707987700 1 --2- 172.21.15.117:0/532996326 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f070006a4b0 0x7f070010ecd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:10.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0707987700 1 --2- 172.21.15.117:0/532996326 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0700107e90 0x7f0700108270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:10.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0707987700 1 --2- 172.21.15.117:0/532996326 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0700069b10 0x7f0700069f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:10.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0707987700 1 -- 172.21.15.117:0/532996326 >> 172.21.15.117:0/532996326 conn(0x7f0700076d80 msgr2=0x7f0700077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:10.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0707987700 1 -- 172.21.15.117:0/532996326 shutdown_connections 2024-03-04T15:52:10.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0707987700 1 -- 172.21.15.117:0/532996326 wait complete. 2024-03-04T15:52:10.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0707987700 1 Processor -- start 2024-03-04T15:52:10.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0707987700 1 -- start start 2024-03-04T15:52:10.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0707987700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0700069b10 0x7f070019d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:10.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0707987700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f070006a4b0 0x7f070019dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:10.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0707987700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0700107e90 0x7f0700197980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:10.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0707987700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0700111660 con 0x7f070006a4b0 2024-03-04T15:52:10.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0707987700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07001114e0 con 0x7f0700069b10 2024-03-04T15:52:10.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0707987700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07001117e0 con 0x7f0700107e90 2024-03-04T15:52:10.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0705f24700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0700107e90 0x7f0700197980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:10.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0704f22700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f070006a4b0 0x7f070019dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:10.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0704f22700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f070006a4b0 0x7f070019dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50560/0 (socket says 172.21.15.117:50560) 2024-03-04T15:52:10.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0705f24700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0700107e90 0x7f0700197980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:57798/0 (socket says 172.21.15.117:57798) 2024-03-04T15:52:10.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0704f22700 1 -- 172.21.15.117:0/1794332256 learned_addr learned my addr 172.21.15.117:0/1794332256 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:52:10.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0705723700 1 --2- 172.21.15.117:0/1794332256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0700069b10 0x7f070019d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:10.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0704f22700 1 -- 172.21.15.117:0/1794332256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0700107e90 msgr2=0x7f0700197980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:10.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0704f22700 1 --2- 172.21.15.117:0/1794332256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0700107e90 0x7f0700197980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:10.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0704f22700 1 -- 172.21.15.117:0/1794332256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0700069b10 msgr2=0x7f070019d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:10.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0704f22700 1 --2- 172.21.15.117:0/1794332256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0700069b10 0x7f070019d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:10.972 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0704f22700 1 -- 172.21.15.117:0/1794332256 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07001980e0 con 0x7f070006a4b0 2024-03-04T15:52:10.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.966+0000 7f0705f24700 1 --2- 172.21.15.117:0/1794332256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0700107e90 0x7f0700197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:52:10.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.970+0000 7f0704f22700 1 --2- 172.21.15.117:0/1794332256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f070006a4b0 0x7f070019dd00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f06f000ec10 tx=0x7f06f000ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:10.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.970+0000 7f06f67fc700 1 -- 172.21.15.117:0/1794332256 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f000cd40 con 0x7f070006a4b0 2024-03-04T15:52:10.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.970+0000 7f06f67fc700 1 -- 172.21.15.117:0/1794332256 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f06f00044a0 con 0x7f070006a4b0 2024-03-04T15:52:10.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.970+0000 7f06f67fc700 1 -- 172.21.15.117:0/1794332256 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f0010430 con 0x7f070006a4b0 2024-03-04T15:52:10.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.970+0000 7f0707987700 1 -- 172.21.15.117:0/1794332256 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07001983d0 con 0x7f070006a4b0 2024-03-04T15:52:10.973 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.970+0000 7f0707987700 1 -- 172.21.15.117:0/1794332256 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07001a4690 con 0x7f070006a4b0 2024-03-04T15:52:10.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.970+0000 7f0707987700 1 -- 172.21.15.117:0/1794332256 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f070010a400 con 0x7f070006a4b0 2024-03-04T15:52:10.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.970+0000 7f06f67fc700 1 -- 172.21.15.117:0/1794332256 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f06f0003f70 con 0x7f070006a4b0 2024-03-04T15:52:10.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.970+0000 7f06f67fc700 1 --2- 172.21.15.117:0/1794332256 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f06ec06d780 0x7f06ec06fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:10.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.970+0000 7f06f67fc700 1 -- 172.21.15.117:0/1794332256 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7f06f008dd50 con 0x7f070006a4b0 2024-03-04T15:52:10.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.970+0000 7f0705723700 1 --2- 172.21.15.117:0/1794332256 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f06ec06d780 0x7f06ec06fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:10.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.970+0000 7f0705723700 1 --2- 172.21.15.117:0/1794332256 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f06ec06d780 0x7f06ec06fc40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f06fc006fd0 tx=0x7f06fc008040 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:10.978 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:10.974+0000 7f06f67fc700 1 -- 172.21.15.117:0/1794332256 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f06f0091030 con 0x7f070006a4b0 2024-03-04T15:52:11.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:10 smithi151 bash[23756]: cluster 2024-03-04T15:52:09.365182+0000 mgr.y (mgr.24428) 84 : cluster 0 pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:11.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.194+0000 7f0707987700 1 -- 172.21.15.117:0/1794332256 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f0700198eb0 con 0x7f070006a4b0 2024-03-04T15:52:11.195 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.194+0000 7f06f67fc700 1 -- 172.21.15.117:0/1794332256 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f06f005b620 con 0x7f070006a4b0 2024-03-04T15:52:11.196 INFO:teuthology.orchestra.run.smithi117.stdout:201863462964 2024-03-04T15:52:11.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.194+0000 7f0707987700 1 -- 172.21.15.117:0/1794332256 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f06ec06d780 msgr2=0x7f06ec06fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:11.198 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.194+0000 7f0707987700 1 --2- 172.21.15.117:0/1794332256 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f06ec06d780 0x7f06ec06fc40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f06fc006fd0 tx=0x7f06fc008040 comp rx=0 tx=0).stop 2024-03-04T15:52:11.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.194+0000 7f0707987700 1 -- 172.21.15.117:0/1794332256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f070006a4b0 msgr2=0x7f070019dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:11.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.194+0000 7f0707987700 1 --2- 172.21.15.117:0/1794332256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f070006a4b0 0x7f070019dd00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f06f000ec10 tx=0x7f06f000ef20 comp rx=0 tx=0).stop 2024-03-04T15:52:11.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.194+0000 7f0707987700 1 -- 172.21.15.117:0/1794332256 shutdown_connections 2024-03-04T15:52:11.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.194+0000 7f0707987700 1 --2- 172.21.15.117:0/1794332256 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f0700107e90 0x7f0700197980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:11.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.194+0000 7f0707987700 1 --2- 172.21.15.117:0/1794332256 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f0700069b10 0x7f070019d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:11.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.194+0000 7f0707987700 1 --2- 172.21.15.117:0/1794332256 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f06ec06d780 0x7f06ec06fc40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:11.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.194+0000 7f0707987700 1 --2- 172.21.15.117:0/1794332256 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f070006a4b0 0x7f070019dd00 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:11.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.194+0000 7f0707987700 1 -- 172.21.15.117:0/1794332256 >> 172.21.15.117:0/1794332256 conn(0x7f0700076d80 msgr2=0x7f07000ffa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:11.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.194+0000 7f0707987700 1 -- 172.21.15.117:0/1794332256 shutdown_connections 2024-03-04T15:52:11.199 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.198+0000 7f0707987700 1 -- 172.21.15.117:0/1794332256 wait complete. 2024-03-04T15:52:11.211 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:10 smithi117 bash[21480]: cluster 2024-03-04T15:52:09.365182+0000 mgr.y (mgr.24428) 84 : cluster 0 pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:11.211 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:10 smithi117 bash[28843]: cluster 2024-03-04T15:52:09.365182+0000 mgr.y (mgr.24428) 84 : cluster 0 pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:11.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.962+0000 7fa8fd61a700 1 -- 172.21.15.117:0/2734232017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8f8074010 msgr2=0x7fa8f80743f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:11.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.962+0000 7fa8fd61a700 1 --2- 172.21.15.117:0/2734232017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8f8074010 0x7fa8f80743f0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fa8e8009990 tx=0x7fa8e8009ca0 comp rx=0 tx=0).stop 2024-03-04T15:52:11.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.962+0000 7fa8fd61a700 1 -- 172.21.15.117:0/2734232017 shutdown_connections 2024-03-04T15:52:11.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.962+0000 7fa8fd61a700 1 --2- 172.21.15.117:0/2734232017 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8f810e870 0x7fa8f8110c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:11.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.962+0000 7fa8fd61a700 1 --2- 172.21.15.117:0/2734232017 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8f8074010 0x7fa8f80743f0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:11.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.962+0000 7fa8fd61a700 1 --2- 172.21.15.117:0/2734232017 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f8074930 0x7fa8f810e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:11.966 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.962+0000 7fa8fd61a700 1 -- 172.21.15.117:0/2734232017 >> 172.21.15.117:0/2734232017 conn(0x7fa8f8078eb0 msgr2=0x7fa8f80792c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:11.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.962+0000 7fa8fd61a700 1 -- 172.21.15.117:0/2734232017 shutdown_connections 2024-03-04T15:52:11.967 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.962+0000 7fa8fd61a700 1 -- 172.21.15.117:0/2734232017 wait complete. 2024-03-04T15:52:11.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.962+0000 7fa8fd61a700 1 Processor -- start 2024-03-04T15:52:11.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.962+0000 7fa8fd61a700 1 -- start start 2024-03-04T15:52:11.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8fd61a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f8074010 0x7fa8f819fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:11.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8fd61a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8f8074930 0x7fa8f819ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:11.968 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8fd61a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8f810e870 0x7fa8f8199b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:11.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8fd61a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f8113820 con 0x7fa8f8074010 2024-03-04T15:52:11.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8fd61a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f81136a0 con 0x7fa8f810e870 2024-03-04T15:52:11.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8fd61a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f81139a0 con 0x7fa8f8074930 2024-03-04T15:52:11.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8f6ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f8074010 0x7fa8f819fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:11.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8f67fc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8f8074930 0x7fa8f819ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:11.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8f6ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f8074010 0x7fa8f819fa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50574/0 (socket says 172.21.15.117:50574) 2024-03-04T15:52:11.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8f6ffd700 1 -- 172.21.15.117:0/3923840764 learned_addr learned my addr 172.21.15.117:0/3923840764 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:52:11.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8f67fc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8f8074930 0x7fa8f819ffa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:57818/0 (socket says 172.21.15.117:57818) 2024-03-04T15:52:11.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8f67fc700 1 -- 172.21.15.117:0/3923840764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8f810e870 msgr2=0x7fa8f8199b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:11.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8f67fc700 1 --2- 172.21.15.117:0/3923840764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8f810e870 0x7fa8f8199b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:11.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8f67fc700 1 -- 172.21.15.117:0/3923840764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f8074010 msgr2=0x7fa8f819fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:11.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8f67fc700 1 --2- 172.21.15.117:0/3923840764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f8074010 0x7fa8f819fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:11.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8f67fc700 1 -- 172.21.15.117:0/3923840764 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f819a410 con 0x7fa8f8074930 2024-03-04T15:52:11.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8f6ffd700 1 --2- 172.21.15.117:0/3923840764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f8074010 0x7fa8f819fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:52:11.969 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8f67fc700 1 --2- 172.21.15.117:0/3923840764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8f8074930 0x7fa8f819ffa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00e9f0 tx=0x7fa8ec00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:11.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8dffff700 1 -- 172.21.15.117:0/3923840764 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec00cb70 con 0x7fa8f8074930 2024-03-04T15:52:11.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8dffff700 1 -- 172.21.15.117:0/3923840764 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fa8ec0044a0 con 0x7fa8f8074930 2024-03-04T15:52:11.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8fd61a700 1 -- 172.21.15.117:0/3923840764 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f819a700 con 0x7fa8f8074930 2024-03-04T15:52:11.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8dffff700 1 -- 172.21.15.117:0/3923840764 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8ec003c70 con 0x7fa8f8074930 2024-03-04T15:52:11.970 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8fd61a700 1 -- 172.21.15.117:0/3923840764 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f81a69e0 con 0x7fa8f8074930 2024-03-04T15:52:11.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8fd61a700 1 -- 172.21.15.117:0/3923840764 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f80756a0 con 0x7fa8f8074930 2024-03-04T15:52:11.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8dffff700 1 -- 172.21.15.117:0/3923840764 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7fa8ec003e50 con 0x7fa8f8074930 2024-03-04T15:52:11.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8dffff700 1 --2- 172.21.15.117:0/3923840764 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fa8e006d780 0x7fa8e006fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:11.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8dffff700 1 -- 172.21.15.117:0/3923840764 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7fa8ec08dc20 con 0x7fa8f8074930 2024-03-04T15:52:11.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8f6ffd700 1 --2- 172.21.15.117:0/3923840764 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fa8e006d780 0x7fa8e006fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:11.971 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.966+0000 7fa8f6ffd700 1 --2- 172.21.15.117:0/3923840764 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fa8e006d780 0x7fa8e006fc40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800b760 tx=0x7fa8e800b630 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:11.974 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:11.974+0000 7fa8dffff700 1 -- 172.21.15.117:0/3923840764 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa8ec05b470 con 0x7fa8f8074930 2024-03-04T15:52:12.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:11 smithi117 bash[28843]: audit 2024-03-04T15:52:11.197808+0000 mon.a (mon.0) 828 : audit 0 from='client.? 172.21.15.117:0/1794332256' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-04T15:52:12.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:11 smithi117 bash[21480]: audit 2024-03-04T15:52:11.197808+0000 mon.a (mon.0) 828 : audit 0 from='client.? 172.21.15.117:0/1794332256' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-04T15:52:12.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:11 smithi151 bash[23756]: audit 2024-03-04T15:52:11.197808+0000 mon.a (mon.0) 828 : audit 0 from='client.? 172.21.15.117:0/1794332256' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-03-04T15:52:12.187 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:12.186+0000 7fa8fd61a700 1 -- 172.21.15.117:0/3923840764 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fa8f804faa0 con 0x7fa8f8074930 2024-03-04T15:52:12.188 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:12.186+0000 7fa8dffff700 1 -- 172.21.15.117:0/3923840764 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa8ec05eec0 con 0x7fa8f8074930 2024-03-04T15:52:12.188 INFO:teuthology.orchestra.run.smithi117.stdout:167503724607 2024-03-04T15:52:12.190 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:12.186+0000 7fa8fd61a700 1 -- 172.21.15.117:0/3923840764 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fa8e006d780 msgr2=0x7fa8e006fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:12.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:12.186+0000 7fa8fd61a700 1 --2- 172.21.15.117:0/3923840764 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fa8e006d780 0x7fa8e006fc40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800b760 tx=0x7fa8e800b630 comp rx=0 tx=0).stop 2024-03-04T15:52:12.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:12.186+0000 7fa8fd61a700 1 -- 172.21.15.117:0/3923840764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8f8074930 msgr2=0x7fa8f819ffa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:12.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:12.186+0000 7fa8fd61a700 1 --2- 172.21.15.117:0/3923840764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8f8074930 0x7fa8f819ffa0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa8ec00e9f0 tx=0x7fa8ec00ed00 comp rx=0 tx=0).stop 2024-03-04T15:52:12.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:12.186+0000 7fa8fd61a700 1 -- 172.21.15.117:0/3923840764 shutdown_connections 2024-03-04T15:52:12.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:12.186+0000 7fa8fd61a700 1 --2- 172.21.15.117:0/3923840764 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fa8e006d780 0x7fa8e006fc40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:12.191 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:12.186+0000 7fa8fd61a700 1 --2- 172.21.15.117:0/3923840764 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa8f8074010 0x7fa8f819fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:12.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:12.186+0000 7fa8fd61a700 1 --2- 172.21.15.117:0/3923840764 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa8f810e870 0x7fa8f8199b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:12.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:12.186+0000 7fa8fd61a700 1 --2- 172.21.15.117:0/3923840764 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa8f8074930 0x7fa8f819ffa0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:12.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:12.186+0000 7fa8fd61a700 1 -- 172.21.15.117:0/3923840764 >> 172.21.15.117:0/3923840764 conn(0x7fa8f8078eb0 msgr2=0x7fa8f81062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:12.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:12.186+0000 7fa8fd61a700 1 -- 172.21.15.117:0/3923840764 shutdown_connections 2024-03-04T15:52:12.192 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:12.186+0000 7fa8fd61a700 1 -- 172.21.15.117:0/3923840764 wait complete. 2024-03-04T15:52:13.118 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:12 smithi117 bash[21480]: cluster 2024-03-04T15:52:11.366595+0000 mgr.y (mgr.24428) 85 : cluster 0 pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:13.119 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:12 smithi117 bash[21480]: audit 2024-03-04T15:52:12.189822+0000 mon.c (mon.2) 64 : audit 0 from='client.? 172.21.15.117:0/3923840764' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-04T15:52:13.119 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:12 smithi117 bash[28843]: cluster 2024-03-04T15:52:11.366595+0000 mgr.y (mgr.24428) 85 : cluster 0 pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:13.119 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:12 smithi117 bash[28843]: audit 2024-03-04T15:52:12.189822+0000 mon.c (mon.2) 64 : audit 0 from='client.? 172.21.15.117:0/3923840764' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-04T15:52:13.120 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182478 got 107374182482 for osd.3 2024-03-04T15:52:13.120 DEBUG:teuthology.parallel:result is None 2024-03-04T15:52:13.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:12 smithi151 bash[23756]: cluster 2024-03-04T15:52:11.366595+0000 mgr.y (mgr.24428) 85 : cluster 0 pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:13.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:12 smithi151 bash[23756]: audit 2024-03-04T15:52:12.189822+0000 mon.c (mon.2) 64 : audit 0 from='client.? 172.21.15.117:0/3923840764' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-03-04T15:52:13.748 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462960 got 201863462964 for osd.6 2024-03-04T15:52:13.748 DEBUG:teuthology.parallel:result is None 2024-03-04T15:52:13.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.774+0000 7f16ca128700 1 -- 172.21.15.117:0/2175255343 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16c410e990 msgr2=0x7f16c4110d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:13.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.774+0000 7f16ca128700 1 --2- 172.21.15.117:0/2175255343 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16c410e990 0x7f16c4110d80 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f16bc00b0a0 tx=0x7f16bc00b3b0 comp rx=0 tx=0).stop 2024-03-04T15:52:13.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.774+0000 7f16ca128700 1 -- 172.21.15.117:0/2175255343 shutdown_connections 2024-03-04T15:52:13.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.774+0000 7f16ca128700 1 --2- 172.21.15.117:0/2175255343 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16c41018f0 0x7f16c4101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:13.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.774+0000 7f16ca128700 1 --2- 172.21.15.117:0/2175255343 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16c410e990 0x7f16c4110d80 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:13.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.774+0000 7f16ca128700 1 --2- 172.21.15.117:0/2175255343 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16c4102210 0x7f16c410e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:13.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.774+0000 7f16ca128700 1 -- 172.21.15.117:0/2175255343 >> 172.21.15.117:0/2175255343 conn(0x7f16c4076d80 msgr2=0x7f16c4077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:13.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.774+0000 7f16ca128700 1 -- 172.21.15.117:0/2175255343 shutdown_connections 2024-03-04T15:52:13.778 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.774+0000 7f16ca128700 1 -- 172.21.15.117:0/2175255343 wait complete. 2024-03-04T15:52:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.774+0000 7f16ca128700 1 Processor -- start 2024-03-04T15:52:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.774+0000 7f16ca128700 1 -- start start 2024-03-04T15:52:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16ca128700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16c41018f0 0x7f16c4199d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16ca128700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16c4102210 0x7f16c419a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16ca128700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16c410e990 0x7f16c419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16ca128700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c4113940 con 0x7f16c4102210 2024-03-04T15:52:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16ca128700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16c41137c0 con 0x7f16c410e990 2024-03-04T15:52:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16ca128700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16c4113ac0 con 0x7f16c41018f0 2024-03-04T15:52:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16c2ffd700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16c4102210 0x7f16c419a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16c37fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16c41018f0 0x7f16c4199d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16c37fe700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16c41018f0 0x7f16c4199d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:57832/0 (socket says 172.21.15.117:57832) 2024-03-04T15:52:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16c37fe700 1 -- 172.21.15.117:0/2179148858 learned_addr learned my addr 172.21.15.117:0/2179148858 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:52:13.780 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16c3fff700 1 --2- 172.21.15.117:0/2179148858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16c410e990 0x7f16c419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:13.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16c37fe700 1 -- 172.21.15.117:0/2179148858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16c410e990 msgr2=0x7f16c419e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:13.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16c37fe700 1 --2- 172.21.15.117:0/2179148858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16c410e990 0x7f16c419e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:13.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16c37fe700 1 -- 172.21.15.117:0/2179148858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16c4102210 msgr2=0x7f16c419a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:13.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16c37fe700 1 --2- 172.21.15.117:0/2179148858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16c4102210 0x7f16c419a2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:13.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16c37fe700 1 -- 172.21.15.117:0/2179148858 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16c419ed90 con 0x7f16c41018f0 2024-03-04T15:52:13.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16c2ffd700 1 --2- 172.21.15.117:0/2179148858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16c4102210 0x7f16c419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:52:13.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16c37fe700 1 --2- 172.21.15.117:0/2179148858 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16c41018f0 0x7f16c4199d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f16b400e9a0 tx=0x7f16b400ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:13.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16c0ff9700 1 -- 172.21.15.117:0/2179148858 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b400cbd0 con 0x7f16c41018f0 2024-03-04T15:52:13.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16c0ff9700 1 -- 172.21.15.117:0/2179148858 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f16b400cd70 con 0x7f16c41018f0 2024-03-04T15:52:13.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16ca128700 1 -- 172.21.15.117:0/2179148858 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16c419f080 con 0x7f16c41018f0 2024-03-04T15:52:13.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16c0ff9700 1 -- 172.21.15.117:0/2179148858 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16b4018680 con 0x7f16c41018f0 2024-03-04T15:52:13.781 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16ca128700 1 -- 172.21.15.117:0/2179148858 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16c41a6900 con 0x7f16c41018f0 2024-03-04T15:52:13.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16ca128700 1 -- 172.21.15.117:0/2179148858 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16c404faa0 con 0x7f16c41018f0 2024-03-04T15:52:13.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16c0ff9700 1 -- 172.21.15.117:0/2179148858 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f16b4010430 con 0x7f16c41018f0 2024-03-04T15:52:13.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16c0ff9700 1 --2- 172.21.15.117:0/2179148858 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f16ac06d780 0x7f16ac06fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:13.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16c0ff9700 1 -- 172.21.15.117:0/2179148858 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7f16b408dbb0 con 0x7f16c41018f0 2024-03-04T15:52:13.782 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16c2ffd700 1 --2- 172.21.15.117:0/2179148858 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f16ac06d780 0x7f16ac06fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:13.783 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.778+0000 7f16c2ffd700 1 --2- 172.21.15.117:0/2179148858 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f16ac06d780 0x7f16ac06fc40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f16c40774c0 tx=0x7f16b800a380 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:13.787 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.786+0000 7f16c0ff9700 1 -- 172.21.15.117:0/2179148858 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f16b405b480 con 0x7f16c41018f0 2024-03-04T15:52:13.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a3189700 1 -- 172.21.15.117:0/2465113317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f619c107220 msgr2=0x7f619c10edd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:13.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a3189700 1 --2- 172.21.15.117:0/2465113317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f619c107220 0x7f619c10edd0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f619400b0a0 tx=0x7f619400b3b0 comp rx=0 tx=0).stop 2024-03-04T15:52:13.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a3189700 1 -- 172.21.15.117:0/2465113317 shutdown_connections 2024-03-04T15:52:13.953 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a3189700 1 --2- 172.21.15.117:0/2465113317 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f619c107220 0x7f619c10edd0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:13.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a3189700 1 --2- 172.21.15.117:0/2465113317 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f619c06a6a0 0x7f619c06aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:13.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a3189700 1 --2- 172.21.15.117:0/2465113317 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f619c06b050 0x7f619c106b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:13.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a3189700 1 -- 172.21.15.117:0/2465113317 >> 172.21.15.117:0/2465113317 conn(0x7f619c077de0 msgr2=0x7f619c0781f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:13.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a3189700 1 -- 172.21.15.117:0/2465113317 shutdown_connections 2024-03-04T15:52:13.954 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a3189700 1 -- 172.21.15.117:0/2465113317 wait complete. 2024-03-04T15:52:13.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a3189700 1 Processor -- start 2024-03-04T15:52:13.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a3189700 1 -- start start 2024-03-04T15:52:13.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a3189700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f619c06a6a0 0x7f619c100d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:13.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a3189700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f619c06b050 0x7f619c1012d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:13.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a3189700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f619c107220 0x7f619c101810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:13.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a3189700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c111690 con 0x7f619c06a6a0 2024-03-04T15:52:13.955 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a3189700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f619c111510 con 0x7f619c06b050 2024-03-04T15:52:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a3189700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f619c111810 con 0x7f619c107220 2024-03-04T15:52:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a0f25700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f619c06a6a0 0x7f619c100d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a1726700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f619c107220 0x7f619c101810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a0f25700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f619c06a6a0 0x7f619c100d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50624/0 (socket says 172.21.15.117:50624) 2024-03-04T15:52:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a0f25700 1 -- 172.21.15.117:0/1824963453 learned_addr learned my addr 172.21.15.117:0/1824963453 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:52:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a0f25700 1 -- 172.21.15.117:0/1824963453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f619c107220 msgr2=0x7f619c101810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a0f25700 1 --2- 172.21.15.117:0/1824963453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f619c107220 0x7f619c101810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f619bfff700 1 --2- 172.21.15.117:0/1824963453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f619c06b050 0x7f619c1012d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a0f25700 1 -- 172.21.15.117:0/1824963453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f619c06b050 msgr2=0x7f619c1012d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a0f25700 1 --2- 172.21.15.117:0/1824963453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f619c06b050 0x7f619c1012d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a0f25700 1 -- 172.21.15.117:0/1824963453 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f619c1a5fd0 con 0x7f619c06a6a0 2024-03-04T15:52:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.950+0000 7f61a1726700 1 --2- 172.21.15.117:0/1824963453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f619c107220 0x7f619c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:52:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.954+0000 7f61a0f25700 1 --2- 172.21.15.117:0/1824963453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f619c06a6a0 0x7f619c100d90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f619000c920 tx=0x7f619000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:13.956 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.954+0000 7f6199ffb700 1 -- 172.21.15.117:0/1824963453 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190007b00 con 0x7f619c06a6a0 2024-03-04T15:52:13.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.954+0000 7f61a3189700 1 -- 172.21.15.117:0/1824963453 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f619c1a62c0 con 0x7f619c06a6a0 2024-03-04T15:52:13.957 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.954+0000 7f6199ffb700 1 -- 172.21.15.117:0/1824963453 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f6190004cb0 con 0x7f619c06a6a0 2024-03-04T15:52:13.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.954+0000 7f6199ffb700 1 -- 172.21.15.117:0/1824963453 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6190005470 con 0x7f619c06a6a0 2024-03-04T15:52:13.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.954+0000 7f61a3189700 1 -- 172.21.15.117:0/1824963453 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f619c1a6850 con 0x7f619c06a6a0 2024-03-04T15:52:13.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.954+0000 7f61a3189700 1 -- 172.21.15.117:0/1824963453 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f619c04faa0 con 0x7f619c06a6a0 2024-03-04T15:52:13.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.954+0000 7f6199ffb700 1 -- 172.21.15.117:0/1824963453 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f6190004750 con 0x7f619c06a6a0 2024-03-04T15:52:13.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.954+0000 7f6199ffb700 1 --2- 172.21.15.117:0/1824963453 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f618c06d780 0x7f618c06fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:13.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.954+0000 7f6199ffb700 1 -- 172.21.15.117:0/1824963453 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7f619008d910 con 0x7f619c06a6a0 2024-03-04T15:52:13.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.954+0000 7f619bfff700 1 --2- 172.21.15.117:0/1824963453 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f618c06d780 0x7f618c06fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:13.958 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.954+0000 7f619bfff700 1 --2- 172.21.15.117:0/1824963453 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f618c06d780 0x7f618c06fc40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f619c1008f0 tx=0x7f6188009260 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:13.962 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:13.958+0000 7f6199ffb700 1 -- 172.21.15.117:0/1824963453 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6190014030 con 0x7f619c06a6a0 2024-03-04T15:52:14.004 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.002+0000 7f16ca128700 1 -- 172.21.15.117:0/2179148858 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f16c41a6b30 con 0x7f16c41018f0 2024-03-04T15:52:14.005 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.002+0000 7f16c0ff9700 1 -- 172.21.15.117:0/2179148858 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f16b405eed0 con 0x7f16c41018f0 2024-03-04T15:52:14.005 INFO:teuthology.orchestra.run.smithi117.stdout:34359738482 2024-03-04T15:52:14.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.006+0000 7f16ca128700 1 -- 172.21.15.117:0/2179148858 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f16ac06d780 msgr2=0x7f16ac06fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:14.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.006+0000 7f16ca128700 1 --2- 172.21.15.117:0/2179148858 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f16ac06d780 0x7f16ac06fc40 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f16c40774c0 tx=0x7f16b800a380 comp rx=0 tx=0).stop 2024-03-04T15:52:14.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.006+0000 7f16ca128700 1 -- 172.21.15.117:0/2179148858 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16c41018f0 msgr2=0x7f16c4199d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:14.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.006+0000 7f16ca128700 1 --2- 172.21.15.117:0/2179148858 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16c41018f0 0x7f16c4199d80 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f16b400e9a0 tx=0x7f16b400ecb0 comp rx=0 tx=0).stop 2024-03-04T15:52:14.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.006+0000 7f16ca128700 1 -- 172.21.15.117:0/2179148858 shutdown_connections 2024-03-04T15:52:14.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.006+0000 7f16ca128700 1 --2- 172.21.15.117:0/2179148858 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f16c410e990 0x7f16c419e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:14.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.006+0000 7f16ca128700 1 --2- 172.21.15.117:0/2179148858 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f16c41018f0 0x7f16c4199d80 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:14.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.006+0000 7f16ca128700 1 --2- 172.21.15.117:0/2179148858 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f16ac06d780 0x7f16ac06fc40 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:14.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.006+0000 7f16ca128700 1 --2- 172.21.15.117:0/2179148858 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f16c4102210 0x7f16c419a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:14.007 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.006+0000 7f16ca128700 1 -- 172.21.15.117:0/2179148858 >> 172.21.15.117:0/2179148858 conn(0x7f16c4076d80 msgr2=0x7f16c40ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:14.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.006+0000 7f16ca128700 1 -- 172.21.15.117:0/2179148858 shutdown_connections 2024-03-04T15:52:14.008 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.006+0000 7f16ca128700 1 -- 172.21.15.117:0/2179148858 wait complete. 2024-03-04T15:52:14.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.166+0000 7f61a3189700 1 -- 172.21.15.117:0/1824963453 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f619c1022c0 con 0x7f619c06a6a0 2024-03-04T15:52:14.171 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.170+0000 7f6199ffb700 1 -- 172.21.15.117:0/1824963453 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f619005b160 con 0x7f619c06a6a0 2024-03-04T15:52:14.171 INFO:teuthology.orchestra.run.smithi117.stdout:77309411423 2024-03-04T15:52:14.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.170+0000 7f61a3189700 1 -- 172.21.15.117:0/1824963453 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f618c06d780 msgr2=0x7f618c06fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:14.173 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.170+0000 7f61a3189700 1 --2- 172.21.15.117:0/1824963453 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f618c06d780 0x7f618c06fc40 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f619c1008f0 tx=0x7f6188009260 comp rx=0 tx=0).stop 2024-03-04T15:52:14.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.170+0000 7f61a3189700 1 -- 172.21.15.117:0/1824963453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f619c06a6a0 msgr2=0x7f619c100d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:14.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.170+0000 7f61a3189700 1 --2- 172.21.15.117:0/1824963453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f619c06a6a0 0x7f619c100d90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f619000c920 tx=0x7f619000cc30 comp rx=0 tx=0).stop 2024-03-04T15:52:14.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.170+0000 7f61a3189700 1 -- 172.21.15.117:0/1824963453 shutdown_connections 2024-03-04T15:52:14.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.170+0000 7f61a3189700 1 --2- 172.21.15.117:0/1824963453 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f618c06d780 0x7f618c06fc40 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:14.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.170+0000 7f61a3189700 1 --2- 172.21.15.117:0/1824963453 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f619c06a6a0 0x7f619c100d90 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:14.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.170+0000 7f61a3189700 1 --2- 172.21.15.117:0/1824963453 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f619c107220 0x7f619c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:14.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.170+0000 7f61a3189700 1 --2- 172.21.15.117:0/1824963453 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f619c06b050 0x7f619c1012d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:14.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.170+0000 7f61a3189700 1 -- 172.21.15.117:0/1824963453 >> 172.21.15.117:0/1824963453 conn(0x7f619c077de0 msgr2=0x7f619c105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:14.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.170+0000 7f61a3189700 1 -- 172.21.15.117:0/1824963453 shutdown_connections 2024-03-04T15:52:14.174 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:14.170+0000 7f61a3189700 1 -- 172.21.15.117:0/1824963453 wait complete. 2024-03-04T15:52:15.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:14 smithi117 bash[21480]: cluster 2024-03-04T15:52:13.367295+0000 mgr.y (mgr.24428) 86 : cluster 0 pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:15.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:14 smithi117 bash[21480]: audit 2024-03-04T15:52:14.006901+0000 mon.c (mon.2) 65 : audit 0 from='client.? 172.21.15.117:0/2179148858' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-04T15:52:15.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:14 smithi117 bash[21480]: audit 2024-03-04T15:52:14.173305+0000 mon.a (mon.0) 829 : audit 0 from='client.? 172.21.15.117:0/1824963453' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-04T15:52:15.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:14 smithi117 bash[28843]: cluster 2024-03-04T15:52:13.367295+0000 mgr.y (mgr.24428) 86 : cluster 0 pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:15.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:14 smithi117 bash[28843]: audit 2024-03-04T15:52:14.006901+0000 mon.c (mon.2) 65 : audit 0 from='client.? 172.21.15.117:0/2179148858' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-04T15:52:15.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:14 smithi117 bash[28843]: audit 2024-03-04T15:52:14.173305+0000 mon.a (mon.0) 829 : audit 0 from='client.? 172.21.15.117:0/1824963453' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-04T15:52:15.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:14 smithi151 bash[23756]: cluster 2024-03-04T15:52:13.367295+0000 mgr.y (mgr.24428) 86 : cluster 0 pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:15.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:14 smithi151 bash[23756]: audit 2024-03-04T15:52:14.006901+0000 mon.c (mon.2) 65 : audit 0 from='client.? 172.21.15.117:0/2179148858' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-03-04T15:52:15.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:14 smithi151 bash[23756]: audit 2024-03-04T15:52:14.173305+0000 mon.a (mon.0) 829 : audit 0 from='client.? 172.21.15.117:0/1824963453' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-03-04T15:52:15.245 INFO:tasks.cephadm.ceph_manager.ceph:need seq 167503724602 got 167503724607 for osd.5 2024-03-04T15:52:15.245 DEBUG:teuthology.parallel:result is None 2024-03-04T15:52:15.384 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6fb3d700 1 -- 172.21.15.117:0/512830992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d68104c30 msgr2=0x7f4d68108c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:15.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6fb3d700 1 --2- 172.21.15.117:0/512830992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d68104c30 0x7f4d68108c80 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4d58009920 tx=0x7f4d58009c30 comp rx=0 tx=0).stop 2024-03-04T15:52:15.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6fb3d700 1 -- 172.21.15.117:0/512830992 shutdown_connections 2024-03-04T15:52:15.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6fb3d700 1 --2- 172.21.15.117:0/512830992 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d68109360 0x7f4d68110f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:15.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6fb3d700 1 --2- 172.21.15.117:0/512830992 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d68104280 0x7f4d68104660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:15.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6fb3d700 1 --2- 172.21.15.117:0/512830992 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d68104c30 0x7f4d68108c80 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:15.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6fb3d700 1 -- 172.21.15.117:0/512830992 >> 172.21.15.117:0/512830992 conn(0x7f4d680ffb10 msgr2=0x7f4d68101f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:15.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6fb3d700 1 -- 172.21.15.117:0/512830992 shutdown_connections 2024-03-04T15:52:15.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6fb3d700 1 -- 172.21.15.117:0/512830992 wait complete. 2024-03-04T15:52:15.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6fb3d700 1 Processor -- start 2024-03-04T15:52:15.385 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6fb3d700 1 -- start start 2024-03-04T15:52:15.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6fb3d700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d68104280 0x7f4d68199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:15.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6fb3d700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d68104c30 0x7f4d6819a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:15.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6fb3d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d68109360 0x7f4d6819e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:15.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6fb3d700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d68113860 con 0x7f4d68109360 2024-03-04T15:52:15.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6fb3d700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d681136e0 con 0x7f4d68104280 2024-03-04T15:52:15.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6fb3d700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d681139e0 con 0x7f4d68104c30 2024-03-04T15:52:15.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6d0d8700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d68104c30 0x7f4d6819a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:15.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6d0d8700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d68104c30 0x7f4d6819a400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:57860/0 (socket says 172.21.15.117:57860) 2024-03-04T15:52:15.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6d0d8700 1 -- 172.21.15.117:0/924594504 learned_addr learned my addr 172.21.15.117:0/924594504 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:52:15.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6e0da700 1 --2- 172.21.15.117:0/924594504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d68109360 0x7f4d6819e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:15.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6d0d8700 1 -- 172.21.15.117:0/924594504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d68104280 msgr2=0x7f4d68199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:15.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6d8d9700 1 --2- 172.21.15.117:0/924594504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d68104280 0x7f4d68199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:15.387 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6d0d8700 1 --2- 172.21.15.117:0/924594504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d68104280 0x7f4d68199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:15.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6d0d8700 1 -- 172.21.15.117:0/924594504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d68109360 msgr2=0x7f4d6819e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:15.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6d0d8700 1 --2- 172.21.15.117:0/924594504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d68109360 0x7f4d6819e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:15.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6d0d8700 1 -- 172.21.15.117:0/924594504 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d6819eed0 con 0x7f4d68104c30 2024-03-04T15:52:15.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6d0d8700 1 --2- 172.21.15.117:0/924594504 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d68104c30 0x7f4d6819a400 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4d58000c00 tx=0x7f4d5800fa70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:15.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d5effd700 1 -- 172.21.15.117:0/924594504 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d5800b790 con 0x7f4d68104c30 2024-03-04T15:52:15.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d5effd700 1 -- 172.21.15.117:0/924594504 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f4d580054c0 con 0x7f4d68104c30 2024-03-04T15:52:15.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6fb3d700 1 -- 172.21.15.117:0/924594504 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d6819f160 con 0x7f4d68104c30 2024-03-04T15:52:15.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d5effd700 1 -- 172.21.15.117:0/924594504 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d58003990 con 0x7f4d68104c30 2024-03-04T15:52:15.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.382+0000 7f4d6fb3d700 1 -- 172.21.15.117:0/924594504 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d681a6a40 con 0x7f4d68104c30 2024-03-04T15:52:15.388 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.386+0000 7f4d6fb3d700 1 -- 172.21.15.117:0/924594504 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d6804faa0 con 0x7f4d68104c30 2024-03-04T15:52:15.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.386+0000 7f4d5effd700 1 -- 172.21.15.117:0/924594504 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f4d58005040 con 0x7f4d68104c30 2024-03-04T15:52:15.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.386+0000 7f4d5effd700 1 --2- 172.21.15.117:0/924594504 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f4d5406d780 0x7f4d5406fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:15.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.386+0000 7f4d5effd700 1 -- 172.21.15.117:0/924594504 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7f4d5808ea20 con 0x7f4d68104c30 2024-03-04T15:52:15.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.386+0000 7f4d6d8d9700 1 --2- 172.21.15.117:0/924594504 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f4d5406d780 0x7f4d5406fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:15.389 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.386+0000 7f4d6d8d9700 1 --2- 172.21.15.117:0/924594504 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f4d5406d780 0x7f4d5406fc40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4d64006fd0 tx=0x7f4d64008040 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:15.393 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.390+0000 7f4d5effd700 1 -- 172.21.15.117:0/924594504 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f4d58013030 con 0x7f4d68104c30 2024-03-04T15:52:15.569 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:52:15 smithi117 systemd[1]: Stopping Ceph alertmanager.a for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T15:52:15.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.598+0000 7f4d6fb3d700 1 -- 172.21.15.117:0/924594504 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f4d681a6c10 con 0x7f4d68104c30 2024-03-04T15:52:15.599 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.598+0000 7f4d5effd700 1 -- 172.21.15.117:0/924594504 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4d58021e40 con 0x7f4d68104c30 2024-03-04T15:52:15.600 INFO:teuthology.orchestra.run.smithi117.stdout:236223201321 2024-03-04T15:52:15.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.598+0000 7f4d6fb3d700 1 -- 172.21.15.117:0/924594504 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f4d5406d780 msgr2=0x7f4d5406fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:15.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.598+0000 7f4d6fb3d700 1 --2- 172.21.15.117:0/924594504 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f4d5406d780 0x7f4d5406fc40 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4d64006fd0 tx=0x7f4d64008040 comp rx=0 tx=0).stop 2024-03-04T15:52:15.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.598+0000 7f4d6fb3d700 1 -- 172.21.15.117:0/924594504 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d68104c30 msgr2=0x7f4d6819a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:15.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.598+0000 7f4d6fb3d700 1 --2- 172.21.15.117:0/924594504 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d68104c30 0x7f4d6819a400 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f4d58000c00 tx=0x7f4d5800fa70 comp rx=0 tx=0).stop 2024-03-04T15:52:15.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.598+0000 7f4d6fb3d700 1 -- 172.21.15.117:0/924594504 shutdown_connections 2024-03-04T15:52:15.602 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.598+0000 7f4d6fb3d700 1 --2- 172.21.15.117:0/924594504 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f4d5406d780 0x7f4d5406fc40 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:15.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.598+0000 7f4d6fb3d700 1 --2- 172.21.15.117:0/924594504 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f4d68109360 0x7f4d6819e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:15.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.598+0000 7f4d6fb3d700 1 --2- 172.21.15.117:0/924594504 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f4d68104280 0x7f4d68199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:15.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.598+0000 7f4d6fb3d700 1 --2- 172.21.15.117:0/924594504 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f4d68104c30 0x7f4d6819a400 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:15.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.598+0000 7f4d6fb3d700 1 -- 172.21.15.117:0/924594504 >> 172.21.15.117:0/924594504 conn(0x7f4d680ffb10 msgr2=0x7f4d681001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:15.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.598+0000 7f4d6fb3d700 1 -- 172.21.15.117:0/924594504 shutdown_connections 2024-03-04T15:52:15.603 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:15.598+0000 7f4d6fb3d700 1 -- 172.21.15.117:0/924594504 wait complete. 2024-03-04T15:52:16.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:15 smithi117 bash[21480]: audit 2024-03-04T15:52:15.601978+0000 mon.c (mon.2) 66 : audit 0 from='client.? 172.21.15.117:0/924594504' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-04T15:52:16.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:15 smithi117 bash[28843]: audit 2024-03-04T15:52:15.601978+0000 mon.c (mon.2) 66 : audit 0 from='client.? 172.21.15.117:0/924594504' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-04T15:52:16.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:15 smithi151 bash[23756]: audit 2024-03-04T15:52:15.601978+0000 mon.c (mon.2) 66 : audit 0 from='client.? 172.21.15.117:0/924594504' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-03-04T15:52:16.569 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:16 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:52:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:52:16.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:52:16 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:52:16.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.834+0000 7faa0a8eb700 1 -- 172.21.15.117:0/2509717291 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa04100f30 msgr2=0x7faa04110d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:16.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.834+0000 7faa0a8eb700 1 --2- 172.21.15.117:0/2509717291 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa04100f30 0x7faa04110d20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7faa000098f0 tx=0x7faa00009c00 comp rx=0 tx=0).stop 2024-03-04T15:52:16.839 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa0a8eb700 1 -- 172.21.15.117:0/2509717291 shutdown_connections 2024-03-04T15:52:16.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa0a8eb700 1 --2- 172.21.15.117:0/2509717291 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa04102ab0 0x7faa04102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:16.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa0a8eb700 1 --2- 172.21.15.117:0/2509717291 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa04100f30 0x7faa04110d20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:16.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa0a8eb700 1 --2- 172.21.15.117:0/2509717291 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa04100570 0x7faa041009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:16.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa0a8eb700 1 -- 172.21.15.117:0/2509717291 >> 172.21.15.117:0/2509717291 conn(0x7faa04075e70 msgr2=0x7faa04076280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:16.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa0a8eb700 1 -- 172.21.15.117:0/2509717291 shutdown_connections 2024-03-04T15:52:16.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa0a8eb700 1 -- 172.21.15.117:0/2509717291 wait complete. 2024-03-04T15:52:16.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa0a8eb700 1 Processor -- start 2024-03-04T15:52:16.840 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa0a8eb700 1 -- start start 2024-03-04T15:52:16.841 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa0a8eb700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa04100570 0x7faa0419f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa0a8eb700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa04100f30 0x7faa0419fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa0a8eb700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa04102ab0 0x7faa04199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa0a8eb700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa041137b0 con 0x7faa04102ab0 2024-03-04T15:52:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa0a8eb700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa04113630 con 0x7faa04100570 2024-03-04T15:52:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa0a8eb700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa04113930 con 0x7faa04100f30 2024-03-04T15:52:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa0a0ea700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa04102ab0 0x7faa04199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa090e8700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa04100f30 0x7faa0419fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa090e8700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa04100f30 0x7faa0419fe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:57886/0 (socket says 172.21.15.117:57886) 2024-03-04T15:52:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa090e8700 1 -- 172.21.15.117:0/4284796194 learned_addr learned my addr 172.21.15.117:0/4284796194 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:52:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa0a0ea700 1 --2- 172.21.15.117:0/4284796194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa04102ab0 0x7faa04199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:50652/0 (socket says 172.21.15.117:50652) 2024-03-04T15:52:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa098e9700 1 --2- 172.21.15.117:0/4284796194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa04100570 0x7faa0419f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa090e8700 1 -- 172.21.15.117:0/4284796194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa04100570 msgr2=0x7faa0419f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa090e8700 1 --2- 172.21.15.117:0/4284796194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa04100570 0x7faa0419f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:16.842 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa090e8700 1 -- 172.21.15.117:0/4284796194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa04102ab0 msgr2=0x7faa04199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:16.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa090e8700 1 --2- 172.21.15.117:0/4284796194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa04102ab0 0x7faa04199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:16.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa090e8700 1 -- 172.21.15.117:0/4284796194 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa0419a260 con 0x7faa04100f30 2024-03-04T15:52:16.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa090e8700 1 --2- 172.21.15.117:0/4284796194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa04100f30 0x7faa0419fe80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400b530 tx=0x7fa9f400b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:16.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7fa9faffd700 1 -- 172.21.15.117:0/4284796194 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4015020 con 0x7faa04100f30 2024-03-04T15:52:16.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7fa9faffd700 1 -- 172.21.15.117:0/4284796194 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fa9f4011cb0 con 0x7faa04100f30 2024-03-04T15:52:16.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa0a8eb700 1 -- 172.21.15.117:0/4284796194 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa0419a550 con 0x7faa04100f30 2024-03-04T15:52:16.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7fa9faffd700 1 -- 172.21.15.117:0/4284796194 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4010d50 con 0x7faa04100f30 2024-03-04T15:52:16.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa0a8eb700 1 -- 172.21.15.117:0/4284796194 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa041a6810 con 0x7faa04100f30 2024-03-04T15:52:16.843 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7faa0a8eb700 1 -- 172.21.15.117:0/4284796194 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa040502f0 con 0x7faa04100f30 2024-03-04T15:52:16.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.838+0000 7fa9faffd700 1 -- 172.21.15.117:0/4284796194 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7fa9f4018020 con 0x7faa04100f30 2024-03-04T15:52:16.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.842+0000 7fa9faffd700 1 --2- 172.21.15.117:0/4284796194 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fa9f006d780 0x7fa9f006fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:16.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.842+0000 7fa9faffd700 1 -- 172.21.15.117:0/4284796194 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7fa9f408d270 con 0x7faa04100f30 2024-03-04T15:52:16.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.842+0000 7faa098e9700 1 --2- 172.21.15.117:0/4284796194 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fa9f006d780 0x7fa9f006fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:16.844 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.842+0000 7faa098e9700 1 --2- 172.21.15.117:0/4284796194 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fa9f006d780 0x7fa9f006fc40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009e60 tx=0x7fa9fc00b040 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:16.847 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:16.846+0000 7fa9faffd700 1 -- 172.21.15.117:0/4284796194 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa9f401f020 con 0x7faa04100f30 2024-03-04T15:52:16.859 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:52:16 smithi117 bash[57922]: ts=2024-03-04T15:52:16.579Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-03-04T15:52:17.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.054+0000 7faa0a8eb700 1 -- 172.21.15.117:0/4284796194 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7faa041a6b00 con 0x7faa04100f30 2024-03-04T15:52:17.057 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.054+0000 7fa9faffd700 1 -- 172.21.15.117:0/4284796194 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa9f405aac0 con 0x7faa04100f30 2024-03-04T15:52:17.058 INFO:teuthology.orchestra.run.smithi117.stdout:55834574952 2024-03-04T15:52:17.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.058+0000 7faa0a8eb700 1 -- 172.21.15.117:0/4284796194 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fa9f006d780 msgr2=0x7fa9f006fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:17.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.058+0000 7faa0a8eb700 1 --2- 172.21.15.117:0/4284796194 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fa9f006d780 0x7fa9f006fc40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fa9fc009e60 tx=0x7fa9fc00b040 comp rx=0 tx=0).stop 2024-03-04T15:52:17.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.058+0000 7faa0a8eb700 1 -- 172.21.15.117:0/4284796194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa04100f30 msgr2=0x7faa0419fe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:17.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.058+0000 7faa0a8eb700 1 --2- 172.21.15.117:0/4284796194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa04100f30 0x7faa0419fe80 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa9f400b530 tx=0x7fa9f400b840 comp rx=0 tx=0).stop 2024-03-04T15:52:17.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.058+0000 7faa0a8eb700 1 -- 172.21.15.117:0/4284796194 shutdown_connections 2024-03-04T15:52:17.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.058+0000 7faa0a8eb700 1 --2- 172.21.15.117:0/4284796194 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fa9f006d780 0x7fa9f006fc40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:17.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.058+0000 7faa0a8eb700 1 --2- 172.21.15.117:0/4284796194 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7faa04102ab0 0x7faa04199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:17.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.058+0000 7faa0a8eb700 1 --2- 172.21.15.117:0/4284796194 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7faa04100570 0x7faa0419f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:17.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.058+0000 7faa0a8eb700 1 --2- 172.21.15.117:0/4284796194 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7faa04100f30 0x7faa0419fe80 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:17.060 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.058+0000 7faa0a8eb700 1 -- 172.21.15.117:0/4284796194 >> 172.21.15.117:0/4284796194 conn(0x7faa04075e70 msgr2=0x7faa040ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:17.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.058+0000 7faa0a8eb700 1 -- 172.21.15.117:0/4284796194 shutdown_connections 2024-03-04T15:52:17.061 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.058+0000 7faa0a8eb700 1 -- 172.21.15.117:0/4284796194 wait complete. 2024-03-04T15:52:17.134 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:16 smithi117 bash[28843]: cluster 2024-03-04T15:52:15.367950+0000 mgr.y (mgr.24428) 87 : cluster 0 pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:17.134 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:16 smithi117 bash[21480]: cluster 2024-03-04T15:52:15.367950+0000 mgr.y (mgr.24428) 87 : cluster 0 pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:17.165 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738478 got 34359738482 for osd.0 2024-03-04T15:52:17.165 DEBUG:teuthology.parallel:result is None 2024-03-04T15:52:17.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:16 smithi151 bash[23756]: cluster 2024-03-04T15:52:15.367950+0000 mgr.y (mgr.24428) 87 : cluster 0 pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:17.283 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411418 got 77309411423 for osd.2 2024-03-04T15:52:17.283 DEBUG:teuthology.parallel:result is None 2024-03-04T15:52:17.553 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547e67a700 1 -- 172.21.15.117:0/2952058969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5478104c20 msgr2=0x7f5478108c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:17.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547e67a700 1 --2- 172.21.15.117:0/2952058969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5478104c20 0x7f5478108c70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f546c0099c0 tx=0x7f546c009cd0 comp rx=0 tx=0).stop 2024-03-04T15:52:17.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547e67a700 1 -- 172.21.15.117:0/2952058969 shutdown_connections 2024-03-04T15:52:17.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547e67a700 1 --2- 172.21.15.117:0/2952058969 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5478104270 0x7f5478104650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:17.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547e67a700 1 --2- 172.21.15.117:0/2952058969 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5478109350 0x7f5478110f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:17.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547e67a700 1 --2- 172.21.15.117:0/2952058969 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5478104c20 0x7f5478108c70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:17.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547e67a700 1 -- 172.21.15.117:0/2952058969 >> 172.21.15.117:0/2952058969 conn(0x7f54780ffb00 msgr2=0x7f5478101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:17.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547e67a700 1 -- 172.21.15.117:0/2952058969 shutdown_connections 2024-03-04T15:52:17.660 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547e67a700 1 -- 172.21.15.117:0/2952058969 wait complete. 2024-03-04T15:52:17.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547e67a700 1 Processor -- start 2024-03-04T15:52:17.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547e67a700 1 -- start start 2024-03-04T15:52:17.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547e67a700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5478104270 0x7f5478199d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:17.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547e67a700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5478104c20 0x7f547819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:17.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547e67a700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5478109350 0x7f547819e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:17.664 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547e67a700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5478113860 con 0x7f5478104c20 2024-03-04T15:52:17.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547e67a700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54781136e0 con 0x7f5478104270 2024-03-04T15:52:17.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547e67a700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54781139e0 con 0x7f5478109350 2024-03-04T15:52:17.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547cc17700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5478109350 0x7f547819e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:17.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547cc17700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5478109350 0x7f547819e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:38404/0 (socket says 172.21.15.117:38404) 2024-03-04T15:52:17.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547cc17700 1 -- 172.21.15.117:0/3933133687 learned_addr learned my addr 172.21.15.117:0/3933133687 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:52:17.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f54777fe700 1 --2- 172.21.15.117:0/3933133687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5478104c20 0x7f547819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:17.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f5477fff700 1 --2- 172.21.15.117:0/3933133687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5478104270 0x7f5478199d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:17.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547cc17700 1 -- 172.21.15.117:0/3933133687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5478104270 msgr2=0x7f5478199d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:17.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547cc17700 1 --2- 172.21.15.117:0/3933133687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5478104270 0x7f5478199d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:17.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547cc17700 1 -- 172.21.15.117:0/3933133687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5478104c20 msgr2=0x7f547819a2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:17.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547cc17700 1 --2- 172.21.15.117:0/3933133687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5478104c20 0x7f547819a2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:17.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547cc17700 1 -- 172.21.15.117:0/3933133687 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f547819ed70 con 0x7f5478109350 2024-03-04T15:52:17.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f54777fe700 1 --2- 172.21.15.117:0/3933133687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5478104c20 0x7f547819a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:52:17.665 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f547cc17700 1 --2- 172.21.15.117:0/3933133687 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5478109350 0x7f547819e630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f547000b970 tx=0x7f547000bc80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:17.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.550+0000 7f54757fa700 1 -- 172.21.15.117:0/3933133687 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5470019070 con 0x7f5478109350 2024-03-04T15:52:17.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.554+0000 7f54757fa700 1 -- 172.21.15.117:0/3933133687 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f547000cdb0 con 0x7f5478109350 2024-03-04T15:52:17.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.554+0000 7f547e67a700 1 -- 172.21.15.117:0/3933133687 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f547819f060 con 0x7f5478109350 2024-03-04T15:52:17.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.554+0000 7f54757fa700 1 -- 172.21.15.117:0/3933133687 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5470014dc0 con 0x7f5478109350 2024-03-04T15:52:17.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.554+0000 7f547e67a700 1 -- 172.21.15.117:0/3933133687 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54781a68e0 con 0x7f5478109350 2024-03-04T15:52:17.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.554+0000 7f547e67a700 1 -- 172.21.15.117:0/3933133687 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f547804faa0 con 0x7f5478109350 2024-03-04T15:52:17.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.554+0000 7f54757fa700 1 -- 172.21.15.117:0/3933133687 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f547001c070 con 0x7f5478109350 2024-03-04T15:52:17.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.554+0000 7f54757fa700 1 --2- 172.21.15.117:0/3933133687 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f546006d780 0x7f546006fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:17.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.554+0000 7f5477fff700 1 --2- 172.21.15.117:0/3933133687 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f546006d780 0x7f546006fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:17.666 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.554+0000 7f54757fa700 1 -- 172.21.15.117:0/3933133687 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7f547008cd30 con 0x7f5478109350 2024-03-04T15:52:17.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.554+0000 7f5477fff700 1 --2- 172.21.15.117:0/3933133687 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f546006d780 0x7f546006fc40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5468009680 tx=0x7f5468009260 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:17.667 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.558+0000 7f54757fa700 1 -- 172.21.15.117:0/3933133687 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f547005e050 con 0x7f5478109350 2024-03-04T15:52:17.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.758+0000 7f547e67a700 1 -- 172.21.15.117:0/3933133687 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f54781a6b10 con 0x7f5478109350 2024-03-04T15:52:17.761 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.758+0000 7f54757fa700 1 -- 172.21.15.117:0/3933133687 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f5470062610 con 0x7f5478109350 2024-03-04T15:52:17.761 INFO:teuthology.orchestra.run.smithi117.stdout:133143986250 2024-03-04T15:52:17.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.762+0000 7f547e67a700 1 -- 172.21.15.117:0/3933133687 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f546006d780 msgr2=0x7f546006fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:17.763 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.762+0000 7f547e67a700 1 --2- 172.21.15.117:0/3933133687 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f546006d780 0x7f546006fc40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f5468009680 tx=0x7f5468009260 comp rx=0 tx=0).stop 2024-03-04T15:52:17.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.762+0000 7f547e67a700 1 -- 172.21.15.117:0/3933133687 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5478109350 msgr2=0x7f547819e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:17.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.762+0000 7f547e67a700 1 --2- 172.21.15.117:0/3933133687 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5478109350 0x7f547819e630 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f547000b970 tx=0x7f547000bc80 comp rx=0 tx=0).stop 2024-03-04T15:52:17.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.762+0000 7f547e67a700 1 -- 172.21.15.117:0/3933133687 shutdown_connections 2024-03-04T15:52:17.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.762+0000 7f547e67a700 1 --2- 172.21.15.117:0/3933133687 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5478109350 0x7f547819e630 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:17.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.762+0000 7f547e67a700 1 --2- 172.21.15.117:0/3933133687 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5478104270 0x7f5478199d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:17.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.762+0000 7f547e67a700 1 --2- 172.21.15.117:0/3933133687 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f546006d780 0x7f546006fc40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:17.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.762+0000 7f547e67a700 1 --2- 172.21.15.117:0/3933133687 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5478104c20 0x7f547819a2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:17.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.762+0000 7f547e67a700 1 -- 172.21.15.117:0/3933133687 >> 172.21.15.117:0/3933133687 conn(0x7f54780ffb00 msgr2=0x7f5478100070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:17.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.762+0000 7f547e67a700 1 -- 172.21.15.117:0/3933133687 shutdown_connections 2024-03-04T15:52:17.764 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:17.762+0000 7f547e67a700 1 -- 172.21.15.117:0/3933133687 wait complete. 2024-03-04T15:52:18.051 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:17 smithi117 bash[21480]: audit 2024-03-04T15:52:16.361458+0000 mgr.y (mgr.24428) 88 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:52:18.051 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:17 smithi117 bash[21480]: audit 2024-03-04T15:52:17.059690+0000 mon.c (mon.2) 67 : audit 0 from='client.? 172.21.15.117:0/4284796194' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-04T15:52:18.051 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:17 smithi117 bash[21480]: audit 2024-03-04T15:52:17.763432+0000 mon.c (mon.2) 68 : audit 0 from='client.? 172.21.15.117:0/3933133687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-04T15:52:18.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:17 smithi151 bash[23756]: audit 2024-03-04T15:52:16.361458+0000 mgr.y (mgr.24428) 88 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:52:18.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:17 smithi151 bash[23756]: audit 2024-03-04T15:52:17.059690+0000 mon.c (mon.2) 67 : audit 0 from='client.? 172.21.15.117:0/4284796194' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-04T15:52:18.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:17 smithi151 bash[23756]: audit 2024-03-04T15:52:17.763432+0000 mon.c (mon.2) 68 : audit 0 from='client.? 172.21.15.117:0/3933133687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-04T15:52:18.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:18 smithi117 bash[28843]: audit 2024-03-04T15:52:16.361458+0000 mgr.y (mgr.24428) 88 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:52:18.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:18 smithi117 bash[28843]: audit 2024-03-04T15:52:17.059690+0000 mon.c (mon.2) 67 : audit 0 from='client.? 172.21.15.117:0/4284796194' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-03-04T15:52:18.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:18 smithi117 bash[28843]: audit 2024-03-04T15:52:17.763432+0000 mon.c (mon.2) 68 : audit 0 from='client.? 172.21.15.117:0/3933133687' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-03-04T15:52:18.711 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201316 got 236223201321 for osd.7 2024-03-04T15:52:18.711 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:52:18 smithi117 bash[62522]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-alertmanager-a 2024-03-04T15:52:18.712 DEBUG:teuthology.parallel:result is None 2024-03-04T15:52:19.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:18 smithi117 bash[21480]: cluster 2024-03-04T15:52:17.369372+0000 mgr.y (mgr.24428) 89 : cluster 0 pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:19.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:18 smithi117 bash[28843]: cluster 2024-03-04T15:52:17.369372+0000 mgr.y (mgr.24428) 89 : cluster 0 pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:19.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:18 smithi151 bash[23756]: cluster 2024-03-04T15:52:17.369372+0000 mgr.y (mgr.24428) 89 : cluster 0 pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:19.410 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:52:19 smithi117 bash[62747]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-alertmanager-a 2024-03-04T15:52:19.410 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:52:19 smithi117 systemd[1]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-03-04T15:52:19.410 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:52:19 smithi117 systemd[1]: Stopped Ceph alertmanager.a for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T15:52:19.410 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:52:19 smithi117 systemd[1]: Started Ceph alertmanager.a for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T15:52:19.411 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574947 got 55834574952 for osd.1 2024-03-04T15:52:19.411 DEBUG:teuthology.parallel:result is None 2024-03-04T15:52:20.005 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986246 got 133143986250 for osd.4 2024-03-04T15:52:20.005 DEBUG:teuthology.parallel:result is None 2024-03-04T15:52:20.005 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-03-04T15:52:20.006 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph pg dump --format=json 2024-03-04T15:52:20.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:20 smithi151 bash[23756]: cluster 2024-03-04T15:52:19.369787+0000 mgr.y (mgr.24428) 90 : cluster 0 pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:20.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:20 smithi151 bash[23756]: audit 2024-03-04T15:52:19.414134+0000 mon.a (mon.0) 830 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:20.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:20 smithi151 bash[23756]: audit 2024-03-04T15:52:19.421424+0000 mon.a (mon.0) 831 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:20.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:20 smithi151 bash[23756]: cephadm 2024-03-04T15:52:19.434509+0000 mgr.y (mgr.24428) 91 : cephadm 1 Reconfiguring prometheus.a (dependencies changed)... 2024-03-04T15:52:20.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:20 smithi151 bash[23756]: cephadm 2024-03-04T15:52:19.670168+0000 mgr.y (mgr.24428) 92 : cephadm 1 Reconfiguring daemon prometheus.a on smithi151 2024-03-04T15:52:20.819 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:52:20 smithi117 bash[62773]: ts=2024-03-04T15:52:20.538Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-03-04T15:52:20.820 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:52:20 smithi117 bash[62773]: ts=2024-03-04T15:52:20.539Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-03-04T15:52:20.820 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:52:20 smithi117 bash[62773]: ts=2024-03-04T15:52:20.540Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.117 port=9094 2024-03-04T15:52:20.820 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:52:20 smithi117 bash[62773]: ts=2024-03-04T15:52:20.541Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-03-04T15:52:20.820 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:52:20 smithi117 bash[62773]: ts=2024-03-04T15:52:20.571Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-04T15:52:20.820 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:52:20 smithi117 bash[62773]: ts=2024-03-04T15:52:20.571Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-03-04T15:52:20.820 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:52:20 smithi117 bash[62773]: ts=2024-03-04T15:52:20.575Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-03-04T15:52:20.820 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:52:20 smithi117 bash[62773]: ts=2024-03-04T15:52:20.575Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-03-04T15:52:20.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:20 smithi117 bash[21480]: cluster 2024-03-04T15:52:19.369787+0000 mgr.y (mgr.24428) 90 : cluster 0 pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:20.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:20 smithi117 bash[21480]: audit 2024-03-04T15:52:19.414134+0000 mon.a (mon.0) 830 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:20.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:20 smithi117 bash[21480]: audit 2024-03-04T15:52:19.421424+0000 mon.a (mon.0) 831 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:20.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:20 smithi117 bash[21480]: cephadm 2024-03-04T15:52:19.434509+0000 mgr.y (mgr.24428) 91 : cephadm 1 Reconfiguring prometheus.a (dependencies changed)... 2024-03-04T15:52:20.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:20 smithi117 bash[21480]: cephadm 2024-03-04T15:52:19.670168+0000 mgr.y (mgr.24428) 92 : cephadm 1 Reconfiguring daemon prometheus.a on smithi151 2024-03-04T15:52:20.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:20 smithi117 bash[28843]: cluster 2024-03-04T15:52:19.369787+0000 mgr.y (mgr.24428) 90 : cluster 0 pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:20.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:20 smithi117 bash[28843]: audit 2024-03-04T15:52:19.414134+0000 mon.a (mon.0) 830 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:20.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:20 smithi117 bash[28843]: audit 2024-03-04T15:52:19.421424+0000 mon.a (mon.0) 831 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:20.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:20 smithi117 bash[28843]: cephadm 2024-03-04T15:52:19.434509+0000 mgr.y (mgr.24428) 91 : cephadm 1 Reconfiguring prometheus.a (dependencies changed)... 2024-03-04T15:52:20.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:20 smithi117 bash[28843]: cephadm 2024-03-04T15:52:19.670168+0000 mgr.y (mgr.24428) 92 : cephadm 1 Reconfiguring daemon prometheus.a on smithi151 2024-03-04T15:52:21.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:21 smithi151 bash[23756]: audit 2024-03-04T15:52:20.511509+0000 mon.c (mon.2) 69 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:52:21.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:21 smithi117 bash[21480]: audit 2024-03-04T15:52:20.511509+0000 mon.c (mon.2) 69 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:52:21.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:21 smithi117 bash[28843]: audit 2024-03-04T15:52:20.511509+0000 mon.c (mon.2) 69 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:52:22.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:22 smithi151 bash[23756]: cluster 2024-03-04T15:52:21.371144+0000 mgr.y (mgr.24428) 93 : cluster 0 pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:22.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:22 smithi117 bash[21480]: cluster 2024-03-04T15:52:21.371144+0000 mgr.y (mgr.24428) 93 : cluster 0 pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:22.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:22 smithi117 bash[28843]: cluster 2024-03-04T15:52:21.371144+0000 mgr.y (mgr.24428) 93 : cluster 0 pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:22.819 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:52:22 smithi117 bash[62773]: ts=2024-03-04T15:52:22.541Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000202714s 2024-03-04T15:52:24.428 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:24 smithi151 systemd[1]: Stopping Ceph prometheus.a for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T15:52:24.428 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:24 smithi151 bash[49217]: ts=2024-03-04T15:52:24.277Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-03-04T15:52:24.428 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:24 smithi151 bash[49217]: ts=2024-03-04T15:52:24.277Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-03-04T15:52:24.428 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:24 smithi151 bash[49217]: ts=2024-03-04T15:52:24.277Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-03-04T15:52:24.428 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:24 smithi151 bash[49217]: ts=2024-03-04T15:52:24.277Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-03-04T15:52:24.428 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:24 smithi151 bash[49217]: ts=2024-03-04T15:52:24.277Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-03-04T15:52:24.428 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:24 smithi151 bash[49217]: ts=2024-03-04T15:52:24.277Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-03-04T15:52:24.428 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:24 smithi151 bash[49217]: ts=2024-03-04T15:52:24.277Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-03-04T15:52:24.429 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:24 smithi151 bash[49217]: ts=2024-03-04T15:52:24.277Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-03-04T15:52:24.429 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:24 smithi151 bash[49217]: ts=2024-03-04T15:52:24.277Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-03-04T15:52:24.429 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:24 smithi151 bash[49217]: ts=2024-03-04T15:52:24.278Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-03-04T15:52:24.429 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:24 smithi151 bash[49217]: ts=2024-03-04T15:52:24.278Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-03-04T15:52:24.429 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:24 smithi151 bash[49217]: ts=2024-03-04T15:52:24.279Z caller=main.go:1147 level=info msg="See you next time!" 2024-03-04T15:52:24.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:24 smithi151 bash[23756]: cluster 2024-03-04T15:52:23.371849+0000 mgr.y (mgr.24428) 94 : cluster 0 pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:24.787 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:52:24.809 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:24 smithi117 bash[21480]: cluster 2024-03-04T15:52:23.371849+0000 mgr.y (mgr.24428) 94 : cluster 0 pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:24.809 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:24 smithi117 bash[28843]: cluster 2024-03-04T15:52:23.371849+0000 mgr.y (mgr.24428) 94 : cluster 0 pgmap v57: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:25.156 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:24 smithi151 bash[51714]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-prometheus-a 2024-03-04T15:52:25.156 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:25 smithi151 bash[51777]: Error response from daemon: No such container: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-prometheus-a 2024-03-04T15:52:25.156 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:25 smithi151 systemd[1]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715@prometheus.a.service: Deactivated successfully. 2024-03-04T15:52:25.157 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:25 smithi151 systemd[1]: Stopped Ceph prometheus.a for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T15:52:25.157 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:25 smithi151 systemd[1]: Started Ceph prometheus.a for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T15:52:25.569 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:25 smithi117 bash[21734]: [04/Mar/2024:15:52:25] ENGINE Bus STOPPING 2024-03-04T15:52:25.569 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:25 smithi117 bash[21734]: [04/Mar/2024:15:52:25] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-04T15:52:25.569 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:25 smithi117 bash[21734]: [04/Mar/2024:15:52:25] ENGINE Bus STOPPED 2024-03-04T15:52:25.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:25 smithi117 bash[21734]: [04/Mar/2024:15:52:25] ENGINE Bus STARTING 2024-03-04T15:52:25.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:25 smithi117 bash[21734]: [04/Mar/2024:15:52:25] ENGINE Serving on http://:::9283 2024-03-04T15:52:25.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:25 smithi117 bash[21734]: [04/Mar/2024:15:52:25] ENGINE Bus STARTED 2024-03-04T15:52:25.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:25 smithi117 bash[21734]: [04/Mar/2024:15:52:25] ENGINE Bus STOPPING 2024-03-04T15:52:25.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:25 smithi117 bash[21734]: [04/Mar/2024:15:52:25] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-04T15:52:25.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:25 smithi117 bash[21734]: [04/Mar/2024:15:52:25] ENGINE Bus STOPPED 2024-03-04T15:52:25.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:25 smithi117 bash[21734]: [04/Mar/2024:15:52:25] ENGINE Bus STARTING 2024-03-04T15:52:25.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:25 smithi117 bash[21734]: [04/Mar/2024:15:52:25] ENGINE Serving on http://:::9283 2024-03-04T15:52:25.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:25 smithi117 bash[21734]: [04/Mar/2024:15:52:25] ENGINE Bus STARTED 2024-03-04T15:52:25.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:25 smithi117 bash[21734]: [04/Mar/2024:15:52:25] ENGINE Bus STOPPING 2024-03-04T15:52:25.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:25 smithi117 bash[21734]: [04/Mar/2024:15:52:25] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-04T15:52:25.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:25 smithi117 bash[21734]: [04/Mar/2024:15:52:25] ENGINE Bus STOPPED 2024-03-04T15:52:25.570 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:25 smithi117 bash[21734]: [04/Mar/2024:15:52:25] ENGINE Bus STARTING 2024-03-04T15:52:25.844 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:25 smithi117 bash[21734]: [04/Mar/2024:15:52:25] ENGINE Serving on http://:::9283 2024-03-04T15:52:25.844 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:25 smithi117 bash[21734]: [04/Mar/2024:15:52:25] ENGINE Bus STARTED 2024-03-04T15:52:26.432 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:52:26.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[23756]: audit 2024-03-04T15:52:25.162143+0000 mon.a (mon.0) 832 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:26.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[23756]: audit 2024-03-04T15:52:25.178053+0000 mon.a (mon.0) 833 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:26.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[23756]: audit 2024-03-04T15:52:25.207867+0000 mon.c (mon.2) 70 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-04T15:52:26.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[23756]: audit 2024-03-04T15:52:25.211122+0000 mon.c (mon.2) 71 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi117.front.sepia.ceph.com:9093"}]: dispatch 2024-03-04T15:52:26.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[23756]: audit 2024-03-04T15:52:25.221579+0000 mon.a (mon.0) 834 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:26.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[23756]: audit 2024-03-04T15:52:25.247338+0000 mon.c (mon.2) 72 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T15:52:26.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[23756]: audit 2024-03-04T15:52:25.252015+0000 mon.c (mon.2) 73 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi151.front.sepia.ceph.com:9095"}]: dispatch 2024-03-04T15:52:26.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[23756]: audit 2024-03-04T15:52:25.264159+0000 mon.a (mon.0) 835 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:26.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[23756]: audit 2024-03-04T15:52:25.282660+0000 mon.c (mon.2) 74 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-04T15:52:26.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[23756]: audit 2024-03-04T15:52:25.288024+0000 mon.c (mon.2) 75 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi151.front.sepia.ceph.com:3000"}]: dispatch 2024-03-04T15:52:26.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[23756]: audit 2024-03-04T15:52:25.294571+0000 mon.a (mon.0) 836 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:26.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[23756]: audit 2024-03-04T15:52:25.318237+0000 mon.c (mon.2) 76 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:52:26.433 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.315Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-03-04T15:52:26.433 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.315Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-03-04T15:52:26.433 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.315Z caller=main.go:570 level=info host_details="(Linux 5.15.0-97-generic #107-Ubuntu SMP Wed Feb 7 13:26:48 UTC 2024 x86_64 smithi151 (none))" 2024-03-04T15:52:26.433 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.315Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-03-04T15:52:26.433 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.315Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-03-04T15:52:26.433 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.317Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-03-04T15:52:26.433 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.317Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-03-04T15:52:26.434 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.318Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-03-04T15:52:26.434 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.318Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-03-04T15:52:26.434 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.321Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-03-04T15:52:26.434 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.321Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=2.14µs 2024-03-04T15:52:26.434 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.321Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-03-04T15:52:26.434 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.322Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-03-04T15:52:26.434 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.322Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-03-04T15:52:26.434 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.322Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=25.178µs wal_replay_duration=978.413µs wbl_replay_duration=169ns total_replay_duration=1.02441ms 2024-03-04T15:52:26.434 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.323Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-03-04T15:52:26.434 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.323Z caller=main.go:1029 level=info msg="TSDB started" 2024-03-04T15:52:26.434 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.323Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-03-04T15:52:26.434 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.348Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=24.187741ms db_storage=922ns remote_storage=1.302µs web_handler=468ns query_engine=1.102µs scrape=286.781µs scrape_sd=99.708µs notify=13.185µs notify_sd=10.366µs rules=23.411861ms tracing=6.474µs 2024-03-04T15:52:26.434 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.348Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-03-04T15:52:26.434 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 15:52:26 smithi151 bash[51804]: ts=2024-03-04T15:52:26.348Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-03-04T15:52:26.496 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[28843]: audit 2024-03-04T15:52:25.162143+0000 mon.a (mon.0) 832 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:26.496 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[28843]: audit 2024-03-04T15:52:25.178053+0000 mon.a (mon.0) 833 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:26.496 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[28843]: audit 2024-03-04T15:52:25.207867+0000 mon.c (mon.2) 70 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-04T15:52:26.496 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[28843]: audit 2024-03-04T15:52:25.211122+0000 mon.c (mon.2) 71 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi117.front.sepia.ceph.com:9093"}]: dispatch 2024-03-04T15:52:26.496 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[28843]: audit 2024-03-04T15:52:25.221579+0000 mon.a (mon.0) 834 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:26.496 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[28843]: audit 2024-03-04T15:52:25.247338+0000 mon.c (mon.2) 72 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T15:52:26.496 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[28843]: audit 2024-03-04T15:52:25.252015+0000 mon.c (mon.2) 73 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi151.front.sepia.ceph.com:9095"}]: dispatch 2024-03-04T15:52:26.496 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[28843]: audit 2024-03-04T15:52:25.264159+0000 mon.a (mon.0) 835 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:26.496 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[28843]: audit 2024-03-04T15:52:25.282660+0000 mon.c (mon.2) 74 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-04T15:52:26.496 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[28843]: audit 2024-03-04T15:52:25.288024+0000 mon.c (mon.2) 75 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi151.front.sepia.ceph.com:3000"}]: dispatch 2024-03-04T15:52:26.496 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[28843]: audit 2024-03-04T15:52:25.294571+0000 mon.a (mon.0) 836 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:26.496 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[28843]: audit 2024-03-04T15:52:25.318237+0000 mon.c (mon.2) 76 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:52:26.497 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[21480]: audit 2024-03-04T15:52:25.162143+0000 mon.a (mon.0) 832 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:26.497 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[21480]: audit 2024-03-04T15:52:25.178053+0000 mon.a (mon.0) 833 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:26.497 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[21480]: audit 2024-03-04T15:52:25.207867+0000 mon.c (mon.2) 70 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-04T15:52:26.497 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[21480]: audit 2024-03-04T15:52:25.211122+0000 mon.c (mon.2) 71 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi117.front.sepia.ceph.com:9093"}]: dispatch 2024-03-04T15:52:26.497 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[21480]: audit 2024-03-04T15:52:25.221579+0000 mon.a (mon.0) 834 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:26.497 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[21480]: audit 2024-03-04T15:52:25.247338+0000 mon.c (mon.2) 72 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T15:52:26.497 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[21480]: audit 2024-03-04T15:52:25.252015+0000 mon.c (mon.2) 73 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi151.front.sepia.ceph.com:9095"}]: dispatch 2024-03-04T15:52:26.497 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[21480]: audit 2024-03-04T15:52:25.264159+0000 mon.a (mon.0) 835 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:26.497 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[21480]: audit 2024-03-04T15:52:25.282660+0000 mon.c (mon.2) 74 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-04T15:52:26.497 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[21480]: audit 2024-03-04T15:52:25.288024+0000 mon.c (mon.2) 75 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi151.front.sepia.ceph.com:3000"}]: dispatch 2024-03-04T15:52:26.497 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[21480]: audit 2024-03-04T15:52:25.294571+0000 mon.a (mon.0) 836 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:26.497 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:26 smithi117 bash[21480]: audit 2024-03-04T15:52:25.318237+0000 mon.c (mon.2) 76 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:52:27.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:27 smithi151 bash[23756]: audit 2024-03-04T15:52:25.208972+0000 mgr.y (mgr.24428) 95 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-04T15:52:27.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:27 smithi151 bash[23756]: audit 2024-03-04T15:52:25.212043+0000 mgr.y (mgr.24428) 96 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi117.front.sepia.ceph.com:9093"}]: dispatch 2024-03-04T15:52:27.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:27 smithi151 bash[23756]: audit 2024-03-04T15:52:25.248649+0000 mgr.y (mgr.24428) 97 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T15:52:27.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:27 smithi151 bash[23756]: audit 2024-03-04T15:52:25.253309+0000 mgr.y (mgr.24428) 98 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi151.front.sepia.ceph.com:9095"}]: dispatch 2024-03-04T15:52:27.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:27 smithi151 bash[23756]: audit 2024-03-04T15:52:25.283550+0000 mgr.y (mgr.24428) 99 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-04T15:52:27.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:27 smithi151 bash[23756]: audit 2024-03-04T15:52:25.288934+0000 mgr.y (mgr.24428) 100 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi151.front.sepia.ceph.com:3000"}]: dispatch 2024-03-04T15:52:27.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:27 smithi151 bash[23756]: cluster 2024-03-04T15:52:25.372327+0000 mgr.y (mgr.24428) 101 : cluster 0 pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:27.456 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f61251cc700 1 -- 172.21.15.117:0/3979950414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6120102880 msgr2=0x7f612010ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:27.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f61251cc700 1 --2- 172.21.15.117:0/3979950414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6120102880 0x7f612010ec70 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f611800b0a0 tx=0x7f611800b3b0 comp rx=0 tx=0).stop 2024-03-04T15:52:27.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f61251cc700 1 -- 172.21.15.117:0/3979950414 shutdown_connections 2024-03-04T15:52:27.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f61251cc700 1 --2- 172.21.15.117:0/3979950414 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6120102880 0x7f612010ec70 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:27.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f61251cc700 1 --2- 172.21.15.117:0/3979950414 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6120107f30 0x7f6120108310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:27.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f61251cc700 1 --2- 172.21.15.117:0/3979950414 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6120101ec0 0x7f6120102340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:27.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f61251cc700 1 -- 172.21.15.117:0/3979950414 >> 172.21.15.117:0/3979950414 conn(0x7f6120078eb0 msgr2=0x7f61200792c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:27.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f61251cc700 1 -- 172.21.15.117:0/3979950414 shutdown_connections 2024-03-04T15:52:27.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f61251cc700 1 -- 172.21.15.117:0/3979950414 wait complete. 2024-03-04T15:52:27.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f61251cc700 1 Processor -- start 2024-03-04T15:52:27.457 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f61251cc700 1 -- start start 2024-03-04T15:52:27.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f61251cc700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6120101ec0 0x7f612019d710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:27.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f61251cc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6120102880 0x7f612019dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:27.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f61251cc700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6120107f30 0x7f6120197920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:27.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f61251cc700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61201116a0 con 0x7f6120102880 2024-03-04T15:52:27.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f61251cc700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6120111520 con 0x7f6120101ec0 2024-03-04T15:52:27.459 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f61251cc700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6120111820 con 0x7f6120107f30 2024-03-04T15:52:27.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f611e59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6120102880 0x7f612019dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:27.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f611e59c700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6120102880 0x7f612019dc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42498/0 (socket says 172.21.15.117:42498) 2024-03-04T15:52:27.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f611e59c700 1 -- 172.21.15.117:0/1816520572 learned_addr learned my addr 172.21.15.117:0/1816520572 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:52:27.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f611f59e700 1 --2- 172.21.15.117:0/1816520572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6120107f30 0x7f6120197920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:27.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f611ed9d700 1 --2- 172.21.15.117:0/1816520572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6120101ec0 0x7f612019d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:27.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f611e59c700 1 -- 172.21.15.117:0/1816520572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6120107f30 msgr2=0x7f6120197920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:27.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f611e59c700 1 --2- 172.21.15.117:0/1816520572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6120107f30 0x7f6120197920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:27.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f611e59c700 1 -- 172.21.15.117:0/1816520572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6120101ec0 msgr2=0x7f612019d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:27.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f611e59c700 1 --2- 172.21.15.117:0/1816520572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6120101ec0 0x7f612019d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:27.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f611e59c700 1 -- 172.21.15.117:0/1816520572 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6120198080 con 0x7f6120102880 2024-03-04T15:52:27.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f611ed9d700 1 --2- 172.21.15.117:0/1816520572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6120101ec0 0x7f612019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-03-04T15:52:27.460 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f611e59c700 1 --2- 172.21.15.117:0/1816520572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6120102880 0x7f612019dc50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f611400eb30 tx=0x7f611400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:27.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f6107fff700 1 -- 172.21.15.117:0/1816520572 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611400cc90 con 0x7f6120102880 2024-03-04T15:52:27.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f6107fff700 1 -- 172.21.15.117:0/1816520572 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f611400ce30 con 0x7f6120102880 2024-03-04T15:52:27.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f61251cc700 1 -- 172.21.15.117:0/1816520572 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6120198370 con 0x7f6120102880 2024-03-04T15:52:27.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f6107fff700 1 -- 172.21.15.117:0/1816520572 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6114003d80 con 0x7f6120102880 2024-03-04T15:52:27.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.454+0000 7f61251cc700 1 -- 172.21.15.117:0/1816520572 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61201a45e0 con 0x7f6120102880 2024-03-04T15:52:27.461 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.458+0000 7f61251cc700 1 -- 172.21.15.117:0/1816520572 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f612004faa0 con 0x7f6120102880 2024-03-04T15:52:27.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.458+0000 7f6107fff700 1 -- 172.21.15.117:0/1816520572 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f6114003f20 con 0x7f6120102880 2024-03-04T15:52:27.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.458+0000 7f6107fff700 1 --2- 172.21.15.117:0/1816520572 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f610806d780 0x7f610806fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:27.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.458+0000 7f6107fff700 1 -- 172.21.15.117:0/1816520572 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7f611408dc70 con 0x7f6120102880 2024-03-04T15:52:27.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.458+0000 7f611ed9d700 1 --2- 172.21.15.117:0/1816520572 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f610806d780 0x7f610806fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:27.462 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.458+0000 7f611ed9d700 1 --2- 172.21.15.117:0/1816520572 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f610806d780 0x7f610806fc40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6110005950 tx=0x7f61100058e0 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:27.465 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.462+0000 7f6107fff700 1 -- 172.21.15.117:0/1816520572 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f611405b540 con 0x7f6120102880 2024-03-04T15:52:27.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:27 smithi117 bash[28843]: audit 2024-03-04T15:52:25.208972+0000 mgr.y (mgr.24428) 95 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-04T15:52:27.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:27 smithi117 bash[28843]: audit 2024-03-04T15:52:25.212043+0000 mgr.y (mgr.24428) 96 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi117.front.sepia.ceph.com:9093"}]: dispatch 2024-03-04T15:52:27.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:27 smithi117 bash[28843]: audit 2024-03-04T15:52:25.248649+0000 mgr.y (mgr.24428) 97 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T15:52:27.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:27 smithi117 bash[28843]: audit 2024-03-04T15:52:25.253309+0000 mgr.y (mgr.24428) 98 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi151.front.sepia.ceph.com:9095"}]: dispatch 2024-03-04T15:52:27.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:27 smithi117 bash[28843]: audit 2024-03-04T15:52:25.283550+0000 mgr.y (mgr.24428) 99 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-04T15:52:27.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:27 smithi117 bash[28843]: audit 2024-03-04T15:52:25.288934+0000 mgr.y (mgr.24428) 100 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi151.front.sepia.ceph.com:3000"}]: dispatch 2024-03-04T15:52:27.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:27 smithi117 bash[28843]: cluster 2024-03-04T15:52:25.372327+0000 mgr.y (mgr.24428) 101 : cluster 0 pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:27.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:27 smithi117 bash[21480]: audit 2024-03-04T15:52:25.208972+0000 mgr.y (mgr.24428) 95 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-03-04T15:52:27.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:27 smithi117 bash[21480]: audit 2024-03-04T15:52:25.212043+0000 mgr.y (mgr.24428) 96 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi117.front.sepia.ceph.com:9093"}]: dispatch 2024-03-04T15:52:27.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:27 smithi117 bash[21480]: audit 2024-03-04T15:52:25.248649+0000 mgr.y (mgr.24428) 97 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-03-04T15:52:27.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:27 smithi117 bash[21480]: audit 2024-03-04T15:52:25.253309+0000 mgr.y (mgr.24428) 98 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi151.front.sepia.ceph.com:9095"}]: dispatch 2024-03-04T15:52:27.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:27 smithi117 bash[21480]: audit 2024-03-04T15:52:25.283550+0000 mgr.y (mgr.24428) 99 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-03-04T15:52:27.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:27 smithi117 bash[21480]: audit 2024-03-04T15:52:25.288934+0000 mgr.y (mgr.24428) 100 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi151.front.sepia.ceph.com:3000"}]: dispatch 2024-03-04T15:52:27.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:27 smithi117 bash[21480]: cluster 2024-03-04T15:52:25.372327+0000 mgr.y (mgr.24428) 101 : cluster 0 pgmap v58: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:27.668 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.666+0000 7f61251cc700 1 -- 172.21.15.117:0/1816520572 --> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6120198e00 con 0x7f610806d780 2024-03-04T15:52:27.681 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.678+0000 7f6107fff700 1 -- 172.21.15.117:0/1816520572 <== mgr.24428 v2:172.21.15.117:6800/2727720514 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+346505 (secure 0 0 0) 0x7f6120198e00 con 0x7f610806d780 2024-03-04T15:52:27.685 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-04T15:52:27.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.682+0000 7f61251cc700 1 -- 172.21.15.117:0/1816520572 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f610806d780 msgr2=0x7f610806fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:27.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.682+0000 7f61251cc700 1 --2- 172.21.15.117:0/1816520572 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f610806d780 0x7f610806fc40 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f6110005950 tx=0x7f61100058e0 comp rx=0 tx=0).stop 2024-03-04T15:52:27.688 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.682+0000 7f61251cc700 1 -- 172.21.15.117:0/1816520572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6120102880 msgr2=0x7f612019dc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:27.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.682+0000 7f61251cc700 1 --2- 172.21.15.117:0/1816520572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6120102880 0x7f612019dc50 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f611400eb30 tx=0x7f611400ee40 comp rx=0 tx=0).stop 2024-03-04T15:52:27.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.682+0000 7f61251cc700 1 -- 172.21.15.117:0/1816520572 shutdown_connections 2024-03-04T15:52:27.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.682+0000 7f61251cc700 1 --2- 172.21.15.117:0/1816520572 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6120107f30 0x7f6120197920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:27.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.682+0000 7f61251cc700 1 --2- 172.21.15.117:0/1816520572 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6120101ec0 0x7f612019d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:27.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.682+0000 7f61251cc700 1 --2- 172.21.15.117:0/1816520572 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f610806d780 0x7f610806fc40 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:27.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.682+0000 7f61251cc700 1 --2- 172.21.15.117:0/1816520572 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6120102880 0x7f612019dc50 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:27.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.682+0000 7f61251cc700 1 -- 172.21.15.117:0/1816520572 >> 172.21.15.117:0/1816520572 conn(0x7f6120078eb0 msgr2=0x7f6120100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:27.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.682+0000 7f61251cc700 1 -- 172.21.15.117:0/1816520572 shutdown_connections 2024-03-04T15:52:27.689 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:27.682+0000 7f61251cc700 1 -- 172.21.15.117:0/1816520572 wait complete. 2024-03-04T15:52:27.689 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2024-03-04T15:52:28.359 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":59,"stamp":"2024-03-04T15:52:27.372538+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596318,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1099,"num_read_kb":945,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":626,"ondisk_log_size":626,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":253624,"kb_used_data":8348,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":749707592,"statfs":{"total":767960285184,"available":767700574208,"internally_reserved":0,"allocated":8548352,"data_stored":4629802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004976"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.210539+0000","last_change":"2024-03-04T15:49:40.454414+0000","last_active":"2024-03-04T15:50:35.210539+0000","last_peered":"2024-03-04T15:50:35.210539+0000","last_clean":"2024-03-04T15:50:35.210539+0000","last_became_active":"2024-03-04T15:49:40.453465+0000","last_became_peered":"2024-03-04T15:49:40.453465+0000","last_unstale":"2024-03-04T15:50:35.210539+0000","last_undegraded":"2024-03-04T15:50:35.210539+0000","last_fullsized":"2024-03-04T15:50:35.210539+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:52:16.951314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.210327+0000","last_change":"2024-03-04T15:49:38.008560+0000","last_active":"2024-03-04T15:50:35.210327+0000","last_peered":"2024-03-04T15:50:35.210327+0000","last_clean":"2024-03-04T15:50:35.210327+0000","last_became_active":"2024-03-04T15:49:38.007045+0000","last_became_peered":"2024-03-04T15:49:38.007045+0000","last_unstale":"2024-03-04T15:50:35.210327+0000","last_undegraded":"2024-03-04T15:50:35.210327+0000","last_fullsized":"2024-03-04T15:50:35.210327+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:54:31.777548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":46,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.215583+0000","last_change":"2024-03-04T15:50:24.435625+0000","last_active":"2024-03-04T15:50:35.215583+0000","last_peered":"2024-03-04T15:50:35.215583+0000","last_clean":"2024-03-04T15:50:35.215583+0000","last_became_active":"2024-03-04T15:50:24.434643+0000","last_became_peered":"2024-03-04T15:50:24.434643+0000","last_unstale":"2024-03-04T15:50:35.215583+0000","last_undegraded":"2024-03-04T15:50:35.215583+0000","last_fullsized":"2024-03-04T15:50:35.215583+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T02:43:46.528518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"68'10","reported_seq":50,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212630+0000","last_change":"2024-03-04T15:49:36.019526+0000","last_active":"2024-03-04T15:50:35.212630+0000","last_peered":"2024-03-04T15:50:35.212630+0000","last_clean":"2024-03-04T15:50:35.212630+0000","last_became_active":"2024-03-04T15:49:36.018951+0000","last_became_peered":"2024-03-04T15:49:36.018951+0000","last_unstale":"2024-03-04T15:50:35.212630+0000","last_undegraded":"2024-03-04T15:50:35.212630+0000","last_fullsized":"2024-03-04T15:50:35.212630+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:20:23.324722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.206505+0000","last_change":"2024-03-04T15:49:40.233909+0000","last_active":"2024-03-04T15:50:35.206505+0000","last_peered":"2024-03-04T15:50:35.206505+0000","last_clean":"2024-03-04T15:50:35.206505+0000","last_became_active":"2024-03-04T15:49:40.232809+0000","last_became_peered":"2024-03-04T15:49:40.232809+0000","last_unstale":"2024-03-04T15:50:35.206505+0000","last_undegraded":"2024-03-04T15:50:35.206505+0000","last_fullsized":"2024-03-04T15:50:35.206505+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:18:55.213547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.206019+0000","last_change":"2024-03-04T15:49:38.021094+0000","last_active":"2024-03-04T15:50:35.206019+0000","last_peered":"2024-03-04T15:50:35.206019+0000","last_clean":"2024-03-04T15:50:35.206019+0000","last_became_active":"2024-03-04T15:49:38.020683+0000","last_became_peered":"2024-03-04T15:49:38.020683+0000","last_unstale":"2024-03-04T15:50:35.206019+0000","last_undegraded":"2024-03-04T15:50:35.206019+0000","last_fullsized":"2024-03-04T15:50:35.206019+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:19:23.943917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212430+0000","last_change":"2024-03-04T15:49:33.722796+0000","last_active":"2024-03-04T15:50:35.212430+0000","last_peered":"2024-03-04T15:50:35.212430+0000","last_clean":"2024-03-04T15:50:35.212430+0000","last_became_active":"2024-03-04T15:49:33.722358+0000","last_became_peered":"2024-03-04T15:49:33.722358+0000","last_unstale":"2024-03-04T15:50:35.212430+0000","last_undegraded":"2024-03-04T15:50:35.212430+0000","last_fullsized":"2024-03-04T15:50:35.212430+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:39:33.208389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"68'11","reported_seq":55,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.215322+0000","last_change":"2024-03-04T15:49:36.019614+0000","last_active":"2024-03-04T15:50:35.215322+0000","last_peered":"2024-03-04T15:50:35.215322+0000","last_clean":"2024-03-04T15:50:35.215322+0000","last_became_active":"2024-03-04T15:49:36.019181+0000","last_became_peered":"2024-03-04T15:49:36.019181+0000","last_unstale":"2024-03-04T15:50:35.215322+0000","last_undegraded":"2024-03-04T15:50:35.215322+0000","last_fullsized":"2024-03-04T15:50:35.215322+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:12:28.372013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123291+0000","last_change":"2024-03-04T15:49:40.234694+0000","last_active":"2024-03-04T15:50:36.123291+0000","last_peered":"2024-03-04T15:50:36.123291+0000","last_clean":"2024-03-04T15:50:36.123291+0000","last_became_active":"2024-03-04T15:49:40.233800+0000","last_became_peered":"2024-03-04T15:49:40.233800+0000","last_unstale":"2024-03-04T15:50:36.123291+0000","last_undegraded":"2024-03-04T15:50:36.123291+0000","last_fullsized":"2024-03-04T15:50:36.123291+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:24:31.116692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.206798+0000","last_change":"2024-03-04T15:50:25.225075+0000","last_active":"2024-03-04T15:50:35.206798+0000","last_peered":"2024-03-04T15:50:35.206798+0000","last_clean":"2024-03-04T15:50:35.206798+0000","last_became_active":"2024-03-04T15:50:25.224111+0000","last_became_peered":"2024-03-04T15:50:25.224111+0000","last_unstale":"2024-03-04T15:50:35.206798+0000","last_undegraded":"2024-03-04T15:50:35.206798+0000","last_fullsized":"2024-03-04T15:50:35.206798+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:36:29.463751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124697+0000","last_change":"2024-03-04T15:49:33.745748+0000","last_active":"2024-03-04T15:50:36.124697+0000","last_peered":"2024-03-04T15:50:36.124697+0000","last_clean":"2024-03-04T15:50:36.124697+0000","last_became_active":"2024-03-04T15:49:33.745020+0000","last_became_peered":"2024-03-04T15:49:33.745020+0000","last_unstale":"2024-03-04T15:50:36.124697+0000","last_undegraded":"2024-03-04T15:50:36.124697+0000","last_fullsized":"2024-03-04T15:50:36.124697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:49:27.659528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"68'15","reported_seq":61,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123893+0000","last_change":"2024-03-04T15:49:35.756011+0000","last_active":"2024-03-04T15:50:36.123893+0000","last_peered":"2024-03-04T15:50:36.123893+0000","last_clean":"2024-03-04T15:50:36.123893+0000","last_became_active":"2024-03-04T15:49:35.753803+0000","last_became_peered":"2024-03-04T15:49:35.753803+0000","last_unstale":"2024-03-04T15:50:36.123893+0000","last_undegraded":"2024-03-04T15:50:36.123893+0000","last_fullsized":"2024-03-04T15:50:36.123893+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:17:08.406115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.209409+0000","last_change":"2024-03-04T15:49:40.223844+0000","last_active":"2024-03-04T15:50:35.209409+0000","last_peered":"2024-03-04T15:50:35.209409+0000","last_clean":"2024-03-04T15:50:35.209409+0000","last_became_active":"2024-03-04T15:49:40.223090+0000","last_became_peered":"2024-03-04T15:49:40.223090+0000","last_unstale":"2024-03-04T15:50:35.209409+0000","last_undegraded":"2024-03-04T15:50:35.209409+0000","last_fullsized":"2024-03-04T15:50:35.209409+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:02:43.864367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.209646+0000","last_change":"2024-03-04T15:49:38.005549+0000","last_active":"2024-03-04T15:50:35.209646+0000","last_peered":"2024-03-04T15:50:35.209646+0000","last_clean":"2024-03-04T15:50:35.209646+0000","last_became_active":"2024-03-04T15:49:38.005216+0000","last_became_peered":"2024-03-04T15:49:38.005216+0000","last_unstale":"2024-03-04T15:50:35.209646+0000","last_undegraded":"2024-03-04T15:50:35.209646+0000","last_fullsized":"2024-03-04T15:50:35.209646+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:24.628355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124270+0000","last_change":"2024-03-04T15:49:33.742659+0000","last_active":"2024-03-04T15:50:36.124270+0000","last_peered":"2024-03-04T15:50:36.124270+0000","last_clean":"2024-03-04T15:50:36.124270+0000","last_became_active":"2024-03-04T15:49:33.742250+0000","last_became_peered":"2024-03-04T15:49:33.742250+0000","last_unstale":"2024-03-04T15:50:36.124270+0000","last_undegraded":"2024-03-04T15:50:36.124270+0000","last_fullsized":"2024-03-04T15:50:36.124270+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:22:39.463025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"68'12","reported_seq":59,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124191+0000","last_change":"2024-03-04T15:49:35.761336+0000","last_active":"2024-03-04T15:50:36.124191+0000","last_peered":"2024-03-04T15:50:36.124191+0000","last_clean":"2024-03-04T15:50:36.124191+0000","last_became_active":"2024-03-04T15:49:35.760529+0000","last_became_peered":"2024-03-04T15:49:35.760529+0000","last_unstale":"2024-03-04T15:50:36.124191+0000","last_undegraded":"2024-03-04T15:50:36.124191+0000","last_fullsized":"2024-03-04T15:50:36.124191+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:34:24.578708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.211629+0000","last_change":"2024-03-04T15:49:40.450091+0000","last_active":"2024-03-04T15:50:35.211629+0000","last_peered":"2024-03-04T15:50:35.211629+0000","last_clean":"2024-03-04T15:50:35.211629+0000","last_became_active":"2024-03-04T15:49:40.448918+0000","last_became_peered":"2024-03-04T15:49:40.448918+0000","last_unstale":"2024-03-04T15:50:35.211629+0000","last_undegraded":"2024-03-04T15:50:35.211629+0000","last_fullsized":"2024-03-04T15:50:35.211629+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:07:41.121850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212016+0000","last_change":"2024-03-04T15:49:38.025651+0000","last_active":"2024-03-04T15:50:35.212016+0000","last_peered":"2024-03-04T15:50:35.212016+0000","last_clean":"2024-03-04T15:50:35.212016+0000","last_became_active":"2024-03-04T15:49:38.024369+0000","last_became_peered":"2024-03-04T15:49:38.024369+0000","last_unstale":"2024-03-04T15:50:35.212016+0000","last_undegraded":"2024-03-04T15:50:35.212016+0000","last_fullsized":"2024-03-04T15:50:35.212016+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:30:02.409366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.210762+0000","last_change":"2024-03-04T15:49:33.734073+0000","last_active":"2024-03-04T15:50:35.210762+0000","last_peered":"2024-03-04T15:50:35.210762+0000","last_clean":"2024-03-04T15:50:35.210762+0000","last_became_active":"2024-03-04T15:49:33.733597+0000","last_became_peered":"2024-03-04T15:49:33.733597+0000","last_unstale":"2024-03-04T15:50:35.210762+0000","last_undegraded":"2024-03-04T15:50:35.210762+0000","last_fullsized":"2024-03-04T15:50:35.210762+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T03:22:06.068088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"68'9","reported_seq":51,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.206775+0000","last_change":"2024-03-04T15:49:36.020344+0000","last_active":"2024-03-04T15:50:35.206775+0000","last_peered":"2024-03-04T15:50:35.206775+0000","last_clean":"2024-03-04T15:50:35.206775+0000","last_became_active":"2024-03-04T15:49:36.018786+0000","last_became_peered":"2024-03-04T15:49:36.018786+0000","last_unstale":"2024-03-04T15:50:35.206775+0000","last_undegraded":"2024-03-04T15:50:35.206775+0000","last_fullsized":"2024-03-04T15:50:35.206775+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:44:52.178972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212482+0000","last_change":"2024-03-04T15:49:40.448772+0000","last_active":"2024-03-04T15:50:35.212482+0000","last_peered":"2024-03-04T15:50:35.212482+0000","last_clean":"2024-03-04T15:50:35.212482+0000","last_became_active":"2024-03-04T15:49:40.447880+0000","last_became_peered":"2024-03-04T15:49:40.447880+0000","last_unstale":"2024-03-04T15:50:35.212482+0000","last_undegraded":"2024-03-04T15:50:35.212482+0000","last_fullsized":"2024-03-04T15:50:35.212482+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:14:12.689534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124780+0000","last_change":"2024-03-04T15:49:38.018359+0000","last_active":"2024-03-04T15:50:36.124780+0000","last_peered":"2024-03-04T15:50:36.124780+0000","last_clean":"2024-03-04T15:50:36.124780+0000","last_became_active":"2024-03-04T15:49:38.017460+0000","last_became_peered":"2024-03-04T15:49:38.017460+0000","last_unstale":"2024-03-04T15:50:36.124780+0000","last_undegraded":"2024-03-04T15:50:36.124780+0000","last_fullsized":"2024-03-04T15:50:36.124780+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T02:34:35.302959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.352368+0000","last_change":"2024-03-04T15:49:33.745021+0000","last_active":"2024-03-04T15:50:35.352368+0000","last_peered":"2024-03-04T15:50:35.352368+0000","last_clean":"2024-03-04T15:50:35.352368+0000","last_became_active":"2024-03-04T15:49:33.744313+0000","last_became_peered":"2024-03-04T15:49:33.744313+0000","last_unstale":"2024-03-04T15:50:35.352368+0000","last_undegraded":"2024-03-04T15:50:35.352368+0000","last_fullsized":"2024-03-04T15:50:35.352368+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:34:48.854618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"68'5","reported_seq":46,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.211729+0000","last_change":"2024-03-04T15:49:35.762179+0000","last_active":"2024-03-04T15:50:35.211729+0000","last_peered":"2024-03-04T15:50:35.211729+0000","last_clean":"2024-03-04T15:50:35.211729+0000","last_became_active":"2024-03-04T15:49:35.761588+0000","last_became_peered":"2024-03-04T15:49:35.761588+0000","last_unstale":"2024-03-04T15:50:35.211729+0000","last_undegraded":"2024-03-04T15:50:35.211729+0000","last_fullsized":"2024-03-04T15:50:35.211729+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:58:43.444571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124273+0000","last_change":"2024-03-04T15:49:40.230342+0000","last_active":"2024-03-04T15:50:36.124273+0000","last_peered":"2024-03-04T15:50:36.124273+0000","last_clean":"2024-03-04T15:50:36.124273+0000","last_became_active":"2024-03-04T15:49:40.229448+0000","last_became_peered":"2024-03-04T15:49:40.229448+0000","last_unstale":"2024-03-04T15:50:36.124273+0000","last_undegraded":"2024-03-04T15:50:36.124273+0000","last_fullsized":"2024-03-04T15:50:36.124273+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:50:39.830014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1f","version":"68'11","reported_seq":80,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:18.952698+0000","last_change":"2024-03-04T15:49:38.025404+0000","last_active":"2024-03-04T15:52:18.952698+0000","last_peered":"2024-03-04T15:52:18.952698+0000","last_clean":"2024-03-04T15:52:18.952698+0000","last_became_active":"2024-03-04T15:49:38.024652+0000","last_became_peered":"2024-03-04T15:49:38.024652+0000","last_unstale":"2024-03-04T15:52:18.952698+0000","last_undegraded":"2024-03-04T15:52:18.952698+0000","last_fullsized":"2024-03-04T15:52:18.952698+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:31:43.271841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"61'1","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.210629+0000","last_change":"2024-03-04T15:49:33.729239+0000","last_active":"2024-03-04T15:50:35.210629+0000","last_peered":"2024-03-04T15:50:35.210629+0000","last_clean":"2024-03-04T15:50:35.210629+0000","last_became_active":"2024-03-04T15:49:33.728380+0000","last_became_peered":"2024-03-04T15:49:33.728380+0000","last_unstale":"2024-03-04T15:50:35.210629+0000","last_undegraded":"2024-03-04T15:50:35.210629+0000","last_fullsized":"2024-03-04T15:50:35.210629+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:49:57.697270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"68'9","reported_seq":51,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212220+0000","last_change":"2024-03-04T15:49:35.745777+0000","last_active":"2024-03-04T15:50:35.212220+0000","last_peered":"2024-03-04T15:50:35.212220+0000","last_clean":"2024-03-04T15:50:35.212220+0000","last_became_active":"2024-03-04T15:49:35.744738+0000","last_became_peered":"2024-03-04T15:49:35.744738+0000","last_unstale":"2024-03-04T15:50:35.212220+0000","last_undegraded":"2024-03-04T15:50:35.212220+0000","last_fullsized":"2024-03-04T15:50:35.212220+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:00:50.454045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"68'1","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124114+0000","last_change":"2024-03-04T15:49:40.224804+0000","last_active":"2024-03-04T15:50:36.124114+0000","last_peered":"2024-03-04T15:50:36.124114+0000","last_clean":"2024-03-04T15:50:36.124114+0000","last_became_active":"2024-03-04T15:49:40.222546+0000","last_became_peered":"2024-03-04T15:49:40.222546+0000","last_unstale":"2024-03-04T15:50:36.124114+0000","last_undegraded":"2024-03-04T15:50:36.124114+0000","last_fullsized":"2024-03-04T15:50:36.124114+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T02:48:38.643953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":370,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.214105+0000","last_change":"2024-03-04T15:49:38.154004+0000","last_active":"2024-03-04T15:50:35.214105+0000","last_peered":"2024-03-04T15:50:35.214105+0000","last_clean":"2024-03-04T15:50:35.214105+0000","last_became_active":"2024-03-04T15:49:38.153103+0000","last_became_peered":"2024-03-04T15:49:38.153103+0000","last_unstale":"2024-03-04T15:50:35.214105+0000","last_undegraded":"2024-03-04T15:50:35.214105+0000","last_fullsized":"2024-03-04T15:50:35.214105+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T02:25:19.106156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"61'1","reported_seq":48,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123198+0000","last_change":"2024-03-04T15:49:33.733649+0000","last_active":"2024-03-04T15:50:36.123198+0000","last_peered":"2024-03-04T15:50:36.123198+0000","last_clean":"2024-03-04T15:50:36.123198+0000","last_became_active":"2024-03-04T15:49:33.732996+0000","last_became_peered":"2024-03-04T15:49:33.732996+0000","last_unstale":"2024-03-04T15:50:36.123198+0000","last_undegraded":"2024-03-04T15:50:36.123198+0000","last_fullsized":"2024-03-04T15:50:36.123198+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:28:42.575711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":875,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"68'15","reported_seq":61,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.125061+0000","last_change":"2024-03-04T15:49:35.741487+0000","last_active":"2024-03-04T15:50:36.125061+0000","last_peered":"2024-03-04T15:50:36.125061+0000","last_clean":"2024-03-04T15:50:36.125061+0000","last_became_active":"2024-03-04T15:49:35.740695+0000","last_became_peered":"2024-03-04T15:49:35.740695+0000","last_unstale":"2024-03-04T15:50:36.125061+0000","last_undegraded":"2024-03-04T15:50:36.125061+0000","last_fullsized":"2024-03-04T15:50:36.125061+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:58:07.692797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"68'1","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.211894+0000","last_change":"2024-03-04T15:49:40.453843+0000","last_active":"2024-03-04T15:50:35.211894+0000","last_peered":"2024-03-04T15:50:35.211894+0000","last_clean":"2024-03-04T15:50:35.211894+0000","last_became_active":"2024-03-04T15:49:40.452961+0000","last_became_peered":"2024-03-04T15:49:40.452961+0000","last_unstale":"2024-03-04T15:50:35.211894+0000","last_undegraded":"2024-03-04T15:50:35.211894+0000","last_fullsized":"2024-03-04T15:50:35.211894+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:48:08.355002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212014+0000","last_change":"2024-03-04T15:49:38.155702+0000","last_active":"2024-03-04T15:50:35.212014+0000","last_peered":"2024-03-04T15:50:35.212014+0000","last_clean":"2024-03-04T15:50:35.212014+0000","last_became_active":"2024-03-04T15:49:38.153847+0000","last_became_peered":"2024-03-04T15:49:38.153847+0000","last_unstale":"2024-03-04T15:50:35.212014+0000","last_undegraded":"2024-03-04T15:50:35.212014+0000","last_fullsized":"2024-03-04T15:50:35.212014+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:59:12.772824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.352985+0000","last_change":"2024-03-04T15:49:33.731621+0000","last_active":"2024-03-04T15:50:35.352985+0000","last_peered":"2024-03-04T15:50:35.352985+0000","last_clean":"2024-03-04T15:50:35.352985+0000","last_became_active":"2024-03-04T15:49:33.730840+0000","last_became_peered":"2024-03-04T15:49:33.730840+0000","last_unstale":"2024-03-04T15:50:35.352985+0000","last_undegraded":"2024-03-04T15:50:35.352985+0000","last_fullsized":"2024-03-04T15:50:35.352985+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:14:25.492830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"68'9","reported_seq":52,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123736+0000","last_change":"2024-03-04T15:49:35.761467+0000","last_active":"2024-03-04T15:50:36.123736+0000","last_peered":"2024-03-04T15:50:36.123736+0000","last_clean":"2024-03-04T15:50:36.123736+0000","last_became_active":"2024-03-04T15:49:35.760732+0000","last_became_peered":"2024-03-04T15:49:35.760732+0000","last_unstale":"2024-03-04T15:50:36.123736+0000","last_undegraded":"2024-03-04T15:50:36.123736+0000","last_fullsized":"2024-03-04T15:50:36.123736+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:36:54.861620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123955+0000","last_change":"2024-03-04T15:49:40.222254+0000","last_active":"2024-03-04T15:50:36.123955+0000","last_peered":"2024-03-04T15:50:36.123955+0000","last_clean":"2024-03-04T15:50:36.123955+0000","last_became_active":"2024-03-04T15:49:40.220585+0000","last_became_peered":"2024-03-04T15:49:40.220585+0000","last_unstale":"2024-03-04T15:50:36.123955+0000","last_undegraded":"2024-03-04T15:50:36.123955+0000","last_fullsized":"2024-03-04T15:50:36.123955+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:44:50.631574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212328+0000","last_change":"2024-03-04T15:49:38.023501+0000","last_active":"2024-03-04T15:50:35.212328+0000","last_peered":"2024-03-04T15:50:35.212328+0000","last_clean":"2024-03-04T15:50:35.212328+0000","last_became_active":"2024-03-04T15:49:38.022802+0000","last_became_peered":"2024-03-04T15:49:38.022802+0000","last_unstale":"2024-03-04T15:50:35.212328+0000","last_undegraded":"2024-03-04T15:50:35.212328+0000","last_fullsized":"2024-03-04T15:50:35.212328+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:57:29.028920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123591+0000","last_change":"2024-03-04T15:49:33.732988+0000","last_active":"2024-03-04T15:50:36.123591+0000","last_peered":"2024-03-04T15:50:36.123591+0000","last_clean":"2024-03-04T15:50:36.123591+0000","last_became_active":"2024-03-04T15:49:33.732188+0000","last_became_peered":"2024-03-04T15:49:33.732188+0000","last_unstale":"2024-03-04T15:50:36.123591+0000","last_undegraded":"2024-03-04T15:50:36.123591+0000","last_fullsized":"2024-03-04T15:50:36.123591+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:57:59.295570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"68'6","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.215161+0000","last_change":"2024-03-04T15:49:35.746507+0000","last_active":"2024-03-04T15:50:35.215161+0000","last_peered":"2024-03-04T15:50:35.215161+0000","last_clean":"2024-03-04T15:50:35.215161+0000","last_became_active":"2024-03-04T15:49:35.746061+0000","last_became_peered":"2024-03-04T15:49:35.746061+0000","last_unstale":"2024-03-04T15:50:35.215161+0000","last_undegraded":"2024-03-04T15:50:35.215161+0000","last_fullsized":"2024-03-04T15:50:35.215161+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T03:17:17.568851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.210016+0000","last_change":"2024-03-04T15:49:40.244574+0000","last_active":"2024-03-04T15:50:35.210016+0000","last_peered":"2024-03-04T15:50:35.210016+0000","last_clean":"2024-03-04T15:50:35.210016+0000","last_became_active":"2024-03-04T15:49:40.241779+0000","last_became_peered":"2024-03-04T15:49:40.241779+0000","last_unstale":"2024-03-04T15:50:35.210016+0000","last_undegraded":"2024-03-04T15:50:35.210016+0000","last_fullsized":"2024-03-04T15:50:35.210016+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:44:16.846417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.207070+0000","last_change":"2024-03-04T15:49:38.015865+0000","last_active":"2024-03-04T15:50:35.207070+0000","last_peered":"2024-03-04T15:50:35.207070+0000","last_clean":"2024-03-04T15:50:35.207070+0000","last_became_active":"2024-03-04T15:49:38.015580+0000","last_became_peered":"2024-03-04T15:49:38.015580+0000","last_unstale":"2024-03-04T15:50:35.207070+0000","last_undegraded":"2024-03-04T15:50:35.207070+0000","last_fullsized":"2024-03-04T15:50:35.207070+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:13:40.591696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124593+0000","last_change":"2024-03-04T15:49:33.739909+0000","last_active":"2024-03-04T15:50:36.124593+0000","last_peered":"2024-03-04T15:50:36.124593+0000","last_clean":"2024-03-04T15:50:36.124593+0000","last_became_active":"2024-03-04T15:49:33.739286+0000","last_became_peered":"2024-03-04T15:49:33.739286+0000","last_unstale":"2024-03-04T15:50:36.124593+0000","last_undegraded":"2024-03-04T15:50:36.124593+0000","last_fullsized":"2024-03-04T15:50:36.124593+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:51:22.451758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"68'10","reported_seq":50,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.211312+0000","last_change":"2024-03-04T15:49:35.752001+0000","last_active":"2024-03-04T15:50:35.211312+0000","last_peered":"2024-03-04T15:50:35.211312+0000","last_clean":"2024-03-04T15:50:35.211312+0000","last_became_active":"2024-03-04T15:49:35.751095+0000","last_became_peered":"2024-03-04T15:49:35.751095+0000","last_unstale":"2024-03-04T15:50:35.211312+0000","last_undegraded":"2024-03-04T15:50:35.211312+0000","last_fullsized":"2024-03-04T15:50:35.211312+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:10:21.653391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.209504+0000","last_change":"2024-03-04T15:49:40.226410+0000","last_active":"2024-03-04T15:50:35.209504+0000","last_peered":"2024-03-04T15:50:35.209504+0000","last_clean":"2024-03-04T15:50:35.209504+0000","last_became_active":"2024-03-04T15:49:40.224534+0000","last_became_peered":"2024-03-04T15:49:40.224534+0000","last_unstale":"2024-03-04T15:50:35.209504+0000","last_undegraded":"2024-03-04T15:50:35.209504+0000","last_fullsized":"2024-03-04T15:50:35.209504+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T02:37:17.273217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123528+0000","last_change":"2024-03-04T15:50:24.707076+0000","last_active":"2024-03-04T15:50:36.123528+0000","last_peered":"2024-03-04T15:50:36.123528+0000","last_clean":"2024-03-04T15:50:36.123528+0000","last_became_active":"2024-03-04T15:50:24.705855+0000","last_became_peered":"2024-03-04T15:50:24.705855+0000","last_unstale":"2024-03-04T15:50:36.123528+0000","last_undegraded":"2024-03-04T15:50:36.123528+0000","last_fullsized":"2024-03-04T15:50:36.123528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:30:15.354268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.352792+0000","last_change":"2024-03-04T15:49:33.723819+0000","last_active":"2024-03-04T15:50:35.352792+0000","last_peered":"2024-03-04T15:50:35.352792+0000","last_clean":"2024-03-04T15:50:35.352792+0000","last_became_active":"2024-03-04T15:49:33.723042+0000","last_became_peered":"2024-03-04T15:49:33.723042+0000","last_unstale":"2024-03-04T15:50:35.352792+0000","last_undegraded":"2024-03-04T15:50:35.352792+0000","last_fullsized":"2024-03-04T15:50:35.352792+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:29:27.784721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"68'9","reported_seq":52,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124808+0000","last_change":"2024-03-04T15:49:35.761256+0000","last_active":"2024-03-04T15:50:36.124808+0000","last_peered":"2024-03-04T15:50:36.124808+0000","last_clean":"2024-03-04T15:50:36.124808+0000","last_became_active":"2024-03-04T15:49:35.759779+0000","last_became_peered":"2024-03-04T15:49:35.759779+0000","last_unstale":"2024-03-04T15:50:36.124808+0000","last_undegraded":"2024-03-04T15:50:36.124808+0000","last_fullsized":"2024-03-04T15:50:36.124808+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:08:55.485292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.214062+0000","last_change":"2024-03-04T15:49:40.230543+0000","last_active":"2024-03-04T15:50:35.214062+0000","last_peered":"2024-03-04T15:50:35.214062+0000","last_clean":"2024-03-04T15:50:35.214062+0000","last_became_active":"2024-03-04T15:49:40.229704+0000","last_became_peered":"2024-03-04T15:49:40.229704+0000","last_unstale":"2024-03-04T15:50:35.214062+0000","last_undegraded":"2024-03-04T15:50:35.214062+0000","last_fullsized":"2024-03-04T15:50:35.214062+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:55:17.972990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"68'11","reported_seq":77,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:18.952644+0000","last_change":"2024-03-04T15:49:38.153570+0000","last_active":"2024-03-04T15:52:18.952644+0000","last_peered":"2024-03-04T15:52:18.952644+0000","last_clean":"2024-03-04T15:52:18.952644+0000","last_became_active":"2024-03-04T15:49:38.152308+0000","last_became_peered":"2024-03-04T15:49:38.152308+0000","last_unstale":"2024-03-04T15:52:18.952644+0000","last_undegraded":"2024-03-04T15:52:18.952644+0000","last_fullsized":"2024-03-04T15:52:18.952644+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:16:51.267725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.211460+0000","last_change":"2024-03-04T15:49:33.722458+0000","last_active":"2024-03-04T15:50:35.211460+0000","last_peered":"2024-03-04T15:50:35.211460+0000","last_clean":"2024-03-04T15:50:35.211460+0000","last_became_active":"2024-03-04T15:49:33.722017+0000","last_became_peered":"2024-03-04T15:49:33.722017+0000","last_unstale":"2024-03-04T15:50:35.211460+0000","last_undegraded":"2024-03-04T15:50:35.211460+0000","last_fullsized":"2024-03-04T15:50:35.211460+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:21:26.960543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"68'9","reported_seq":52,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.211273+0000","last_change":"2024-03-04T15:49:35.761391+0000","last_active":"2024-03-04T15:50:35.211273+0000","last_peered":"2024-03-04T15:50:35.211273+0000","last_clean":"2024-03-04T15:50:35.211273+0000","last_became_active":"2024-03-04T15:49:35.761044+0000","last_became_peered":"2024-03-04T15:49:35.761044+0000","last_unstale":"2024-03-04T15:50:35.211273+0000","last_undegraded":"2024-03-04T15:50:35.211273+0000","last_fullsized":"2024-03-04T15:50:35.211273+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:18:46.800261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.213350+0000","last_change":"2024-03-04T15:49:40.229860+0000","last_active":"2024-03-04T15:50:35.213350+0000","last_peered":"2024-03-04T15:50:35.213350+0000","last_clean":"2024-03-04T15:50:35.213350+0000","last_became_active":"2024-03-04T15:49:40.228885+0000","last_became_peered":"2024-03-04T15:49:40.228885+0000","last_unstale":"2024-03-04T15:50:35.213350+0000","last_undegraded":"2024-03-04T15:50:35.213350+0000","last_fullsized":"2024-03-04T15:50:35.213350+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:42:49.125554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"68'11","reported_seq":76,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:18.954319+0000","last_change":"2024-03-04T15:49:38.149577+0000","last_active":"2024-03-04T15:52:18.954319+0000","last_peered":"2024-03-04T15:52:18.954319+0000","last_clean":"2024-03-04T15:52:18.954319+0000","last_became_active":"2024-03-04T15:49:38.148679+0000","last_became_peered":"2024-03-04T15:49:38.148679+0000","last_unstale":"2024-03-04T15:52:18.954319+0000","last_undegraded":"2024-03-04T15:52:18.954319+0000","last_fullsized":"2024-03-04T15:52:18.954319+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:35:37.381831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123694+0000","last_change":"2024-03-04T15:49:33.734387+0000","last_active":"2024-03-04T15:50:36.123694+0000","last_peered":"2024-03-04T15:50:36.123694+0000","last_clean":"2024-03-04T15:50:36.123694+0000","last_became_active":"2024-03-04T15:49:33.733203+0000","last_became_peered":"2024-03-04T15:49:33.733203+0000","last_unstale":"2024-03-04T15:50:36.123694+0000","last_undegraded":"2024-03-04T15:50:36.123694+0000","last_fullsized":"2024-03-04T15:50:36.123694+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:38:58.200689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"68'11","reported_seq":55,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124494+0000","last_change":"2024-03-04T15:49:36.018304+0000","last_active":"2024-03-04T15:50:36.124494+0000","last_peered":"2024-03-04T15:50:36.124494+0000","last_clean":"2024-03-04T15:50:36.124494+0000","last_became_active":"2024-03-04T15:49:36.017280+0000","last_became_peered":"2024-03-04T15:49:36.017280+0000","last_unstale":"2024-03-04T15:50:36.124494+0000","last_undegraded":"2024-03-04T15:50:36.124494+0000","last_fullsized":"2024-03-04T15:50:36.124494+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:44.305666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124183+0000","last_change":"2024-03-04T15:49:40.449911+0000","last_active":"2024-03-04T15:50:36.124183+0000","last_peered":"2024-03-04T15:50:36.124183+0000","last_clean":"2024-03-04T15:50:36.124183+0000","last_became_active":"2024-03-04T15:49:40.448339+0000","last_became_peered":"2024-03-04T15:49:40.448339+0000","last_unstale":"2024-03-04T15:50:36.124183+0000","last_undegraded":"2024-03-04T15:50:36.124183+0000","last_fullsized":"2024-03-04T15:50:36.124183+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:05:04.659649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212132+0000","last_change":"2024-03-04T15:49:38.149380+0000","last_active":"2024-03-04T15:50:35.212132+0000","last_peered":"2024-03-04T15:50:35.212132+0000","last_clean":"2024-03-04T15:50:35.212132+0000","last_became_active":"2024-03-04T15:49:38.148466+0000","last_became_peered":"2024-03-04T15:49:38.148466+0000","last_unstale":"2024-03-04T15:50:35.212132+0000","last_undegraded":"2024-03-04T15:50:35.212132+0000","last_fullsized":"2024-03-04T15:50:35.212132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:14:17.425748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.352254+0000","last_change":"2024-03-04T15:49:33.729849+0000","last_active":"2024-03-04T15:50:35.352254+0000","last_peered":"2024-03-04T15:50:35.352254+0000","last_clean":"2024-03-04T15:50:35.352254+0000","last_became_active":"2024-03-04T15:49:33.727548+0000","last_became_peered":"2024-03-04T15:49:33.727548+0000","last_unstale":"2024-03-04T15:50:35.352254+0000","last_undegraded":"2024-03-04T15:50:35.352254+0000","last_fullsized":"2024-03-04T15:50:35.352254+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:33:15.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"68'4","reported_seq":42,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.352124+0000","last_change":"2024-03-04T15:49:35.763348+0000","last_active":"2024-03-04T15:50:35.352124+0000","last_peered":"2024-03-04T15:50:35.352124+0000","last_clean":"2024-03-04T15:50:35.352124+0000","last_became_active":"2024-03-04T15:49:35.762268+0000","last_became_peered":"2024-03-04T15:49:35.762268+0000","last_unstale":"2024-03-04T15:50:35.352124+0000","last_undegraded":"2024-03-04T15:50:35.352124+0000","last_fullsized":"2024-03-04T15:50:35.352124+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:01:10.129428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124839+0000","last_change":"2024-03-04T15:49:40.231754+0000","last_active":"2024-03-04T15:50:36.124839+0000","last_peered":"2024-03-04T15:50:36.124839+0000","last_clean":"2024-03-04T15:50:36.124839+0000","last_became_active":"2024-03-04T15:49:40.230895+0000","last_became_peered":"2024-03-04T15:49:40.230895+0000","last_unstale":"2024-03-04T15:50:36.124839+0000","last_undegraded":"2024-03-04T15:50:36.124839+0000","last_fullsized":"2024-03-04T15:50:36.124839+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:56:45.443819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.213076+0000","last_change":"2024-03-04T15:49:38.154343+0000","last_active":"2024-03-04T15:50:35.213076+0000","last_peered":"2024-03-04T15:50:35.213076+0000","last_clean":"2024-03-04T15:50:35.213076+0000","last_became_active":"2024-03-04T15:49:38.153419+0000","last_became_peered":"2024-03-04T15:49:38.153419+0000","last_unstale":"2024-03-04T15:50:35.213076+0000","last_undegraded":"2024-03-04T15:50:35.213076+0000","last_fullsized":"2024-03-04T15:50:35.213076+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:04:51.005645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124095+0000","last_change":"2024-03-04T15:49:33.731519+0000","last_active":"2024-03-04T15:50:36.124095+0000","last_peered":"2024-03-04T15:50:36.124095+0000","last_clean":"2024-03-04T15:50:36.124095+0000","last_became_active":"2024-03-04T15:49:33.731099+0000","last_became_peered":"2024-03-04T15:49:33.731099+0000","last_unstale":"2024-03-04T15:50:36.124095+0000","last_undegraded":"2024-03-04T15:50:36.124095+0000","last_fullsized":"2024-03-04T15:50:36.124095+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:21:46.553452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"68'11","reported_seq":55,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124592+0000","last_change":"2024-03-04T15:49:35.762217+0000","last_active":"2024-03-04T15:50:36.124592+0000","last_peered":"2024-03-04T15:50:36.124592+0000","last_clean":"2024-03-04T15:50:36.124592+0000","last_became_active":"2024-03-04T15:49:35.761361+0000","last_became_peered":"2024-03-04T15:49:35.761361+0000","last_unstale":"2024-03-04T15:50:36.124592+0000","last_undegraded":"2024-03-04T15:50:36.124592+0000","last_fullsized":"2024-03-04T15:50:36.124592+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:53:35.491762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.211902+0000","last_change":"2024-03-04T15:49:40.242353+0000","last_active":"2024-03-04T15:50:35.211902+0000","last_peered":"2024-03-04T15:50:35.211902+0000","last_clean":"2024-03-04T15:50:35.211902+0000","last_became_active":"2024-03-04T15:49:40.239419+0000","last_became_peered":"2024-03-04T15:49:40.239419+0000","last_unstale":"2024-03-04T15:50:35.211902+0000","last_undegraded":"2024-03-04T15:50:35.211902+0000","last_fullsized":"2024-03-04T15:50:35.211902+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:13:28.962703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"61'2","reported_seq":55,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.213243+0000","last_change":"2024-03-04T15:49:33.743679+0000","last_active":"2024-03-04T15:50:35.213243+0000","last_peered":"2024-03-04T15:50:35.213243+0000","last_clean":"2024-03-04T15:50:35.213243+0000","last_became_active":"2024-03-04T15:49:33.742860+0000","last_became_peered":"2024-03-04T15:49:33.742860+0000","last_unstale":"2024-03-04T15:50:35.213243+0000","last_undegraded":"2024-03-04T15:50:35.213243+0000","last_fullsized":"2024-03-04T15:50:35.213243+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:36:01.739238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"68'11","reported_seq":55,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124793+0000","last_change":"2024-03-04T15:49:35.760228+0000","last_active":"2024-03-04T15:50:36.124793+0000","last_peered":"2024-03-04T15:50:36.124793+0000","last_clean":"2024-03-04T15:50:36.124793+0000","last_became_active":"2024-03-04T15:49:35.759385+0000","last_became_peered":"2024-03-04T15:49:35.759385+0000","last_unstale":"2024-03-04T15:50:36.124793+0000","last_undegraded":"2024-03-04T15:50:36.124793+0000","last_fullsized":"2024-03-04T15:50:36.124793+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:25:25.525878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"68'11","reported_seq":76,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:18.952807+0000","last_change":"2024-03-04T15:49:38.008342+0000","last_active":"2024-03-04T15:52:18.952807+0000","last_peered":"2024-03-04T15:52:18.952807+0000","last_clean":"2024-03-04T15:52:18.952807+0000","last_became_active":"2024-03-04T15:49:38.008072+0000","last_became_peered":"2024-03-04T15:49:38.008072+0000","last_unstale":"2024-03-04T15:52:18.952807+0000","last_undegraded":"2024-03-04T15:52:18.952807+0000","last_fullsized":"2024-03-04T15:52:18.952807+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:08:15.812918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.209595+0000","last_change":"2024-03-04T15:49:40.234479+0000","last_active":"2024-03-04T15:50:35.209595+0000","last_peered":"2024-03-04T15:50:35.209595+0000","last_clean":"2024-03-04T15:50:35.209595+0000","last_became_active":"2024-03-04T15:49:40.233708+0000","last_became_peered":"2024-03-04T15:49:40.233708+0000","last_unstale":"2024-03-04T15:50:35.209595+0000","last_undegraded":"2024-03-04T15:50:35.209595+0000","last_fullsized":"2024-03-04T15:50:35.209595+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:57:42.295900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.125409+0000","last_change":"2024-03-04T15:49:33.742626+0000","last_active":"2024-03-04T15:50:36.125409+0000","last_peered":"2024-03-04T15:50:36.125409+0000","last_clean":"2024-03-04T15:50:36.125409+0000","last_became_active":"2024-03-04T15:49:33.742298+0000","last_became_peered":"2024-03-04T15:49:33.742298+0000","last_unstale":"2024-03-04T15:50:36.125409+0000","last_undegraded":"2024-03-04T15:50:36.125409+0000","last_fullsized":"2024-03-04T15:50:36.125409+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:21:39.371802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"68'16","reported_seq":93,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:18.952649+0000","last_change":"2024-03-04T15:49:36.018752+0000","last_active":"2024-03-04T15:52:18.952649+0000","last_peered":"2024-03-04T15:52:18.952649+0000","last_clean":"2024-03-04T15:52:18.952649+0000","last_became_active":"2024-03-04T15:49:36.017875+0000","last_became_peered":"2024-03-04T15:49:36.017875+0000","last_unstale":"2024-03-04T15:52:18.952649+0000","last_undegraded":"2024-03-04T15:52:18.952649+0000","last_fullsized":"2024-03-04T15:52:18.952649+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:03:37.662284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124033+0000","last_change":"2024-03-04T15:50:24.435649+0000","last_active":"2024-03-04T15:50:36.124033+0000","last_peered":"2024-03-04T15:50:36.124033+0000","last_clean":"2024-03-04T15:50:36.124033+0000","last_became_active":"2024-03-04T15:50:24.434494+0000","last_became_peered":"2024-03-04T15:50:24.434494+0000","last_unstale":"2024-03-04T15:50:36.124033+0000","last_undegraded":"2024-03-04T15:50:36.124033+0000","last_fullsized":"2024-03-04T15:50:36.124033+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:37:36.962551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212520+0000","last_change":"2024-03-04T15:49:40.240101+0000","last_active":"2024-03-04T15:50:35.212520+0000","last_peered":"2024-03-04T15:50:35.212520+0000","last_clean":"2024-03-04T15:50:35.212520+0000","last_became_active":"2024-03-04T15:49:40.238274+0000","last_became_peered":"2024-03-04T15:49:40.238274+0000","last_unstale":"2024-03-04T15:50:35.212520+0000","last_undegraded":"2024-03-04T15:50:35.212520+0000","last_fullsized":"2024-03-04T15:50:35.212520+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T02:29:38.047202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123810+0000","last_change":"2024-03-04T15:49:33.736608+0000","last_active":"2024-03-04T15:50:36.123810+0000","last_peered":"2024-03-04T15:50:36.123810+0000","last_clean":"2024-03-04T15:50:36.123810+0000","last_became_active":"2024-03-04T15:49:33.735769+0000","last_became_peered":"2024-03-04T15:49:33.735769+0000","last_unstale":"2024-03-04T15:50:36.123810+0000","last_undegraded":"2024-03-04T15:50:36.123810+0000","last_fullsized":"2024-03-04T15:50:36.123810+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T02:22:24.567798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"68'19","reported_seq":71,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.208039+0000","last_change":"2024-03-04T15:49:35.745883+0000","last_active":"2024-03-04T15:50:35.208039+0000","last_peered":"2024-03-04T15:50:35.208039+0000","last_clean":"2024-03-04T15:50:35.208039+0000","last_became_active":"2024-03-04T15:49:35.744948+0000","last_became_peered":"2024-03-04T15:49:35.744948+0000","last_unstale":"2024-03-04T15:50:35.208039+0000","last_undegraded":"2024-03-04T15:50:35.208039+0000","last_fullsized":"2024-03-04T15:50:35.208039+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:30:22.582548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.125318+0000","last_change":"2024-03-04T15:49:38.021820+0000","last_active":"2024-03-04T15:50:36.125318+0000","last_peered":"2024-03-04T15:50:36.125318+0000","last_clean":"2024-03-04T15:50:36.125318+0000","last_became_active":"2024-03-04T15:49:38.020920+0000","last_became_peered":"2024-03-04T15:49:38.020920+0000","last_unstale":"2024-03-04T15:50:36.125318+0000","last_undegraded":"2024-03-04T15:50:36.125318+0000","last_fullsized":"2024-03-04T15:50:36.125318+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:56:56.529484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124306+0000","last_change":"2024-03-04T15:49:40.450577+0000","last_active":"2024-03-04T15:50:36.124306+0000","last_peered":"2024-03-04T15:50:36.124306+0000","last_clean":"2024-03-04T15:50:36.124306+0000","last_became_active":"2024-03-04T15:49:40.447504+0000","last_became_peered":"2024-03-04T15:49:40.447504+0000","last_unstale":"2024-03-04T15:50:36.124306+0000","last_undegraded":"2024-03-04T15:50:36.124306+0000","last_fullsized":"2024-03-04T15:50:36.124306+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:12:58.834631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"68'18","reported_seq":68,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124384+0000","last_change":"2024-03-04T15:49:36.017775+0000","last_active":"2024-03-04T15:50:36.124384+0000","last_peered":"2024-03-04T15:50:36.124384+0000","last_clean":"2024-03-04T15:50:36.124384+0000","last_became_active":"2024-03-04T15:49:36.017336+0000","last_became_peered":"2024-03-04T15:49:36.017336+0000","last_unstale":"2024-03-04T15:50:36.124384+0000","last_undegraded":"2024-03-04T15:50:36.124384+0000","last_fullsized":"2024-03-04T15:50:36.124384+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:45:15.781821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124205+0000","last_change":"2024-03-04T15:49:33.722318+0000","last_active":"2024-03-04T15:50:36.124205+0000","last_peered":"2024-03-04T15:50:36.124205+0000","last_clean":"2024-03-04T15:50:36.124205+0000","last_became_active":"2024-03-04T15:49:33.721824+0000","last_became_peered":"2024-03-04T15:49:33.721824+0000","last_unstale":"2024-03-04T15:50:36.124205+0000","last_undegraded":"2024-03-04T15:50:36.124205+0000","last_fullsized":"2024-03-04T15:50:36.124205+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:48:53.316898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124634+0000","last_change":"2024-03-04T15:49:38.024580+0000","last_active":"2024-03-04T15:50:36.124634+0000","last_peered":"2024-03-04T15:50:36.124634+0000","last_clean":"2024-03-04T15:50:36.124634+0000","last_became_active":"2024-03-04T15:49:38.023731+0000","last_became_peered":"2024-03-04T15:49:38.023731+0000","last_unstale":"2024-03-04T15:50:36.124634+0000","last_undegraded":"2024-03-04T15:50:36.124634+0000","last_fullsized":"2024-03-04T15:50:36.124634+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:49:24.817743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123990+0000","last_change":"2024-03-04T15:49:40.237131+0000","last_active":"2024-03-04T15:50:36.123990+0000","last_peered":"2024-03-04T15:50:36.123990+0000","last_clean":"2024-03-04T15:50:36.123990+0000","last_became_active":"2024-03-04T15:49:40.236283+0000","last_became_peered":"2024-03-04T15:49:40.236283+0000","last_unstale":"2024-03-04T15:50:36.123990+0000","last_undegraded":"2024-03-04T15:50:36.123990+0000","last_fullsized":"2024-03-04T15:50:36.123990+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:42:48.192315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"68'10","reported_seq":50,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.211102+0000","last_change":"2024-03-04T15:49:35.758734+0000","last_active":"2024-03-04T15:50:35.211102+0000","last_peered":"2024-03-04T15:50:35.211102+0000","last_clean":"2024-03-04T15:50:35.211102+0000","last_became_active":"2024-03-04T15:49:35.758030+0000","last_became_peered":"2024-03-04T15:49:35.758030+0000","last_unstale":"2024-03-04T15:50:35.211102+0000","last_undegraded":"2024-03-04T15:50:35.211102+0000","last_fullsized":"2024-03-04T15:50:35.211102+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:30:01.936137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123399+0000","last_change":"2024-03-04T15:49:33.735443+0000","last_active":"2024-03-04T15:50:36.123399+0000","last_peered":"2024-03-04T15:50:36.123399+0000","last_clean":"2024-03-04T15:50:36.123399+0000","last_became_active":"2024-03-04T15:49:33.734226+0000","last_became_peered":"2024-03-04T15:49:33.734226+0000","last_unstale":"2024-03-04T15:50:36.123399+0000","last_undegraded":"2024-03-04T15:50:36.123399+0000","last_fullsized":"2024-03-04T15:50:36.123399+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:45:36.310086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":283,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:37.377774+0000","last_change":"2024-03-04T15:48:57.791739+0000","last_active":"2024-03-04T15:50:37.377774+0000","last_peered":"2024-03-04T15:50:37.377774+0000","last_clean":"2024-03-04T15:50:37.377774+0000","last_became_active":"2024-03-04T15:48:57.790648+0000","last_became_peered":"2024-03-04T15:48:57.790648+0000","last_unstale":"2024-03-04T15:50:37.377774+0000","last_undegraded":"2024-03-04T15:50:37.377774+0000","last_fullsized":"2024-03-04T15:50:37.377774+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:44:31.731812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:44:31.731812+0000","last_clean_scrub_stamp":"2024-03-04T15:44:31.731812+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:47:26.809771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.352653+0000","last_change":"2024-03-04T15:49:38.153971+0000","last_active":"2024-03-04T15:50:35.352653+0000","last_peered":"2024-03-04T15:50:35.352653+0000","last_clean":"2024-03-04T15:50:35.352653+0000","last_became_active":"2024-03-04T15:49:38.153129+0000","last_became_peered":"2024-03-04T15:49:38.153129+0000","last_unstale":"2024-03-04T15:50:35.352653+0000","last_undegraded":"2024-03-04T15:50:35.352653+0000","last_fullsized":"2024-03-04T15:50:35.352653+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:39:29.323685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.125226+0000","last_change":"2024-03-04T15:49:40.236245+0000","last_active":"2024-03-04T15:50:36.125226+0000","last_peered":"2024-03-04T15:50:36.125226+0000","last_clean":"2024-03-04T15:50:36.125226+0000","last_became_active":"2024-03-04T15:49:40.230772+0000","last_became_peered":"2024-03-04T15:49:40.230772+0000","last_unstale":"2024-03-04T15:50:36.125226+0000","last_undegraded":"2024-03-04T15:50:36.125226+0000","last_fullsized":"2024-03-04T15:50:36.125226+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:19:57.650635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"68'14","reported_seq":57,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.210501+0000","last_change":"2024-03-04T15:49:35.742578+0000","last_active":"2024-03-04T15:50:35.210501+0000","last_peered":"2024-03-04T15:50:35.210501+0000","last_clean":"2024-03-04T15:50:35.210501+0000","last_became_active":"2024-03-04T15:49:35.741577+0000","last_became_peered":"2024-03-04T15:49:35.741577+0000","last_unstale":"2024-03-04T15:50:35.210501+0000","last_undegraded":"2024-03-04T15:50:35.210501+0000","last_fullsized":"2024-03-04T15:50:35.210501+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:46:40.777731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"61'1","reported_seq":48,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123358+0000","last_change":"2024-03-04T15:49:33.745052+0000","last_active":"2024-03-04T15:50:36.123358+0000","last_peered":"2024-03-04T15:50:36.123358+0000","last_clean":"2024-03-04T15:50:36.123358+0000","last_became_active":"2024-03-04T15:49:33.744105+0000","last_became_peered":"2024-03-04T15:49:33.744105+0000","last_unstale":"2024-03-04T15:50:36.123358+0000","last_undegraded":"2024-03-04T15:50:36.123358+0000","last_fullsized":"2024-03-04T15:50:36.123358+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:45:27.817273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.213592+0000","last_change":"2024-03-04T15:49:38.024959+0000","last_active":"2024-03-04T15:50:35.213592+0000","last_peered":"2024-03-04T15:50:35.213592+0000","last_clean":"2024-03-04T15:50:35.213592+0000","last_became_active":"2024-03-04T15:49:38.024138+0000","last_became_peered":"2024-03-04T15:49:38.024138+0000","last_unstale":"2024-03-04T15:50:35.213592+0000","last_undegraded":"2024-03-04T15:50:35.213592+0000","last_fullsized":"2024-03-04T15:50:35.213592+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:23:31.269691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124930+0000","last_change":"2024-03-04T15:49:40.448899+0000","last_active":"2024-03-04T15:50:36.124930+0000","last_peered":"2024-03-04T15:50:36.124930+0000","last_clean":"2024-03-04T15:50:36.124930+0000","last_became_active":"2024-03-04T15:49:40.447962+0000","last_became_peered":"2024-03-04T15:49:40.447962+0000","last_unstale":"2024-03-04T15:50:36.124930+0000","last_undegraded":"2024-03-04T15:50:36.124930+0000","last_fullsized":"2024-03-04T15:50:36.124930+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:05:43.644548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"71'2","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.352325+0000","last_change":"2024-03-04T15:49:48.250908+0000","last_active":"2024-03-04T15:50:35.352325+0000","last_peered":"2024-03-04T15:50:35.352325+0000","last_clean":"2024-03-04T15:50:35.352325+0000","last_became_active":"2024-03-04T15:49:38.000176+0000","last_became_peered":"2024-03-04T15:49:38.000176+0000","last_unstale":"2024-03-04T15:50:35.352325+0000","last_undegraded":"2024-03-04T15:50:35.352325+0000","last_fullsized":"2024-03-04T15:50:35.352325+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:09:44.006918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002567518,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124441+0000","last_change":"2024-03-04T15:49:33.750901+0000","last_active":"2024-03-04T15:50:36.124441+0000","last_peered":"2024-03-04T15:50:36.124441+0000","last_clean":"2024-03-04T15:50:36.124441+0000","last_became_active":"2024-03-04T15:49:33.750070+0000","last_became_peered":"2024-03-04T15:49:33.750070+0000","last_unstale":"2024-03-04T15:50:36.124441+0000","last_undegraded":"2024-03-04T15:50:36.124441+0000","last_fullsized":"2024-03-04T15:50:36.124441+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:43:06.430464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"68'30","reported_seq":121,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:18.954296+0000","last_change":"2024-03-04T15:49:36.019151+0000","last_active":"2024-03-04T15:52:18.954296+0000","last_peered":"2024-03-04T15:52:18.954296+0000","last_clean":"2024-03-04T15:52:18.954296+0000","last_became_active":"2024-03-04T15:49:36.017482+0000","last_became_peered":"2024-03-04T15:49:36.017482+0000","last_unstale":"2024-03-04T15:52:18.954296+0000","last_undegraded":"2024-03-04T15:52:18.954296+0000","last_fullsized":"2024-03-04T15:52:18.954296+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:17:04.424822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"68'11","reported_seq":77,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:18.954324+0000","last_change":"2024-03-04T15:49:38.154171+0000","last_active":"2024-03-04T15:52:18.954324+0000","last_peered":"2024-03-04T15:52:18.954324+0000","last_clean":"2024-03-04T15:52:18.954324+0000","last_became_active":"2024-03-04T15:49:38.153230+0000","last_became_peered":"2024-03-04T15:49:38.153230+0000","last_unstale":"2024-03-04T15:52:18.954324+0000","last_undegraded":"2024-03-04T15:52:18.954324+0000","last_fullsized":"2024-03-04T15:52:18.954324+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:04:34.338590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.207110+0000","last_change":"2024-03-04T15:49:40.238777+0000","last_active":"2024-03-04T15:50:35.207110+0000","last_peered":"2024-03-04T15:50:35.207110+0000","last_clean":"2024-03-04T15:50:35.207110+0000","last_became_active":"2024-03-04T15:49:40.238132+0000","last_became_peered":"2024-03-04T15:49:40.238132+0000","last_unstale":"2024-03-04T15:50:35.207110+0000","last_undegraded":"2024-03-04T15:50:35.207110+0000","last_fullsized":"2024-03-04T15:50:35.207110+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:17:07.947883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.206190+0000","last_change":"2024-03-04T15:49:48.245050+0000","last_active":"2024-03-04T15:50:35.206190+0000","last_peered":"2024-03-04T15:50:35.206190+0000","last_clean":"2024-03-04T15:50:35.206190+0000","last_became_active":"2024-03-04T15:49:38.152238+0000","last_became_peered":"2024-03-04T15:49:38.152238+0000","last_unstale":"2024-03-04T15:50:35.206190+0000","last_undegraded":"2024-03-04T15:50:35.206190+0000","last_fullsized":"2024-03-04T15:50:35.206190+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:43:48.329784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052544289999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"61'1","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124690+0000","last_change":"2024-03-04T15:49:33.732491+0000","last_active":"2024-03-04T15:50:36.124690+0000","last_peered":"2024-03-04T15:50:36.124690+0000","last_clean":"2024-03-04T15:50:36.124690+0000","last_became_active":"2024-03-04T15:49:33.731911+0000","last_became_peered":"2024-03-04T15:49:33.731911+0000","last_unstale":"2024-03-04T15:50:36.124690+0000","last_undegraded":"2024-03-04T15:50:36.124690+0000","last_fullsized":"2024-03-04T15:50:36.124690+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T03:41:49.855451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"68'13","reported_seq":62,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.210907+0000","last_change":"2024-03-04T15:49:35.761285+0000","last_active":"2024-03-04T15:50:35.210907+0000","last_peered":"2024-03-04T15:50:35.210907+0000","last_clean":"2024-03-04T15:50:35.210907+0000","last_became_active":"2024-03-04T15:49:35.760870+0000","last_became_peered":"2024-03-04T15:49:35.760870+0000","last_unstale":"2024-03-04T15:50:35.210907+0000","last_undegraded":"2024-03-04T15:50:35.210907+0000","last_fullsized":"2024-03-04T15:50:35.210907+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:10:46.714373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"68'11","reported_seq":76,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:18.954399+0000","last_change":"2024-03-04T15:49:38.154931+0000","last_active":"2024-03-04T15:52:18.954399+0000","last_peered":"2024-03-04T15:52:18.954399+0000","last_clean":"2024-03-04T15:52:18.954399+0000","last_became_active":"2024-03-04T15:49:38.153768+0000","last_became_peered":"2024-03-04T15:49:38.153768+0000","last_unstale":"2024-03-04T15:52:18.954399+0000","last_undegraded":"2024-03-04T15:52:18.954399+0000","last_fullsized":"2024-03-04T15:52:18.954399+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T03:13:19.860639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124117+0000","last_change":"2024-03-04T15:49:40.457188+0000","last_active":"2024-03-04T15:50:36.124117+0000","last_peered":"2024-03-04T15:50:36.124117+0000","last_clean":"2024-03-04T15:50:36.124117+0000","last_became_active":"2024-03-04T15:49:40.455923+0000","last_became_peered":"2024-03-04T15:49:40.455923+0000","last_unstale":"2024-03-04T15:50:36.124117+0000","last_undegraded":"2024-03-04T15:50:36.124117+0000","last_fullsized":"2024-03-04T15:50:36.124117+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:50:49.499327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"72'5","reported_seq":189,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:25.066132+0000","last_change":"2024-03-04T15:49:49.018527+0000","last_active":"2024-03-04T15:52:25.066132+0000","last_peered":"2024-03-04T15:52:25.066132+0000","last_clean":"2024-03-04T15:52:25.066132+0000","last_became_active":"2024-03-04T15:49:38.025846+0000","last_became_peered":"2024-03-04T15:49:38.025846+0000","last_unstale":"2024-03-04T15:52:25.066132+0000","last_undegraded":"2024-03-04T15:52:25.066132+0000","last_fullsized":"2024-03-04T15:52:25.066132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T03:12:49.580578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097583999999999995,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":149,"num_read_kb":144,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.352956+0000","last_change":"2024-03-04T15:49:33.743893+0000","last_active":"2024-03-04T15:50:35.352956+0000","last_peered":"2024-03-04T15:50:35.352956+0000","last_clean":"2024-03-04T15:50:35.352956+0000","last_became_active":"2024-03-04T15:49:33.743521+0000","last_became_peered":"2024-03-04T15:49:33.743521+0000","last_unstale":"2024-03-04T15:50:35.352956+0000","last_undegraded":"2024-03-04T15:50:35.352956+0000","last_fullsized":"2024-03-04T15:50:35.352956+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T02:53:56.520019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"68'12","reported_seq":54,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.214467+0000","last_change":"2024-03-04T15:49:35.742623+0000","last_active":"2024-03-04T15:50:35.214467+0000","last_peered":"2024-03-04T15:50:35.214467+0000","last_clean":"2024-03-04T15:50:35.214467+0000","last_became_active":"2024-03-04T15:49:35.741672+0000","last_became_peered":"2024-03-04T15:49:35.741672+0000","last_unstale":"2024-03-04T15:50:35.214467+0000","last_undegraded":"2024-03-04T15:50:35.214467+0000","last_fullsized":"2024-03-04T15:50:35.214467+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:14:44.466768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.206612+0000","last_change":"2024-03-04T15:49:37.997336+0000","last_active":"2024-03-04T15:50:35.206612+0000","last_peered":"2024-03-04T15:50:35.206612+0000","last_clean":"2024-03-04T15:50:35.206612+0000","last_became_active":"2024-03-04T15:49:37.996986+0000","last_became_peered":"2024-03-04T15:49:37.996986+0000","last_unstale":"2024-03-04T15:50:35.206612+0000","last_undegraded":"2024-03-04T15:50:35.206612+0000","last_fullsized":"2024-03-04T15:50:35.206612+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T03:20:51.222665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.211126+0000","last_change":"2024-03-04T15:49:40.449478+0000","last_active":"2024-03-04T15:50:35.211126+0000","last_peered":"2024-03-04T15:50:35.211126+0000","last_clean":"2024-03-04T15:50:35.211126+0000","last_became_active":"2024-03-04T15:49:40.448321+0000","last_became_peered":"2024-03-04T15:49:40.448321+0000","last_unstale":"2024-03-04T15:50:35.211126+0000","last_undegraded":"2024-03-04T15:50:35.211126+0000","last_fullsized":"2024-03-04T15:50:35.211126+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T03:00:26.668940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"68'11","reported_seq":79,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:18.952662+0000","last_change":"2024-03-04T15:49:38.021034+0000","last_active":"2024-03-04T15:52:18.952662+0000","last_peered":"2024-03-04T15:52:18.952662+0000","last_clean":"2024-03-04T15:52:18.952662+0000","last_became_active":"2024-03-04T15:49:38.020598+0000","last_became_peered":"2024-03-04T15:49:38.020598+0000","last_unstale":"2024-03-04T15:52:18.952662+0000","last_undegraded":"2024-03-04T15:52:18.952662+0000","last_fullsized":"2024-03-04T15:52:18.952662+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:08:38.560356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123561+0000","last_change":"2024-03-04T15:49:33.745414+0000","last_active":"2024-03-04T15:50:36.123561+0000","last_peered":"2024-03-04T15:50:36.123561+0000","last_clean":"2024-03-04T15:50:36.123561+0000","last_became_active":"2024-03-04T15:49:33.744726+0000","last_became_peered":"2024-03-04T15:49:33.744726+0000","last_unstale":"2024-03-04T15:50:36.123561+0000","last_undegraded":"2024-03-04T15:50:36.123561+0000","last_fullsized":"2024-03-04T15:50:36.123561+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:07:14.994749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"68'12","reported_seq":58,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.206460+0000","last_change":"2024-03-04T15:49:36.018293+0000","last_active":"2024-03-04T15:50:35.206460+0000","last_peered":"2024-03-04T15:50:35.206460+0000","last_clean":"2024-03-04T15:50:35.206460+0000","last_became_active":"2024-03-04T15:49:36.017401+0000","last_became_peered":"2024-03-04T15:49:36.017401+0000","last_unstale":"2024-03-04T15:50:35.206460+0000","last_undegraded":"2024-03-04T15:50:35.206460+0000","last_fullsized":"2024-03-04T15:50:35.206460+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:31:35.332038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124320+0000","last_change":"2024-03-04T15:49:40.223056+0000","last_active":"2024-03-04T15:50:36.124320+0000","last_peered":"2024-03-04T15:50:36.124320+0000","last_clean":"2024-03-04T15:50:36.124320+0000","last_became_active":"2024-03-04T15:49:40.222141+0000","last_became_peered":"2024-03-04T15:49:40.222141+0000","last_unstale":"2024-03-04T15:50:36.124320+0000","last_undegraded":"2024-03-04T15:50:36.124320+0000","last_fullsized":"2024-03-04T15:50:36.124320+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:50:29.163434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124722+0000","last_change":"2024-03-04T15:49:38.023066+0000","last_active":"2024-03-04T15:50:36.124722+0000","last_peered":"2024-03-04T15:50:36.124722+0000","last_clean":"2024-03-04T15:50:36.124722+0000","last_became_active":"2024-03-04T15:49:38.022328+0000","last_became_peered":"2024-03-04T15:49:38.022328+0000","last_unstale":"2024-03-04T15:50:36.124722+0000","last_undegraded":"2024-03-04T15:50:36.124722+0000","last_fullsized":"2024-03-04T15:50:36.124722+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:39:42.087976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.125014+0000","last_change":"2024-03-04T15:49:33.731253+0000","last_active":"2024-03-04T15:50:36.125014+0000","last_peered":"2024-03-04T15:50:36.125014+0000","last_clean":"2024-03-04T15:50:36.125014+0000","last_became_active":"2024-03-04T15:49:33.730734+0000","last_became_peered":"2024-03-04T15:49:33.730734+0000","last_unstale":"2024-03-04T15:50:36.125014+0000","last_undegraded":"2024-03-04T15:50:36.125014+0000","last_fullsized":"2024-03-04T15:50:36.125014+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:13:57.535530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"68'15","reported_seq":60,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212490+0000","last_change":"2024-03-04T15:49:35.749494+0000","last_active":"2024-03-04T15:50:35.212490+0000","last_peered":"2024-03-04T15:50:35.212490+0000","last_clean":"2024-03-04T15:50:35.212490+0000","last_became_active":"2024-03-04T15:49:35.748825+0000","last_became_peered":"2024-03-04T15:49:35.748825+0000","last_unstale":"2024-03-04T15:50:35.212490+0000","last_undegraded":"2024-03-04T15:50:35.212490+0000","last_fullsized":"2024-03-04T15:50:35.212490+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:28:12.248198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212748+0000","last_change":"2024-03-04T15:49:40.234279+0000","last_active":"2024-03-04T15:50:35.212748+0000","last_peered":"2024-03-04T15:50:35.212748+0000","last_clean":"2024-03-04T15:50:35.212748+0000","last_became_active":"2024-03-04T15:49:40.233338+0000","last_became_peered":"2024-03-04T15:49:40.233338+0000","last_unstale":"2024-03-04T15:50:35.212748+0000","last_undegraded":"2024-03-04T15:50:35.212748+0000","last_fullsized":"2024-03-04T15:50:35.212748+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:31:56.525426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.125265+0000","last_change":"2024-03-04T15:49:33.731659+0000","last_active":"2024-03-04T15:50:36.125265+0000","last_peered":"2024-03-04T15:50:36.125265+0000","last_clean":"2024-03-04T15:50:36.125265+0000","last_became_active":"2024-03-04T15:49:33.730899+0000","last_became_peered":"2024-03-04T15:49:33.730899+0000","last_unstale":"2024-03-04T15:50:36.125265+0000","last_undegraded":"2024-03-04T15:50:36.125265+0000","last_fullsized":"2024-03-04T15:50:36.125265+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T02:46:34.770313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"68'9","reported_seq":51,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.210886+0000","last_change":"2024-03-04T15:49:35.746336+0000","last_active":"2024-03-04T15:50:35.210886+0000","last_peered":"2024-03-04T15:50:35.210886+0000","last_clean":"2024-03-04T15:50:35.210886+0000","last_became_active":"2024-03-04T15:49:35.745920+0000","last_became_peered":"2024-03-04T15:49:35.745920+0000","last_unstale":"2024-03-04T15:50:35.210886+0000","last_undegraded":"2024-03-04T15:50:35.210886+0000","last_fullsized":"2024-03-04T15:50:35.210886+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:12:58.978436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.211160+0000","last_change":"2024-03-04T15:49:38.005369+0000","last_active":"2024-03-04T15:50:35.211160+0000","last_peered":"2024-03-04T15:50:35.211160+0000","last_clean":"2024-03-04T15:50:35.211160+0000","last_became_active":"2024-03-04T15:49:38.005061+0000","last_became_peered":"2024-03-04T15:49:38.005061+0000","last_unstale":"2024-03-04T15:50:35.211160+0000","last_undegraded":"2024-03-04T15:50:35.211160+0000","last_fullsized":"2024-03-04T15:50:35.211160+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:34:22.171049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.125310+0000","last_change":"2024-03-04T15:49:40.245338+0000","last_active":"2024-03-04T15:50:36.125310+0000","last_peered":"2024-03-04T15:50:36.125310+0000","last_clean":"2024-03-04T15:50:36.125310+0000","last_became_active":"2024-03-04T15:49:40.244547+0000","last_became_peered":"2024-03-04T15:49:40.244547+0000","last_unstale":"2024-03-04T15:50:36.125310+0000","last_undegraded":"2024-03-04T15:50:36.125310+0000","last_fullsized":"2024-03-04T15:50:36.125310+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:04:33.291376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"68'11","reported_seq":76,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:18.952767+0000","last_change":"2024-03-04T15:49:38.021279+0000","last_active":"2024-03-04T15:52:18.952767+0000","last_peered":"2024-03-04T15:52:18.952767+0000","last_clean":"2024-03-04T15:52:18.952767+0000","last_became_active":"2024-03-04T15:49:38.020844+0000","last_became_peered":"2024-03-04T15:49:38.020844+0000","last_unstale":"2024-03-04T15:52:18.952767+0000","last_undegraded":"2024-03-04T15:52:18.952767+0000","last_fullsized":"2024-03-04T15:52:18.952767+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:02:30.542297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124480+0000","last_change":"2024-03-04T15:49:33.748854+0000","last_active":"2024-03-04T15:50:36.124480+0000","last_peered":"2024-03-04T15:50:36.124480+0000","last_clean":"2024-03-04T15:50:36.124480+0000","last_became_active":"2024-03-04T15:49:33.748356+0000","last_became_peered":"2024-03-04T15:49:33.748356+0000","last_unstale":"2024-03-04T15:50:36.124480+0000","last_undegraded":"2024-03-04T15:50:36.124480+0000","last_fullsized":"2024-03-04T15:50:36.124480+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:55:49.935117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"68'19","reported_seq":67,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.352494+0000","last_change":"2024-03-04T15:49:35.753931+0000","last_active":"2024-03-04T15:50:35.352494+0000","last_peered":"2024-03-04T15:50:35.352494+0000","last_clean":"2024-03-04T15:50:35.352494+0000","last_became_active":"2024-03-04T15:49:35.752842+0000","last_became_peered":"2024-03-04T15:49:35.752842+0000","last_unstale":"2024-03-04T15:50:35.352494+0000","last_undegraded":"2024-03-04T15:50:35.352494+0000","last_fullsized":"2024-03-04T15:50:35.352494+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:25:43.530434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123180+0000","last_change":"2024-03-04T15:49:40.225692+0000","last_active":"2024-03-04T15:50:36.123180+0000","last_peered":"2024-03-04T15:50:36.123180+0000","last_clean":"2024-03-04T15:50:36.123180+0000","last_became_active":"2024-03-04T15:49:40.224006+0000","last_became_peered":"2024-03-04T15:49:40.224006+0000","last_unstale":"2024-03-04T15:50:36.123180+0000","last_undegraded":"2024-03-04T15:50:36.123180+0000","last_fullsized":"2024-03-04T15:50:36.123180+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:55:44.359786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124667+0000","last_change":"2024-03-04T15:49:33.725281+0000","last_active":"2024-03-04T15:50:36.124667+0000","last_peered":"2024-03-04T15:50:36.124667+0000","last_clean":"2024-03-04T15:50:36.124667+0000","last_became_active":"2024-03-04T15:49:33.724481+0000","last_became_peered":"2024-03-04T15:49:33.724481+0000","last_unstale":"2024-03-04T15:50:36.124667+0000","last_undegraded":"2024-03-04T15:50:36.124667+0000","last_fullsized":"2024-03-04T15:50:36.124667+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:33:47.773620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"68'17","reported_seq":64,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124671+0000","last_change":"2024-03-04T15:49:35.764116+0000","last_active":"2024-03-04T15:50:36.124671+0000","last_peered":"2024-03-04T15:50:36.124671+0000","last_clean":"2024-03-04T15:50:36.124671+0000","last_became_active":"2024-03-04T15:49:35.763229+0000","last_became_peered":"2024-03-04T15:49:35.763229+0000","last_unstale":"2024-03-04T15:50:36.124671+0000","last_undegraded":"2024-03-04T15:50:36.124671+0000","last_fullsized":"2024-03-04T15:50:36.124671+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:06:55.820085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.352264+0000","last_change":"2024-03-04T15:49:38.148607+0000","last_active":"2024-03-04T15:50:35.352264+0000","last_peered":"2024-03-04T15:50:35.352264+0000","last_clean":"2024-03-04T15:50:35.352264+0000","last_became_active":"2024-03-04T15:49:38.147960+0000","last_became_peered":"2024-03-04T15:49:38.147960+0000","last_unstale":"2024-03-04T15:50:35.352264+0000","last_undegraded":"2024-03-04T15:50:35.352264+0000","last_fullsized":"2024-03-04T15:50:35.352264+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:12:26.050916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.209699+0000","last_change":"2024-03-04T15:49:40.220474+0000","last_active":"2024-03-04T15:50:35.209699+0000","last_peered":"2024-03-04T15:50:35.209699+0000","last_clean":"2024-03-04T15:50:35.209699+0000","last_became_active":"2024-03-04T15:49:40.219620+0000","last_became_peered":"2024-03-04T15:49:40.219620+0000","last_unstale":"2024-03-04T15:50:35.209699+0000","last_undegraded":"2024-03-04T15:50:35.209699+0000","last_fullsized":"2024-03-04T15:50:35.209699+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:05:26.162699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124752+0000","last_change":"2024-03-04T15:49:33.736207+0000","last_active":"2024-03-04T15:50:36.124752+0000","last_peered":"2024-03-04T15:50:36.124752+0000","last_clean":"2024-03-04T15:50:36.124752+0000","last_became_active":"2024-03-04T15:49:33.735658+0000","last_became_peered":"2024-03-04T15:49:33.735658+0000","last_unstale":"2024-03-04T15:50:36.124752+0000","last_undegraded":"2024-03-04T15:50:36.124752+0000","last_fullsized":"2024-03-04T15:50:36.124752+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:39:47.411302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"68'10","reported_seq":51,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123591+0000","last_change":"2024-03-04T15:49:35.769383+0000","last_active":"2024-03-04T15:50:36.123591+0000","last_peered":"2024-03-04T15:50:36.123591+0000","last_clean":"2024-03-04T15:50:36.123591+0000","last_became_active":"2024-03-04T15:49:35.768373+0000","last_became_peered":"2024-03-04T15:49:35.768373+0000","last_unstale":"2024-03-04T15:50:36.123591+0000","last_undegraded":"2024-03-04T15:50:36.123591+0000","last_fullsized":"2024-03-04T15:50:36.123591+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T03:07:08.632363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212212+0000","last_change":"2024-03-04T15:49:38.029335+0000","last_active":"2024-03-04T15:50:35.212212+0000","last_peered":"2024-03-04T15:50:35.212212+0000","last_clean":"2024-03-04T15:50:35.212212+0000","last_became_active":"2024-03-04T15:49:38.028509+0000","last_became_peered":"2024-03-04T15:49:38.028509+0000","last_unstale":"2024-03-04T15:50:35.212212+0000","last_undegraded":"2024-03-04T15:50:35.212212+0000","last_fullsized":"2024-03-04T15:50:35.212212+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:29:48.184599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124044+0000","last_change":"2024-03-04T15:49:40.455789+0000","last_active":"2024-03-04T15:50:36.124044+0000","last_peered":"2024-03-04T15:50:36.124044+0000","last_clean":"2024-03-04T15:50:36.124044+0000","last_became_active":"2024-03-04T15:49:40.454939+0000","last_became_peered":"2024-03-04T15:49:40.454939+0000","last_unstale":"2024-03-04T15:50:36.124044+0000","last_undegraded":"2024-03-04T15:50:36.124044+0000","last_fullsized":"2024-03-04T15:50:36.124044+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:50:23.549656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.125121+0000","last_change":"2024-03-04T15:49:33.726708+0000","last_active":"2024-03-04T15:50:36.125121+0000","last_peered":"2024-03-04T15:50:36.125121+0000","last_clean":"2024-03-04T15:50:36.125121+0000","last_became_active":"2024-03-04T15:49:33.726158+0000","last_became_peered":"2024-03-04T15:49:33.726158+0000","last_unstale":"2024-03-04T15:50:36.125121+0000","last_undegraded":"2024-03-04T15:50:36.125121+0000","last_fullsized":"2024-03-04T15:50:36.125121+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:58:13.034366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"68'15","reported_seq":61,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124721+0000","last_change":"2024-03-04T15:49:35.760510+0000","last_active":"2024-03-04T15:50:36.124721+0000","last_peered":"2024-03-04T15:50:36.124721+0000","last_clean":"2024-03-04T15:50:36.124721+0000","last_became_active":"2024-03-04T15:49:35.759479+0000","last_became_peered":"2024-03-04T15:49:35.759479+0000","last_unstale":"2024-03-04T15:50:36.124721+0000","last_undegraded":"2024-03-04T15:50:36.124721+0000","last_fullsized":"2024-03-04T15:50:36.124721+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:18:40.099689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123096+0000","last_change":"2024-03-04T15:49:38.154184+0000","last_active":"2024-03-04T15:50:36.123096+0000","last_peered":"2024-03-04T15:50:36.123096+0000","last_clean":"2024-03-04T15:50:36.123096+0000","last_became_active":"2024-03-04T15:49:38.153478+0000","last_became_peered":"2024-03-04T15:49:38.153478+0000","last_unstale":"2024-03-04T15:50:36.123096+0000","last_undegraded":"2024-03-04T15:50:36.123096+0000","last_fullsized":"2024-03-04T15:50:36.123096+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:22:35.525839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":383,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":149,"num_read_kb":144,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":701,"num_read_kb":458,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1457,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201324,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32600,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712552,"statfs":{"total":95995035648,"available":95961653248,"internally_reserved":0,"allocated":1454080,"data_stored":948859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462968,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32572,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712580,"statfs":{"total":95995035648,"available":95961681920,"internally_reserved":0,"allocated":1429504,"data_stored":946698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574955,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":847872,"data_stored":357397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738485,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32592,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712560,"statfs":{"total":95995035648,"available":95961661440,"internally_reserved":0,"allocated":1445888,"data_stored":947362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411426,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":831488,"data_stored":356758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182486,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":831488,"data_stored":356698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986253,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":872448,"data_stored":358488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724610,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":835584,"data_stored":357542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T15:52:28.369 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph pg dump --format=json 2024-03-04T15:52:28.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:28 smithi151 bash[23756]: audit 2024-03-04T15:52:26.369556+0000 mgr.y (mgr.24428) 102 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:52:28.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:28 smithi117 bash[21480]: audit 2024-03-04T15:52:26.369556+0000 mgr.y (mgr.24428) 102 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:52:28.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:28 smithi117 bash[28843]: audit 2024-03-04T15:52:26.369556+0000 mgr.y (mgr.24428) 102 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:52:29.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:29 smithi151 bash[23756]: cluster 2024-03-04T15:52:27.373289+0000 mgr.y (mgr.24428) 103 : cluster 0 pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:29.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:29 smithi151 bash[23756]: audit 2024-03-04T15:52:27.671149+0000 mgr.y (mgr.24428) 104 : audit 0 from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T15:52:29.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:29 smithi117 bash[21480]: cluster 2024-03-04T15:52:27.373289+0000 mgr.y (mgr.24428) 103 : cluster 0 pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:29.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:29 smithi117 bash[21480]: audit 2024-03-04T15:52:27.671149+0000 mgr.y (mgr.24428) 104 : audit 0 from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T15:52:29.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:29 smithi117 bash[28843]: cluster 2024-03-04T15:52:27.373289+0000 mgr.y (mgr.24428) 103 : cluster 0 pgmap v59: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:29.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:29 smithi117 bash[28843]: audit 2024-03-04T15:52:27.671149+0000 mgr.y (mgr.24428) 104 : audit 0 from='client.14712 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T15:52:30.819 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 15:52:30 smithi117 bash[62773]: ts=2024-03-04T15:52:30.544Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003655557s 2024-03-04T15:52:31.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:31 smithi151 bash[23756]: cluster 2024-03-04T15:52:29.373887+0000 mgr.y (mgr.24428) 105 : cluster 0 pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:31.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:31 smithi117 bash[21480]: cluster 2024-03-04T15:52:29.373887+0000 mgr.y (mgr.24428) 105 : cluster 0 pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:31.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:31 smithi117 bash[28843]: cluster 2024-03-04T15:52:29.373887+0000 mgr.y (mgr.24428) 105 : cluster 0 pgmap v60: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:32 smithi151 bash[23756]: audit 2024-03-04T15:52:31.359324+0000 mon.a (mon.0) 837 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:32 smithi151 bash[23756]: audit 2024-03-04T15:52:31.367270+0000 mon.a (mon.0) 838 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:32 smithi151 bash[23756]: cluster 2024-03-04T15:52:31.374641+0000 mgr.y (mgr.24428) 106 : cluster 0 pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:32 smithi151 bash[23756]: audit 2024-03-04T15:52:31.462961+0000 mon.a (mon.0) 839 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:32 smithi151 bash[23756]: audit 2024-03-04T15:52:31.475327+0000 mon.a (mon.0) 840 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:32 smithi151 bash[23756]: audit 2024-03-04T15:52:31.992237+0000 mon.c (mon.2) 77 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:52:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:32 smithi151 bash[23756]: audit 2024-03-04T15:52:31.994144+0000 mon.c (mon.2) 78 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:52:32.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:32 smithi151 bash[23756]: audit 2024-03-04T15:52:32.005401+0000 mon.a (mon.0) 841 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:32.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:32 smithi151 bash[23756]: audit 2024-03-04T15:52:32.091454+0000 mon.c (mon.2) 79 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:52:32.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:32 smithi151 bash[23756]: audit 2024-03-04T15:52:32.093994+0000 mon.c (mon.2) 80 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:52:32.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:32 smithi151 bash[23756]: audit 2024-03-04T15:52:32.095180+0000 mon.c (mon.2) 81 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:52:32.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:32 smithi151 bash[23756]: audit 2024-03-04T15:52:32.105009+0000 mon.a (mon.0) 842 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:32.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[21480]: audit 2024-03-04T15:52:31.359324+0000 mon.a (mon.0) 837 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[21480]: audit 2024-03-04T15:52:31.367270+0000 mon.a (mon.0) 838 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[21480]: cluster 2024-03-04T15:52:31.374641+0000 mgr.y (mgr.24428) 106 : cluster 0 pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[21480]: audit 2024-03-04T15:52:31.462961+0000 mon.a (mon.0) 839 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[21480]: audit 2024-03-04T15:52:31.475327+0000 mon.a (mon.0) 840 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[21480]: audit 2024-03-04T15:52:31.992237+0000 mon.c (mon.2) 77 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:52:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[21480]: audit 2024-03-04T15:52:31.994144+0000 mon.c (mon.2) 78 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:52:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[21480]: audit 2024-03-04T15:52:32.005401+0000 mon.a (mon.0) 841 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[21480]: audit 2024-03-04T15:52:32.091454+0000 mon.c (mon.2) 79 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:52:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[21480]: audit 2024-03-04T15:52:32.093994+0000 mon.c (mon.2) 80 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:52:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[21480]: audit 2024-03-04T15:52:32.095180+0000 mon.c (mon.2) 81 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:52:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[21480]: audit 2024-03-04T15:52:32.105009+0000 mon.a (mon.0) 842 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[28843]: audit 2024-03-04T15:52:31.359324+0000 mon.a (mon.0) 837 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[28843]: audit 2024-03-04T15:52:31.367270+0000 mon.a (mon.0) 838 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[28843]: cluster 2024-03-04T15:52:31.374641+0000 mgr.y (mgr.24428) 106 : cluster 0 pgmap v61: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[28843]: audit 2024-03-04T15:52:31.462961+0000 mon.a (mon.0) 839 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[28843]: audit 2024-03-04T15:52:31.475327+0000 mon.a (mon.0) 840 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[28843]: audit 2024-03-04T15:52:31.992237+0000 mon.c (mon.2) 77 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:52:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[28843]: audit 2024-03-04T15:52:31.994144+0000 mon.c (mon.2) 78 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:52:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[28843]: audit 2024-03-04T15:52:32.005401+0000 mon.a (mon.0) 841 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[28843]: audit 2024-03-04T15:52:32.091454+0000 mon.c (mon.2) 79 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:52:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[28843]: audit 2024-03-04T15:52:32.093994+0000 mon.c (mon.2) 80 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:52:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[28843]: audit 2024-03-04T15:52:32.095180+0000 mon.c (mon.2) 81 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:52:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:32 smithi117 bash[28843]: audit 2024-03-04T15:52:32.105009+0000 mon.a (mon.0) 842 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:52:32.896 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:52:34.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:34 smithi151 bash[23756]: cluster 2024-03-04T15:52:33.375222+0000 mgr.y (mgr.24428) 107 : cluster 0 pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:34.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:34 smithi117 bash[21480]: cluster 2024-03-04T15:52:33.375222+0000 mgr.y (mgr.24428) 107 : cluster 0 pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:34.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:34 smithi117 bash[28843]: cluster 2024-03-04T15:52:33.375222+0000 mgr.y (mgr.24428) 107 : cluster 0 pgmap v62: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:35.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bcd3d8700 1 -- 172.21.15.117:0/2824435242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc806a430 msgr2=0x7f5bc810e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:35.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bcd3d8700 1 --2- 172.21.15.117:0/2824435242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc806a430 0x7f5bc810e3b0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f5bb8009990 tx=0x7f5bb8009ca0 comp rx=0 tx=0).stop 2024-03-04T15:52:35.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bcd3d8700 1 -- 172.21.15.117:0/2824435242 shutdown_connections 2024-03-04T15:52:35.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bcd3d8700 1 --2- 172.21.15.117:0/2824435242 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bc810e8f0 0x7f5bc8110ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:35.540 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bcd3d8700 1 --2- 172.21.15.117:0/2824435242 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bc8069b10 0x7f5bc8069ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:35.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bcd3d8700 1 --2- 172.21.15.117:0/2824435242 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc806a430 0x7f5bc810e3b0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:35.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bcd3d8700 1 -- 172.21.15.117:0/2824435242 >> 172.21.15.117:0/2824435242 conn(0x7f5bc8076d80 msgr2=0x7f5bc8077190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:35.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bcd3d8700 1 -- 172.21.15.117:0/2824435242 shutdown_connections 2024-03-04T15:52:35.541 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bcd3d8700 1 -- 172.21.15.117:0/2824435242 wait complete. 2024-03-04T15:52:35.542 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bcd3d8700 1 Processor -- start 2024-03-04T15:52:35.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bcd3d8700 1 -- start start 2024-03-04T15:52:35.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bcd3d8700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bc8069b10 0x7f5bc819f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:35.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bcd3d8700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc806a430 0x7f5bc819fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:35.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bcd3d8700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bc810e8f0 0x7f5bc8199ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:35.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bcd3d8700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc8113820 con 0x7f5bc806a430 2024-03-04T15:52:35.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bcd3d8700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc81136a0 con 0x7f5bc810e8f0 2024-03-04T15:52:35.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bcd3d8700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc81139a0 con 0x7f5bc8069b10 2024-03-04T15:52:35.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bc6ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bc8069b10 0x7f5bc819f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:35.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bc67fc700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc806a430 0x7f5bc819fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:35.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bc6ffd700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bc8069b10 0x7f5bc819f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:53378/0 (socket says 172.21.15.117:53378) 2024-03-04T15:52:35.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bc6ffd700 1 -- 172.21.15.117:0/860155181 learned_addr learned my addr 172.21.15.117:0/860155181 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:52:35.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bc77fe700 1 --2- 172.21.15.117:0/860155181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bc810e8f0 0x7f5bc8199ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:35.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bc67fc700 1 -- 172.21.15.117:0/860155181 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bc8069b10 msgr2=0x7f5bc819f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:35.543 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bc67fc700 1 --2- 172.21.15.117:0/860155181 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bc8069b10 0x7f5bc819f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:35.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bc67fc700 1 -- 172.21.15.117:0/860155181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bc810e8f0 msgr2=0x7f5bc8199ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:35.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bc67fc700 1 --2- 172.21.15.117:0/860155181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bc810e8f0 0x7f5bc8199ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:35.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bc67fc700 1 -- 172.21.15.117:0/860155181 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc819a210 con 0x7f5bc806a430 2024-03-04T15:52:35.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bc6ffd700 1 --2- 172.21.15.117:0/860155181 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bc8069b10 0x7f5bc819f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T15:52:35.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bc67fc700 1 --2- 172.21.15.117:0/860155181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc806a430 0x7f5bc819fe30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80059d0 tx=0x7f5bb80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:35.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5baffff700 1 -- 172.21.15.117:0/860155181 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8027070 con 0x7f5bc806a430 2024-03-04T15:52:35.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5baffff700 1 -- 172.21.15.117:0/860155181 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f5bb8022470 con 0x7f5bc806a430 2024-03-04T15:52:35.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bcd3d8700 1 -- 172.21.15.117:0/860155181 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc819a4a0 con 0x7f5bc806a430 2024-03-04T15:52:35.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5baffff700 1 -- 172.21.15.117:0/860155181 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8017650 con 0x7f5bc806a430 2024-03-04T15:52:35.544 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.538+0000 7f5bcd3d8700 1 -- 172.21.15.117:0/860155181 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc81a67c0 con 0x7f5bc806a430 2024-03-04T15:52:35.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.542+0000 7f5bcd3d8700 1 -- 172.21.15.117:0/860155181 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc804faa0 con 0x7f5bc806a430 2024-03-04T15:52:35.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.542+0000 7f5baffff700 1 -- 172.21.15.117:0/860155181 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f5bb802a070 con 0x7f5bc806a430 2024-03-04T15:52:35.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.542+0000 7f5baffff700 1 --2- 172.21.15.117:0/860155181 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f5bb006d780 0x7f5bb006fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:35.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.542+0000 7f5baffff700 1 -- 172.21.15.117:0/860155181 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7f5bb808ea80 con 0x7f5bc806a430 2024-03-04T15:52:35.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.542+0000 7f5bc6ffd700 1 --2- 172.21.15.117:0/860155181 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f5bb006d780 0x7f5bb006fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:35.545 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.542+0000 7f5bc6ffd700 1 --2- 172.21.15.117:0/860155181 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f5bb006d780 0x7f5bb006fc40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5bc819abf0 tx=0x7f5bbc009260 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:35.549 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.546+0000 7f5baffff700 1 -- 172.21.15.117:0/860155181 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f5bb8013070 con 0x7f5bc806a430 2024-03-04T15:52:35.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.746+0000 7f5bcd3d8700 1 -- 172.21.15.117:0/860155181 --> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5bc8100440 con 0x7f5bb006d780 2024-03-04T15:52:35.765 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.762+0000 7f5baffff700 1 -- 172.21.15.117:0/860155181 <== mgr.24428 v2:172.21.15.117:6800/2727720514 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+346505 (secure 0 0 0) 0x7f5bc8100440 con 0x7f5bb006d780 2024-03-04T15:52:35.770 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-04T15:52:35.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.766+0000 7f5bcd3d8700 1 -- 172.21.15.117:0/860155181 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f5bb006d780 msgr2=0x7f5bb006fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:35.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.766+0000 7f5bcd3d8700 1 --2- 172.21.15.117:0/860155181 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f5bb006d780 0x7f5bb006fc40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5bc819abf0 tx=0x7f5bbc009260 comp rx=0 tx=0).stop 2024-03-04T15:52:35.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.766+0000 7f5bcd3d8700 1 -- 172.21.15.117:0/860155181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc806a430 msgr2=0x7f5bc819fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:35.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.766+0000 7f5bcd3d8700 1 --2- 172.21.15.117:0/860155181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc806a430 0x7f5bc819fe30 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5bb80059d0 tx=0x7f5bb80047b0 comp rx=0 tx=0).stop 2024-03-04T15:52:35.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.766+0000 7f5bcd3d8700 1 -- 172.21.15.117:0/860155181 shutdown_connections 2024-03-04T15:52:35.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.766+0000 7f5bcd3d8700 1 --2- 172.21.15.117:0/860155181 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f5bc810e8f0 0x7f5bc8199ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:35.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.766+0000 7f5bcd3d8700 1 --2- 172.21.15.117:0/860155181 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f5bc8069b10 0x7f5bc819f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:35.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.766+0000 7f5bcd3d8700 1 --2- 172.21.15.117:0/860155181 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f5bb006d780 0x7f5bb006fc40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:35.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.766+0000 7f5bcd3d8700 1 --2- 172.21.15.117:0/860155181 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f5bc806a430 0x7f5bc819fe30 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:35.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.766+0000 7f5bcd3d8700 1 -- 172.21.15.117:0/860155181 >> 172.21.15.117:0/860155181 conn(0x7f5bc8076d80 msgr2=0x7f5bc80ffd30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:35.773 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.766+0000 7f5bcd3d8700 1 -- 172.21.15.117:0/860155181 shutdown_connections 2024-03-04T15:52:35.774 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:35.766+0000 7f5bcd3d8700 1 -- 172.21.15.117:0/860155181 wait complete. 2024-03-04T15:52:35.774 INFO:teuthology.orchestra.run.smithi117.stderr:dumped all 2024-03-04T15:52:36.409 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:52:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:52:36.410 INFO:teuthology.orchestra.run.smithi117.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2024-03-04T15:52:35.375433+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596318,"num_objects":198,"num_object_clones":0,"num_object_copies":594,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":198,"num_whiteouts":0,"num_read":1104,"num_read_kb":950,"num_write":615,"num_write_kb":2461,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":626,"ondisk_log_size":626,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":253624,"kb_used_data":8348,"kb_used_omap":12,"kb_used_meta":245171,"kb_avail":749707592,"statfs":{"total":767960285184,"available":767700574208,"internally_reserved":0,"allocated":8548352,"data_stored":4629802,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12752,"internal_metadata":251055664},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003976"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.210539+0000","last_change":"2024-03-04T15:49:40.454414+0000","last_active":"2024-03-04T15:50:35.210539+0000","last_peered":"2024-03-04T15:50:35.210539+0000","last_clean":"2024-03-04T15:50:35.210539+0000","last_became_active":"2024-03-04T15:49:40.453465+0000","last_became_peered":"2024-03-04T15:49:40.453465+0000","last_unstale":"2024-03-04T15:50:35.210539+0000","last_undegraded":"2024-03-04T15:50:35.210539+0000","last_fullsized":"2024-03-04T15:50:35.210539+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:52:16.951314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.210327+0000","last_change":"2024-03-04T15:49:38.008560+0000","last_active":"2024-03-04T15:50:35.210327+0000","last_peered":"2024-03-04T15:50:35.210327+0000","last_clean":"2024-03-04T15:50:35.210327+0000","last_became_active":"2024-03-04T15:49:38.007045+0000","last_became_peered":"2024-03-04T15:49:38.007045+0000","last_unstale":"2024-03-04T15:50:35.210327+0000","last_undegraded":"2024-03-04T15:50:35.210327+0000","last_fullsized":"2024-03-04T15:50:35.210327+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T15:54:31.777548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":46,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.215583+0000","last_change":"2024-03-04T15:50:24.435625+0000","last_active":"2024-03-04T15:50:35.215583+0000","last_peered":"2024-03-04T15:50:35.215583+0000","last_clean":"2024-03-04T15:50:35.215583+0000","last_became_active":"2024-03-04T15:50:24.434643+0000","last_became_peered":"2024-03-04T15:50:24.434643+0000","last_unstale":"2024-03-04T15:50:35.215583+0000","last_undegraded":"2024-03-04T15:50:35.215583+0000","last_fullsized":"2024-03-04T15:50:35.215583+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T02:43:46.528518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,4],"acting":[0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1e","version":"68'10","reported_seq":50,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212630+0000","last_change":"2024-03-04T15:49:36.019526+0000","last_active":"2024-03-04T15:50:35.212630+0000","last_peered":"2024-03-04T15:50:35.212630+0000","last_clean":"2024-03-04T15:50:35.212630+0000","last_became_active":"2024-03-04T15:49:36.018951+0000","last_became_peered":"2024-03-04T15:49:36.018951+0000","last_unstale":"2024-03-04T15:50:35.212630+0000","last_undegraded":"2024-03-04T15:50:35.212630+0000","last_fullsized":"2024-03-04T15:50:35.212630+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:20:23.324722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,2],"acting":[3,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.206505+0000","last_change":"2024-03-04T15:49:40.233909+0000","last_active":"2024-03-04T15:50:35.206505+0000","last_peered":"2024-03-04T15:50:35.206505+0000","last_clean":"2024-03-04T15:50:35.206505+0000","last_became_active":"2024-03-04T15:49:40.232809+0000","last_became_peered":"2024-03-04T15:49:40.232809+0000","last_unstale":"2024-03-04T15:50:35.206505+0000","last_undegraded":"2024-03-04T15:50:35.206505+0000","last_fullsized":"2024-03-04T15:50:35.206505+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:18:55.213547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.206019+0000","last_change":"2024-03-04T15:49:38.021094+0000","last_active":"2024-03-04T15:50:35.206019+0000","last_peered":"2024-03-04T15:50:35.206019+0000","last_clean":"2024-03-04T15:50:35.206019+0000","last_became_active":"2024-03-04T15:49:38.020683+0000","last_became_peered":"2024-03-04T15:49:38.020683+0000","last_unstale":"2024-03-04T15:50:35.206019+0000","last_undegraded":"2024-03-04T15:50:35.206019+0000","last_fullsized":"2024-03-04T15:50:35.206019+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:19:23.943917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212430+0000","last_change":"2024-03-04T15:49:33.722796+0000","last_active":"2024-03-04T15:50:35.212430+0000","last_peered":"2024-03-04T15:50:35.212430+0000","last_clean":"2024-03-04T15:50:35.212430+0000","last_became_active":"2024-03-04T15:49:33.722358+0000","last_became_peered":"2024-03-04T15:49:33.722358+0000","last_unstale":"2024-03-04T15:50:35.212430+0000","last_undegraded":"2024-03-04T15:50:35.212430+0000","last_fullsized":"2024-03-04T15:50:35.212430+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:39:33.208389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"68'11","reported_seq":55,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.215322+0000","last_change":"2024-03-04T15:49:36.019614+0000","last_active":"2024-03-04T15:50:35.215322+0000","last_peered":"2024-03-04T15:50:35.215322+0000","last_clean":"2024-03-04T15:50:35.215322+0000","last_became_active":"2024-03-04T15:49:36.019181+0000","last_became_peered":"2024-03-04T15:49:36.019181+0000","last_unstale":"2024-03-04T15:50:35.215322+0000","last_undegraded":"2024-03-04T15:50:35.215322+0000","last_fullsized":"2024-03-04T15:50:35.215322+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:12:28.372013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,2],"acting":[0,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123291+0000","last_change":"2024-03-04T15:49:40.234694+0000","last_active":"2024-03-04T15:50:36.123291+0000","last_peered":"2024-03-04T15:50:36.123291+0000","last_clean":"2024-03-04T15:50:36.123291+0000","last_became_active":"2024-03-04T15:49:40.233800+0000","last_became_peered":"2024-03-04T15:49:40.233800+0000","last_unstale":"2024-03-04T15:50:36.123291+0000","last_undegraded":"2024-03-04T15:50:36.123291+0000","last_fullsized":"2024-03-04T15:50:36.123291+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:24:31.116692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.206798+0000","last_change":"2024-03-04T15:50:25.225075+0000","last_active":"2024-03-04T15:50:35.206798+0000","last_peered":"2024-03-04T15:50:35.206798+0000","last_clean":"2024-03-04T15:50:35.206798+0000","last_became_active":"2024-03-04T15:50:25.224111+0000","last_became_peered":"2024-03-04T15:50:25.224111+0000","last_unstale":"2024-03-04T15:50:35.206798+0000","last_undegraded":"2024-03-04T15:50:35.206798+0000","last_fullsized":"2024-03-04T15:50:35.206798+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:36:29.463751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124697+0000","last_change":"2024-03-04T15:49:33.745748+0000","last_active":"2024-03-04T15:50:36.124697+0000","last_peered":"2024-03-04T15:50:36.124697+0000","last_clean":"2024-03-04T15:50:36.124697+0000","last_became_active":"2024-03-04T15:49:33.745020+0000","last_became_peered":"2024-03-04T15:49:33.745020+0000","last_unstale":"2024-03-04T15:50:36.124697+0000","last_undegraded":"2024-03-04T15:50:36.124697+0000","last_fullsized":"2024-03-04T15:50:36.124697+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:49:27.659528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"68'15","reported_seq":61,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123893+0000","last_change":"2024-03-04T15:49:35.756011+0000","last_active":"2024-03-04T15:50:36.123893+0000","last_peered":"2024-03-04T15:50:36.123893+0000","last_clean":"2024-03-04T15:50:36.123893+0000","last_became_active":"2024-03-04T15:49:35.753803+0000","last_became_peered":"2024-03-04T15:49:35.753803+0000","last_unstale":"2024-03-04T15:50:36.123893+0000","last_undegraded":"2024-03-04T15:50:36.123893+0000","last_fullsized":"2024-03-04T15:50:36.123893+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:17:08.406115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.209409+0000","last_change":"2024-03-04T15:49:40.223844+0000","last_active":"2024-03-04T15:50:35.209409+0000","last_peered":"2024-03-04T15:50:35.209409+0000","last_clean":"2024-03-04T15:50:35.209409+0000","last_became_active":"2024-03-04T15:49:40.223090+0000","last_became_peered":"2024-03-04T15:49:40.223090+0000","last_unstale":"2024-03-04T15:50:35.209409+0000","last_undegraded":"2024-03-04T15:50:35.209409+0000","last_fullsized":"2024-03-04T15:50:35.209409+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:02:43.864367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.209646+0000","last_change":"2024-03-04T15:49:38.005549+0000","last_active":"2024-03-04T15:50:35.209646+0000","last_peered":"2024-03-04T15:50:35.209646+0000","last_clean":"2024-03-04T15:50:35.209646+0000","last_became_active":"2024-03-04T15:49:38.005216+0000","last_became_peered":"2024-03-04T15:49:38.005216+0000","last_unstale":"2024-03-04T15:50:35.209646+0000","last_undegraded":"2024-03-04T15:50:35.209646+0000","last_fullsized":"2024-03-04T15:50:35.209646+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:03:24.628355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124270+0000","last_change":"2024-03-04T15:49:33.742659+0000","last_active":"2024-03-04T15:50:36.124270+0000","last_peered":"2024-03-04T15:50:36.124270+0000","last_clean":"2024-03-04T15:50:36.124270+0000","last_became_active":"2024-03-04T15:49:33.742250+0000","last_became_peered":"2024-03-04T15:49:33.742250+0000","last_unstale":"2024-03-04T15:50:36.124270+0000","last_undegraded":"2024-03-04T15:50:36.124270+0000","last_fullsized":"2024-03-04T15:50:36.124270+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:22:39.463025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"68'12","reported_seq":59,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124191+0000","last_change":"2024-03-04T15:49:35.761336+0000","last_active":"2024-03-04T15:50:36.124191+0000","last_peered":"2024-03-04T15:50:36.124191+0000","last_clean":"2024-03-04T15:50:36.124191+0000","last_became_active":"2024-03-04T15:49:35.760529+0000","last_became_peered":"2024-03-04T15:49:35.760529+0000","last_unstale":"2024-03-04T15:50:36.124191+0000","last_undegraded":"2024-03-04T15:50:36.124191+0000","last_fullsized":"2024-03-04T15:50:36.124191+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:34:24.578708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.211629+0000","last_change":"2024-03-04T15:49:40.450091+0000","last_active":"2024-03-04T15:50:35.211629+0000","last_peered":"2024-03-04T15:50:35.211629+0000","last_clean":"2024-03-04T15:50:35.211629+0000","last_became_active":"2024-03-04T15:49:40.448918+0000","last_became_peered":"2024-03-04T15:49:40.448918+0000","last_unstale":"2024-03-04T15:50:35.211629+0000","last_undegraded":"2024-03-04T15:50:35.211629+0000","last_fullsized":"2024-03-04T15:50:35.211629+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:07:41.121850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212016+0000","last_change":"2024-03-04T15:49:38.025651+0000","last_active":"2024-03-04T15:50:35.212016+0000","last_peered":"2024-03-04T15:50:35.212016+0000","last_clean":"2024-03-04T15:50:35.212016+0000","last_became_active":"2024-03-04T15:49:38.024369+0000","last_became_peered":"2024-03-04T15:49:38.024369+0000","last_unstale":"2024-03-04T15:50:35.212016+0000","last_undegraded":"2024-03-04T15:50:35.212016+0000","last_fullsized":"2024-03-04T15:50:35.212016+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:30:02.409366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":39,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.210762+0000","last_change":"2024-03-04T15:49:33.734073+0000","last_active":"2024-03-04T15:50:35.210762+0000","last_peered":"2024-03-04T15:50:35.210762+0000","last_clean":"2024-03-04T15:50:35.210762+0000","last_became_active":"2024-03-04T15:49:33.733597+0000","last_became_peered":"2024-03-04T15:49:33.733597+0000","last_unstale":"2024-03-04T15:50:35.210762+0000","last_undegraded":"2024-03-04T15:50:35.210762+0000","last_fullsized":"2024-03-04T15:50:35.210762+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T03:22:06.068088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"68'9","reported_seq":51,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.206775+0000","last_change":"2024-03-04T15:49:36.020344+0000","last_active":"2024-03-04T15:50:35.206775+0000","last_peered":"2024-03-04T15:50:35.206775+0000","last_clean":"2024-03-04T15:50:35.206775+0000","last_became_active":"2024-03-04T15:49:36.018786+0000","last_became_peered":"2024-03-04T15:49:36.018786+0000","last_unstale":"2024-03-04T15:50:35.206775+0000","last_undegraded":"2024-03-04T15:50:35.206775+0000","last_fullsized":"2024-03-04T15:50:35.206775+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:44:52.178972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212482+0000","last_change":"2024-03-04T15:49:40.448772+0000","last_active":"2024-03-04T15:50:35.212482+0000","last_peered":"2024-03-04T15:50:35.212482+0000","last_clean":"2024-03-04T15:50:35.212482+0000","last_became_active":"2024-03-04T15:49:40.447880+0000","last_became_peered":"2024-03-04T15:49:40.447880+0000","last_unstale":"2024-03-04T15:50:35.212482+0000","last_undegraded":"2024-03-04T15:50:35.212482+0000","last_fullsized":"2024-03-04T15:50:35.212482+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:14:12.689534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124780+0000","last_change":"2024-03-04T15:49:38.018359+0000","last_active":"2024-03-04T15:50:36.124780+0000","last_peered":"2024-03-04T15:50:36.124780+0000","last_clean":"2024-03-04T15:50:36.124780+0000","last_became_active":"2024-03-04T15:49:38.017460+0000","last_became_peered":"2024-03-04T15:49:38.017460+0000","last_unstale":"2024-03-04T15:50:36.124780+0000","last_undegraded":"2024-03-04T15:50:36.124780+0000","last_fullsized":"2024-03-04T15:50:36.124780+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T02:34:35.302959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.352368+0000","last_change":"2024-03-04T15:49:33.745021+0000","last_active":"2024-03-04T15:50:35.352368+0000","last_peered":"2024-03-04T15:50:35.352368+0000","last_clean":"2024-03-04T15:50:35.352368+0000","last_became_active":"2024-03-04T15:49:33.744313+0000","last_became_peered":"2024-03-04T15:49:33.744313+0000","last_unstale":"2024-03-04T15:50:35.352368+0000","last_undegraded":"2024-03-04T15:50:35.352368+0000","last_fullsized":"2024-03-04T15:50:35.352368+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:34:48.854618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1b","version":"68'5","reported_seq":46,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.211729+0000","last_change":"2024-03-04T15:49:35.762179+0000","last_active":"2024-03-04T15:50:35.211729+0000","last_peered":"2024-03-04T15:50:35.211729+0000","last_clean":"2024-03-04T15:50:35.211729+0000","last_became_active":"2024-03-04T15:49:35.761588+0000","last_became_peered":"2024-03-04T15:49:35.761588+0000","last_unstale":"2024-03-04T15:50:35.211729+0000","last_undegraded":"2024-03-04T15:50:35.211729+0000","last_fullsized":"2024-03-04T15:50:35.211729+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:58:43.444571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,7],"acting":[0,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124273+0000","last_change":"2024-03-04T15:49:40.230342+0000","last_active":"2024-03-04T15:50:36.124273+0000","last_peered":"2024-03-04T15:50:36.124273+0000","last_clean":"2024-03-04T15:50:36.124273+0000","last_became_active":"2024-03-04T15:49:40.229448+0000","last_became_peered":"2024-03-04T15:49:40.229448+0000","last_unstale":"2024-03-04T15:50:36.124273+0000","last_undegraded":"2024-03-04T15:50:36.124273+0000","last_fullsized":"2024-03-04T15:50:36.124273+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:50:39.830014+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1f","version":"68'11","reported_seq":81,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:23.956955+0000","last_change":"2024-03-04T15:49:38.025404+0000","last_active":"2024-03-04T15:52:23.956955+0000","last_peered":"2024-03-04T15:52:23.956955+0000","last_clean":"2024-03-04T15:52:23.956955+0000","last_became_active":"2024-03-04T15:49:38.024652+0000","last_became_peered":"2024-03-04T15:49:38.024652+0000","last_unstale":"2024-03-04T15:52:23.956955+0000","last_undegraded":"2024-03-04T15:52:23.956955+0000","last_fullsized":"2024-03-04T15:52:23.956955+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:31:43.271841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"61'1","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.210629+0000","last_change":"2024-03-04T15:49:33.729239+0000","last_active":"2024-03-04T15:50:35.210629+0000","last_peered":"2024-03-04T15:50:35.210629+0000","last_clean":"2024-03-04T15:50:35.210629+0000","last_became_active":"2024-03-04T15:49:33.728380+0000","last_became_peered":"2024-03-04T15:49:33.728380+0000","last_unstale":"2024-03-04T15:50:35.210629+0000","last_undegraded":"2024-03-04T15:50:35.210629+0000","last_fullsized":"2024-03-04T15:50:35.210629+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:49:57.697270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"68'9","reported_seq":51,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212220+0000","last_change":"2024-03-04T15:49:35.745777+0000","last_active":"2024-03-04T15:50:35.212220+0000","last_peered":"2024-03-04T15:50:35.212220+0000","last_clean":"2024-03-04T15:50:35.212220+0000","last_became_active":"2024-03-04T15:49:35.744738+0000","last_became_peered":"2024-03-04T15:49:35.744738+0000","last_unstale":"2024-03-04T15:50:35.212220+0000","last_undegraded":"2024-03-04T15:50:35.212220+0000","last_fullsized":"2024-03-04T15:50:35.212220+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:00:50.454045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"68'1","reported_seq":30,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124114+0000","last_change":"2024-03-04T15:49:40.224804+0000","last_active":"2024-03-04T15:50:36.124114+0000","last_peered":"2024-03-04T15:50:36.124114+0000","last_clean":"2024-03-04T15:50:36.124114+0000","last_became_active":"2024-03-04T15:49:40.222546+0000","last_became_peered":"2024-03-04T15:49:40.222546+0000","last_unstale":"2024-03-04T15:50:36.124114+0000","last_undegraded":"2024-03-04T15:50:36.124114+0000","last_fullsized":"2024-03-04T15:50:36.124114+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T02:48:38.643953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":370,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.214105+0000","last_change":"2024-03-04T15:49:38.154004+0000","last_active":"2024-03-04T15:50:35.214105+0000","last_peered":"2024-03-04T15:50:35.214105+0000","last_clean":"2024-03-04T15:50:35.214105+0000","last_became_active":"2024-03-04T15:49:38.153103+0000","last_became_peered":"2024-03-04T15:49:38.153103+0000","last_unstale":"2024-03-04T15:50:35.214105+0000","last_undegraded":"2024-03-04T15:50:35.214105+0000","last_fullsized":"2024-03-04T15:50:35.214105+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T02:25:19.106156+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"61'1","reported_seq":48,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123198+0000","last_change":"2024-03-04T15:49:33.733649+0000","last_active":"2024-03-04T15:50:36.123198+0000","last_peered":"2024-03-04T15:50:36.123198+0000","last_clean":"2024-03-04T15:50:36.123198+0000","last_became_active":"2024-03-04T15:49:33.732996+0000","last_became_peered":"2024-03-04T15:49:33.732996+0000","last_unstale":"2024-03-04T15:50:36.123198+0000","last_undegraded":"2024-03-04T15:50:36.123198+0000","last_fullsized":"2024-03-04T15:50:36.123198+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:28:42.575711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":875,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"68'15","reported_seq":61,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.125061+0000","last_change":"2024-03-04T15:49:35.741487+0000","last_active":"2024-03-04T15:50:36.125061+0000","last_peered":"2024-03-04T15:50:36.125061+0000","last_clean":"2024-03-04T15:50:36.125061+0000","last_became_active":"2024-03-04T15:49:35.740695+0000","last_became_peered":"2024-03-04T15:49:35.740695+0000","last_unstale":"2024-03-04T15:50:36.125061+0000","last_undegraded":"2024-03-04T15:50:36.125061+0000","last_fullsized":"2024-03-04T15:50:36.125061+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:58:07.692797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.13","version":"68'1","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.211894+0000","last_change":"2024-03-04T15:49:40.453843+0000","last_active":"2024-03-04T15:50:35.211894+0000","last_peered":"2024-03-04T15:50:35.211894+0000","last_clean":"2024-03-04T15:50:35.211894+0000","last_became_active":"2024-03-04T15:49:40.452961+0000","last_became_peered":"2024-03-04T15:49:40.452961+0000","last_unstale":"2024-03-04T15:50:35.211894+0000","last_undegraded":"2024-03-04T15:50:35.211894+0000","last_fullsized":"2024-03-04T15:50:35.211894+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:48:08.355002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212014+0000","last_change":"2024-03-04T15:49:38.155702+0000","last_active":"2024-03-04T15:50:35.212014+0000","last_peered":"2024-03-04T15:50:35.212014+0000","last_clean":"2024-03-04T15:50:35.212014+0000","last_became_active":"2024-03-04T15:49:38.153847+0000","last_became_peered":"2024-03-04T15:49:38.153847+0000","last_unstale":"2024-03-04T15:50:35.212014+0000","last_undegraded":"2024-03-04T15:50:35.212014+0000","last_fullsized":"2024-03-04T15:50:35.212014+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:59:12.772824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.352985+0000","last_change":"2024-03-04T15:49:33.731621+0000","last_active":"2024-03-04T15:50:35.352985+0000","last_peered":"2024-03-04T15:50:35.352985+0000","last_clean":"2024-03-04T15:50:35.352985+0000","last_became_active":"2024-03-04T15:49:33.730840+0000","last_became_peered":"2024-03-04T15:49:33.730840+0000","last_unstale":"2024-03-04T15:50:35.352985+0000","last_undegraded":"2024-03-04T15:50:35.352985+0000","last_fullsized":"2024-03-04T15:50:35.352985+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:14:25.492830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"68'9","reported_seq":52,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123736+0000","last_change":"2024-03-04T15:49:35.761467+0000","last_active":"2024-03-04T15:50:36.123736+0000","last_peered":"2024-03-04T15:50:36.123736+0000","last_clean":"2024-03-04T15:50:36.123736+0000","last_became_active":"2024-03-04T15:49:35.760732+0000","last_became_peered":"2024-03-04T15:49:35.760732+0000","last_unstale":"2024-03-04T15:50:36.123736+0000","last_undegraded":"2024-03-04T15:50:36.123736+0000","last_fullsized":"2024-03-04T15:50:36.123736+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:36:54.861620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,1],"acting":[5,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123955+0000","last_change":"2024-03-04T15:49:40.222254+0000","last_active":"2024-03-04T15:50:36.123955+0000","last_peered":"2024-03-04T15:50:36.123955+0000","last_clean":"2024-03-04T15:50:36.123955+0000","last_became_active":"2024-03-04T15:49:40.220585+0000","last_became_peered":"2024-03-04T15:49:40.220585+0000","last_unstale":"2024-03-04T15:50:36.123955+0000","last_undegraded":"2024-03-04T15:50:36.123955+0000","last_fullsized":"2024-03-04T15:50:36.123955+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:44:50.631574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212328+0000","last_change":"2024-03-04T15:49:38.023501+0000","last_active":"2024-03-04T15:50:35.212328+0000","last_peered":"2024-03-04T15:50:35.212328+0000","last_clean":"2024-03-04T15:50:35.212328+0000","last_became_active":"2024-03-04T15:49:38.022802+0000","last_became_peered":"2024-03-04T15:49:38.022802+0000","last_unstale":"2024-03-04T15:50:35.212328+0000","last_undegraded":"2024-03-04T15:50:35.212328+0000","last_fullsized":"2024-03-04T15:50:35.212328+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:57:29.028920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123591+0000","last_change":"2024-03-04T15:49:33.732988+0000","last_active":"2024-03-04T15:50:36.123591+0000","last_peered":"2024-03-04T15:50:36.123591+0000","last_clean":"2024-03-04T15:50:36.123591+0000","last_became_active":"2024-03-04T15:49:33.732188+0000","last_became_peered":"2024-03-04T15:49:33.732188+0000","last_unstale":"2024-03-04T15:50:36.123591+0000","last_undegraded":"2024-03-04T15:50:36.123591+0000","last_fullsized":"2024-03-04T15:50:36.123591+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:57:59.295570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"68'6","reported_seq":45,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.215161+0000","last_change":"2024-03-04T15:49:35.746507+0000","last_active":"2024-03-04T15:50:35.215161+0000","last_peered":"2024-03-04T15:50:35.215161+0000","last_clean":"2024-03-04T15:50:35.215161+0000","last_became_active":"2024-03-04T15:49:35.746061+0000","last_became_peered":"2024-03-04T15:49:35.746061+0000","last_unstale":"2024-03-04T15:50:35.215161+0000","last_undegraded":"2024-03-04T15:50:35.215161+0000","last_fullsized":"2024-03-04T15:50:35.215161+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T03:17:17.568851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,3],"acting":[0,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.210016+0000","last_change":"2024-03-04T15:49:40.244574+0000","last_active":"2024-03-04T15:50:35.210016+0000","last_peered":"2024-03-04T15:50:35.210016+0000","last_clean":"2024-03-04T15:50:35.210016+0000","last_became_active":"2024-03-04T15:49:40.241779+0000","last_became_peered":"2024-03-04T15:49:40.241779+0000","last_unstale":"2024-03-04T15:50:35.210016+0000","last_undegraded":"2024-03-04T15:50:35.210016+0000","last_fullsized":"2024-03-04T15:50:35.210016+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:44:16.846417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.207070+0000","last_change":"2024-03-04T15:49:38.015865+0000","last_active":"2024-03-04T15:50:35.207070+0000","last_peered":"2024-03-04T15:50:35.207070+0000","last_clean":"2024-03-04T15:50:35.207070+0000","last_became_active":"2024-03-04T15:49:38.015580+0000","last_became_peered":"2024-03-04T15:49:38.015580+0000","last_unstale":"2024-03-04T15:50:35.207070+0000","last_undegraded":"2024-03-04T15:50:35.207070+0000","last_fullsized":"2024-03-04T15:50:35.207070+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:13:40.591696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124593+0000","last_change":"2024-03-04T15:49:33.739909+0000","last_active":"2024-03-04T15:50:36.124593+0000","last_peered":"2024-03-04T15:50:36.124593+0000","last_clean":"2024-03-04T15:50:36.124593+0000","last_became_active":"2024-03-04T15:49:33.739286+0000","last_became_peered":"2024-03-04T15:49:33.739286+0000","last_unstale":"2024-03-04T15:50:36.124593+0000","last_undegraded":"2024-03-04T15:50:36.124593+0000","last_fullsized":"2024-03-04T15:50:36.124593+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:51:22.451758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"68'10","reported_seq":50,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.211312+0000","last_change":"2024-03-04T15:49:35.752001+0000","last_active":"2024-03-04T15:50:35.211312+0000","last_peered":"2024-03-04T15:50:35.211312+0000","last_clean":"2024-03-04T15:50:35.211312+0000","last_became_active":"2024-03-04T15:49:35.751095+0000","last_became_peered":"2024-03-04T15:49:35.751095+0000","last_unstale":"2024-03-04T15:50:35.211312+0000","last_undegraded":"2024-03-04T15:50:35.211312+0000","last_fullsized":"2024-03-04T15:50:35.211312+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:10:21.653391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,6],"acting":[4,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.209504+0000","last_change":"2024-03-04T15:49:40.226410+0000","last_active":"2024-03-04T15:50:35.209504+0000","last_peered":"2024-03-04T15:50:35.209504+0000","last_clean":"2024-03-04T15:50:35.209504+0000","last_became_active":"2024-03-04T15:49:40.224534+0000","last_became_peered":"2024-03-04T15:49:40.224534+0000","last_unstale":"2024-03-04T15:50:35.209504+0000","last_undegraded":"2024-03-04T15:50:35.209504+0000","last_fullsized":"2024-03-04T15:50:35.209504+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T02:37:17.273217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123528+0000","last_change":"2024-03-04T15:50:24.707076+0000","last_active":"2024-03-04T15:50:36.123528+0000","last_peered":"2024-03-04T15:50:36.123528+0000","last_clean":"2024-03-04T15:50:36.123528+0000","last_became_active":"2024-03-04T15:50:24.705855+0000","last_became_peered":"2024-03-04T15:50:24.705855+0000","last_unstale":"2024-03-04T15:50:36.123528+0000","last_undegraded":"2024-03-04T15:50:36.123528+0000","last_fullsized":"2024-03-04T15:50:36.123528+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:30:15.354268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.352792+0000","last_change":"2024-03-04T15:49:33.723819+0000","last_active":"2024-03-04T15:50:35.352792+0000","last_peered":"2024-03-04T15:50:35.352792+0000","last_clean":"2024-03-04T15:50:35.352792+0000","last_became_active":"2024-03-04T15:49:33.723042+0000","last_became_peered":"2024-03-04T15:49:33.723042+0000","last_unstale":"2024-03-04T15:50:35.352792+0000","last_undegraded":"2024-03-04T15:50:35.352792+0000","last_fullsized":"2024-03-04T15:50:35.352792+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:29:27.784721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"68'9","reported_seq":52,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124808+0000","last_change":"2024-03-04T15:49:35.761256+0000","last_active":"2024-03-04T15:50:36.124808+0000","last_peered":"2024-03-04T15:50:36.124808+0000","last_clean":"2024-03-04T15:50:36.124808+0000","last_became_active":"2024-03-04T15:49:35.759779+0000","last_became_peered":"2024-03-04T15:49:35.759779+0000","last_unstale":"2024-03-04T15:50:36.124808+0000","last_undegraded":"2024-03-04T15:50:36.124808+0000","last_fullsized":"2024-03-04T15:50:36.124808+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:08:55.485292+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,4],"acting":[7,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.214062+0000","last_change":"2024-03-04T15:49:40.230543+0000","last_active":"2024-03-04T15:50:35.214062+0000","last_peered":"2024-03-04T15:50:35.214062+0000","last_clean":"2024-03-04T15:50:35.214062+0000","last_became_active":"2024-03-04T15:49:40.229704+0000","last_became_peered":"2024-03-04T15:49:40.229704+0000","last_unstale":"2024-03-04T15:50:35.214062+0000","last_undegraded":"2024-03-04T15:50:35.214062+0000","last_fullsized":"2024-03-04T15:50:35.214062+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:55:17.972990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.15","version":"68'11","reported_seq":78,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:23.956752+0000","last_change":"2024-03-04T15:49:38.153570+0000","last_active":"2024-03-04T15:52:23.956752+0000","last_peered":"2024-03-04T15:52:23.956752+0000","last_clean":"2024-03-04T15:52:23.956752+0000","last_became_active":"2024-03-04T15:49:38.152308+0000","last_became_peered":"2024-03-04T15:49:38.152308+0000","last_unstale":"2024-03-04T15:52:23.956752+0000","last_undegraded":"2024-03-04T15:52:23.956752+0000","last_fullsized":"2024-03-04T15:52:23.956752+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:16:51.267725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.211460+0000","last_change":"2024-03-04T15:49:33.722458+0000","last_active":"2024-03-04T15:50:35.211460+0000","last_peered":"2024-03-04T15:50:35.211460+0000","last_clean":"2024-03-04T15:50:35.211460+0000","last_became_active":"2024-03-04T15:49:33.722017+0000","last_became_peered":"2024-03-04T15:49:33.722017+0000","last_unstale":"2024-03-04T15:50:35.211460+0000","last_undegraded":"2024-03-04T15:50:35.211460+0000","last_fullsized":"2024-03-04T15:50:35.211460+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:21:26.960543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"68'9","reported_seq":52,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.211273+0000","last_change":"2024-03-04T15:49:35.761391+0000","last_active":"2024-03-04T15:50:35.211273+0000","last_peered":"2024-03-04T15:50:35.211273+0000","last_clean":"2024-03-04T15:50:35.211273+0000","last_became_active":"2024-03-04T15:49:35.761044+0000","last_became_peered":"2024-03-04T15:49:35.761044+0000","last_unstale":"2024-03-04T15:50:35.211273+0000","last_undegraded":"2024-03-04T15:50:35.211273+0000","last_fullsized":"2024-03-04T15:50:35.211273+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:18:46.800261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.213350+0000","last_change":"2024-03-04T15:49:40.229860+0000","last_active":"2024-03-04T15:50:35.213350+0000","last_peered":"2024-03-04T15:50:35.213350+0000","last_clean":"2024-03-04T15:50:35.213350+0000","last_became_active":"2024-03-04T15:49:40.228885+0000","last_became_peered":"2024-03-04T15:49:40.228885+0000","last_unstale":"2024-03-04T15:50:35.213350+0000","last_undegraded":"2024-03-04T15:50:35.213350+0000","last_fullsized":"2024-03-04T15:50:35.213350+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:42:49.125554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.14","version":"68'11","reported_seq":77,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:23.959020+0000","last_change":"2024-03-04T15:49:38.149577+0000","last_active":"2024-03-04T15:52:23.959020+0000","last_peered":"2024-03-04T15:52:23.959020+0000","last_clean":"2024-03-04T15:52:23.959020+0000","last_became_active":"2024-03-04T15:49:38.148679+0000","last_became_peered":"2024-03-04T15:49:38.148679+0000","last_unstale":"2024-03-04T15:52:23.959020+0000","last_undegraded":"2024-03-04T15:52:23.959020+0000","last_fullsized":"2024-03-04T15:52:23.959020+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:35:37.381831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123694+0000","last_change":"2024-03-04T15:49:33.734387+0000","last_active":"2024-03-04T15:50:36.123694+0000","last_peered":"2024-03-04T15:50:36.123694+0000","last_clean":"2024-03-04T15:50:36.123694+0000","last_became_active":"2024-03-04T15:49:33.733203+0000","last_became_peered":"2024-03-04T15:49:33.733203+0000","last_unstale":"2024-03-04T15:50:36.123694+0000","last_undegraded":"2024-03-04T15:50:36.123694+0000","last_fullsized":"2024-03-04T15:50:36.123694+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:38:58.200689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"68'11","reported_seq":55,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124494+0000","last_change":"2024-03-04T15:49:36.018304+0000","last_active":"2024-03-04T15:50:36.124494+0000","last_peered":"2024-03-04T15:50:36.124494+0000","last_clean":"2024-03-04T15:50:36.124494+0000","last_became_active":"2024-03-04T15:49:36.017280+0000","last_became_peered":"2024-03-04T15:49:36.017280+0000","last_unstale":"2024-03-04T15:50:36.124494+0000","last_undegraded":"2024-03-04T15:50:36.124494+0000","last_fullsized":"2024-03-04T15:50:36.124494+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:41:44.305666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,2],"acting":[7,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124183+0000","last_change":"2024-03-04T15:49:40.449911+0000","last_active":"2024-03-04T15:50:36.124183+0000","last_peered":"2024-03-04T15:50:36.124183+0000","last_clean":"2024-03-04T15:50:36.124183+0000","last_became_active":"2024-03-04T15:49:40.448339+0000","last_became_peered":"2024-03-04T15:49:40.448339+0000","last_unstale":"2024-03-04T15:50:36.124183+0000","last_undegraded":"2024-03-04T15:50:36.124183+0000","last_fullsized":"2024-03-04T15:50:36.124183+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:05:04.659649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212132+0000","last_change":"2024-03-04T15:49:38.149380+0000","last_active":"2024-03-04T15:50:35.212132+0000","last_peered":"2024-03-04T15:50:35.212132+0000","last_clean":"2024-03-04T15:50:35.212132+0000","last_became_active":"2024-03-04T15:49:38.148466+0000","last_became_peered":"2024-03-04T15:49:38.148466+0000","last_unstale":"2024-03-04T15:50:35.212132+0000","last_undegraded":"2024-03-04T15:50:35.212132+0000","last_fullsized":"2024-03-04T15:50:35.212132+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:14:17.425748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.352254+0000","last_change":"2024-03-04T15:49:33.729849+0000","last_active":"2024-03-04T15:50:35.352254+0000","last_peered":"2024-03-04T15:50:35.352254+0000","last_clean":"2024-03-04T15:50:35.352254+0000","last_became_active":"2024-03-04T15:49:33.727548+0000","last_became_peered":"2024-03-04T15:49:33.727548+0000","last_unstale":"2024-03-04T15:50:35.352254+0000","last_undegraded":"2024-03-04T15:50:35.352254+0000","last_fullsized":"2024-03-04T15:50:35.352254+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:33:15.112491+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"68'4","reported_seq":42,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.352124+0000","last_change":"2024-03-04T15:49:35.763348+0000","last_active":"2024-03-04T15:50:35.352124+0000","last_peered":"2024-03-04T15:50:35.352124+0000","last_clean":"2024-03-04T15:50:35.352124+0000","last_became_active":"2024-03-04T15:49:35.762268+0000","last_became_peered":"2024-03-04T15:49:35.762268+0000","last_unstale":"2024-03-04T15:50:35.352124+0000","last_undegraded":"2024-03-04T15:50:35.352124+0000","last_fullsized":"2024-03-04T15:50:35.352124+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:01:10.129428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124839+0000","last_change":"2024-03-04T15:49:40.231754+0000","last_active":"2024-03-04T15:50:36.124839+0000","last_peered":"2024-03-04T15:50:36.124839+0000","last_clean":"2024-03-04T15:50:36.124839+0000","last_became_active":"2024-03-04T15:49:40.230895+0000","last_became_peered":"2024-03-04T15:49:40.230895+0000","last_unstale":"2024-03-04T15:50:36.124839+0000","last_undegraded":"2024-03-04T15:50:36.124839+0000","last_fullsized":"2024-03-04T15:50:36.124839+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:56:45.443819+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.213076+0000","last_change":"2024-03-04T15:49:38.154343+0000","last_active":"2024-03-04T15:50:35.213076+0000","last_peered":"2024-03-04T15:50:35.213076+0000","last_clean":"2024-03-04T15:50:35.213076+0000","last_became_active":"2024-03-04T15:49:38.153419+0000","last_became_peered":"2024-03-04T15:49:38.153419+0000","last_unstale":"2024-03-04T15:50:35.213076+0000","last_undegraded":"2024-03-04T15:50:35.213076+0000","last_fullsized":"2024-03-04T15:50:35.213076+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:04:51.005645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124095+0000","last_change":"2024-03-04T15:49:33.731519+0000","last_active":"2024-03-04T15:50:36.124095+0000","last_peered":"2024-03-04T15:50:36.124095+0000","last_clean":"2024-03-04T15:50:36.124095+0000","last_became_active":"2024-03-04T15:49:33.731099+0000","last_became_peered":"2024-03-04T15:49:33.731099+0000","last_unstale":"2024-03-04T15:50:36.124095+0000","last_undegraded":"2024-03-04T15:50:36.124095+0000","last_fullsized":"2024-03-04T15:50:36.124095+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:21:46.553452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"68'11","reported_seq":55,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124592+0000","last_change":"2024-03-04T15:49:35.762217+0000","last_active":"2024-03-04T15:50:36.124592+0000","last_peered":"2024-03-04T15:50:36.124592+0000","last_clean":"2024-03-04T15:50:36.124592+0000","last_became_active":"2024-03-04T15:49:35.761361+0000","last_became_peered":"2024-03-04T15:49:35.761361+0000","last_unstale":"2024-03-04T15:50:36.124592+0000","last_undegraded":"2024-03-04T15:50:36.124592+0000","last_fullsized":"2024-03-04T15:50:36.124592+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:53:35.491762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.211902+0000","last_change":"2024-03-04T15:49:40.242353+0000","last_active":"2024-03-04T15:50:35.211902+0000","last_peered":"2024-03-04T15:50:35.211902+0000","last_clean":"2024-03-04T15:50:35.211902+0000","last_became_active":"2024-03-04T15:49:40.239419+0000","last_became_peered":"2024-03-04T15:49:40.239419+0000","last_unstale":"2024-03-04T15:50:35.211902+0000","last_undegraded":"2024-03-04T15:50:35.211902+0000","last_fullsized":"2024-03-04T15:50:35.211902+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:13:28.962703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.f","version":"61'2","reported_seq":55,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.213243+0000","last_change":"2024-03-04T15:49:33.743679+0000","last_active":"2024-03-04T15:50:35.213243+0000","last_peered":"2024-03-04T15:50:35.213243+0000","last_clean":"2024-03-04T15:50:35.213243+0000","last_became_active":"2024-03-04T15:49:33.742860+0000","last_became_peered":"2024-03-04T15:49:33.742860+0000","last_unstale":"2024-03-04T15:50:35.213243+0000","last_undegraded":"2024-03-04T15:50:35.213243+0000","last_fullsized":"2024-03-04T15:50:35.213243+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:36:01.739238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"68'11","reported_seq":55,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124793+0000","last_change":"2024-03-04T15:49:35.760228+0000","last_active":"2024-03-04T15:50:36.124793+0000","last_peered":"2024-03-04T15:50:36.124793+0000","last_clean":"2024-03-04T15:50:36.124793+0000","last_became_active":"2024-03-04T15:49:35.759385+0000","last_became_peered":"2024-03-04T15:49:35.759385+0000","last_unstale":"2024-03-04T15:50:36.124793+0000","last_undegraded":"2024-03-04T15:50:36.124793+0000","last_fullsized":"2024-03-04T15:50:36.124793+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:25:25.525878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"68'11","reported_seq":77,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:23.956949+0000","last_change":"2024-03-04T15:49:38.008342+0000","last_active":"2024-03-04T15:52:23.956949+0000","last_peered":"2024-03-04T15:52:23.956949+0000","last_clean":"2024-03-04T15:52:23.956949+0000","last_became_active":"2024-03-04T15:49:38.008072+0000","last_became_peered":"2024-03-04T15:49:38.008072+0000","last_unstale":"2024-03-04T15:52:23.956949+0000","last_undegraded":"2024-03-04T15:52:23.956949+0000","last_fullsized":"2024-03-04T15:52:23.956949+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:08:15.812918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.209595+0000","last_change":"2024-03-04T15:49:40.234479+0000","last_active":"2024-03-04T15:50:35.209595+0000","last_peered":"2024-03-04T15:50:35.209595+0000","last_clean":"2024-03-04T15:50:35.209595+0000","last_became_active":"2024-03-04T15:49:40.233708+0000","last_became_peered":"2024-03-04T15:49:40.233708+0000","last_unstale":"2024-03-04T15:50:35.209595+0000","last_undegraded":"2024-03-04T15:50:35.209595+0000","last_fullsized":"2024-03-04T15:50:35.209595+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:57:42.295900+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.125409+0000","last_change":"2024-03-04T15:49:33.742626+0000","last_active":"2024-03-04T15:50:36.125409+0000","last_peered":"2024-03-04T15:50:36.125409+0000","last_clean":"2024-03-04T15:50:36.125409+0000","last_became_active":"2024-03-04T15:49:33.742298+0000","last_became_peered":"2024-03-04T15:49:33.742298+0000","last_unstale":"2024-03-04T15:50:36.125409+0000","last_undegraded":"2024-03-04T15:50:36.125409+0000","last_fullsized":"2024-03-04T15:50:36.125409+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:21:39.371802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"68'16","reported_seq":94,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:23.957174+0000","last_change":"2024-03-04T15:49:36.018752+0000","last_active":"2024-03-04T15:52:23.957174+0000","last_peered":"2024-03-04T15:52:23.957174+0000","last_clean":"2024-03-04T15:52:23.957174+0000","last_became_active":"2024-03-04T15:49:36.017875+0000","last_became_peered":"2024-03-04T15:49:36.017875+0000","last_unstale":"2024-03-04T15:52:23.957174+0000","last_undegraded":"2024-03-04T15:52:23.957174+0000","last_fullsized":"2024-03-04T15:52:23.957174+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:03:37.662284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,2],"acting":[5,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124100+0000","last_change":"2024-03-04T15:50:24.435649+0000","last_active":"2024-03-04T15:50:36.124100+0000","last_peered":"2024-03-04T15:50:36.124100+0000","last_clean":"2024-03-04T15:50:36.124100+0000","last_became_active":"2024-03-04T15:50:24.434494+0000","last_became_peered":"2024-03-04T15:50:24.434494+0000","last_unstale":"2024-03-04T15:50:36.124100+0000","last_undegraded":"2024-03-04T15:50:36.124100+0000","last_fullsized":"2024-03-04T15:50:36.124100+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:37:36.962551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212520+0000","last_change":"2024-03-04T15:49:40.240101+0000","last_active":"2024-03-04T15:50:35.212520+0000","last_peered":"2024-03-04T15:50:35.212520+0000","last_clean":"2024-03-04T15:50:35.212520+0000","last_became_active":"2024-03-04T15:49:40.238274+0000","last_became_peered":"2024-03-04T15:49:40.238274+0000","last_unstale":"2024-03-04T15:50:35.212520+0000","last_undegraded":"2024-03-04T15:50:35.212520+0000","last_fullsized":"2024-03-04T15:50:35.212520+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T02:29:38.047202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123810+0000","last_change":"2024-03-04T15:49:33.736608+0000","last_active":"2024-03-04T15:50:36.123810+0000","last_peered":"2024-03-04T15:50:36.123810+0000","last_clean":"2024-03-04T15:50:36.123810+0000","last_became_active":"2024-03-04T15:49:33.735769+0000","last_became_peered":"2024-03-04T15:49:33.735769+0000","last_unstale":"2024-03-04T15:50:36.123810+0000","last_undegraded":"2024-03-04T15:50:36.123810+0000","last_fullsized":"2024-03-04T15:50:36.123810+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T02:22:24.567798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"68'19","reported_seq":71,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.208039+0000","last_change":"2024-03-04T15:49:35.745883+0000","last_active":"2024-03-04T15:50:35.208039+0000","last_peered":"2024-03-04T15:50:35.208039+0000","last_clean":"2024-03-04T15:50:35.208039+0000","last_became_active":"2024-03-04T15:49:35.744948+0000","last_became_peered":"2024-03-04T15:49:35.744948+0000","last_unstale":"2024-03-04T15:50:35.208039+0000","last_undegraded":"2024-03-04T15:50:35.208039+0000","last_fullsized":"2024-03-04T15:50:35.208039+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:30:22.582548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,6],"acting":[4,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.125318+0000","last_change":"2024-03-04T15:49:38.021820+0000","last_active":"2024-03-04T15:50:36.125318+0000","last_peered":"2024-03-04T15:50:36.125318+0000","last_clean":"2024-03-04T15:50:36.125318+0000","last_became_active":"2024-03-04T15:49:38.020920+0000","last_became_peered":"2024-03-04T15:49:38.020920+0000","last_unstale":"2024-03-04T15:50:36.125318+0000","last_undegraded":"2024-03-04T15:50:36.125318+0000","last_fullsized":"2024-03-04T15:50:36.125318+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:56:56.529484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124306+0000","last_change":"2024-03-04T15:49:40.450577+0000","last_active":"2024-03-04T15:50:36.124306+0000","last_peered":"2024-03-04T15:50:36.124306+0000","last_clean":"2024-03-04T15:50:36.124306+0000","last_became_active":"2024-03-04T15:49:40.447504+0000","last_became_peered":"2024-03-04T15:49:40.447504+0000","last_unstale":"2024-03-04T15:50:36.124306+0000","last_undegraded":"2024-03-04T15:50:36.124306+0000","last_fullsized":"2024-03-04T15:50:36.124306+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:12:58.834631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"68'18","reported_seq":68,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124384+0000","last_change":"2024-03-04T15:49:36.017775+0000","last_active":"2024-03-04T15:50:36.124384+0000","last_peered":"2024-03-04T15:50:36.124384+0000","last_clean":"2024-03-04T15:50:36.124384+0000","last_became_active":"2024-03-04T15:49:36.017336+0000","last_became_peered":"2024-03-04T15:49:36.017336+0000","last_unstale":"2024-03-04T15:50:36.124384+0000","last_undegraded":"2024-03-04T15:50:36.124384+0000","last_fullsized":"2024-03-04T15:50:36.124384+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:45:15.781821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124205+0000","last_change":"2024-03-04T15:49:33.722318+0000","last_active":"2024-03-04T15:50:36.124205+0000","last_peered":"2024-03-04T15:50:36.124205+0000","last_clean":"2024-03-04T15:50:36.124205+0000","last_became_active":"2024-03-04T15:49:33.721824+0000","last_became_peered":"2024-03-04T15:49:33.721824+0000","last_unstale":"2024-03-04T15:50:36.124205+0000","last_undegraded":"2024-03-04T15:50:36.124205+0000","last_fullsized":"2024-03-04T15:50:36.124205+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:48:53.316898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124634+0000","last_change":"2024-03-04T15:49:38.024580+0000","last_active":"2024-03-04T15:50:36.124634+0000","last_peered":"2024-03-04T15:50:36.124634+0000","last_clean":"2024-03-04T15:50:36.124634+0000","last_became_active":"2024-03-04T15:49:38.023731+0000","last_became_peered":"2024-03-04T15:49:38.023731+0000","last_unstale":"2024-03-04T15:50:36.124634+0000","last_undegraded":"2024-03-04T15:50:36.124634+0000","last_fullsized":"2024-03-04T15:50:36.124634+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:49:24.817743+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123990+0000","last_change":"2024-03-04T15:49:40.237131+0000","last_active":"2024-03-04T15:50:36.123990+0000","last_peered":"2024-03-04T15:50:36.123990+0000","last_clean":"2024-03-04T15:50:36.123990+0000","last_became_active":"2024-03-04T15:49:40.236283+0000","last_became_peered":"2024-03-04T15:49:40.236283+0000","last_unstale":"2024-03-04T15:50:36.123990+0000","last_undegraded":"2024-03-04T15:50:36.123990+0000","last_fullsized":"2024-03-04T15:50:36.123990+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:42:48.192315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"68'10","reported_seq":50,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.211102+0000","last_change":"2024-03-04T15:49:35.758734+0000","last_active":"2024-03-04T15:50:35.211102+0000","last_peered":"2024-03-04T15:50:35.211102+0000","last_clean":"2024-03-04T15:50:35.211102+0000","last_became_active":"2024-03-04T15:49:35.758030+0000","last_became_peered":"2024-03-04T15:49:35.758030+0000","last_unstale":"2024-03-04T15:50:35.211102+0000","last_undegraded":"2024-03-04T15:50:35.211102+0000","last_fullsized":"2024-03-04T15:50:35.211102+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:30:01.936137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123399+0000","last_change":"2024-03-04T15:49:33.735443+0000","last_active":"2024-03-04T15:50:36.123399+0000","last_peered":"2024-03-04T15:50:36.123399+0000","last_clean":"2024-03-04T15:50:36.123399+0000","last_became_active":"2024-03-04T15:49:33.734226+0000","last_became_peered":"2024-03-04T15:49:33.734226+0000","last_unstale":"2024-03-04T15:50:36.123399+0000","last_undegraded":"2024-03-04T15:50:36.123399+0000","last_fullsized":"2024-03-04T15:50:36.123399+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:45:36.310086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"75'128","reported_seq":283,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:37.377774+0000","last_change":"2024-03-04T15:48:57.791739+0000","last_active":"2024-03-04T15:50:37.377774+0000","last_peered":"2024-03-04T15:50:37.377774+0000","last_clean":"2024-03-04T15:50:37.377774+0000","last_became_active":"2024-03-04T15:48:57.790648+0000","last_became_peered":"2024-03-04T15:48:57.790648+0000","last_unstale":"2024-03-04T15:50:37.377774+0000","last_undegraded":"2024-03-04T15:50:37.377774+0000","last_fullsized":"2024-03-04T15:50:37.377774+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:44:31.731812+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:44:31.731812+0000","last_clean_scrub_stamp":"2024-03-04T15:44:31.731812+0000","objects_scrubbed":0,"log_size":128,"log_dups_size":0,"ondisk_log_size":128,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:47:26.809771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.352653+0000","last_change":"2024-03-04T15:49:38.153971+0000","last_active":"2024-03-04T15:50:35.352653+0000","last_peered":"2024-03-04T15:50:35.352653+0000","last_clean":"2024-03-04T15:50:35.352653+0000","last_became_active":"2024-03-04T15:49:38.153129+0000","last_became_peered":"2024-03-04T15:49:38.153129+0000","last_unstale":"2024-03-04T15:50:35.352653+0000","last_undegraded":"2024-03-04T15:50:35.352653+0000","last_fullsized":"2024-03-04T15:50:35.352653+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:39:29.323685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.125226+0000","last_change":"2024-03-04T15:49:40.236245+0000","last_active":"2024-03-04T15:50:36.125226+0000","last_peered":"2024-03-04T15:50:36.125226+0000","last_clean":"2024-03-04T15:50:36.125226+0000","last_became_active":"2024-03-04T15:49:40.230772+0000","last_became_peered":"2024-03-04T15:49:40.230772+0000","last_unstale":"2024-03-04T15:50:36.125226+0000","last_undegraded":"2024-03-04T15:50:36.125226+0000","last_fullsized":"2024-03-04T15:50:36.125226+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:19:57.650635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"68'14","reported_seq":57,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.210501+0000","last_change":"2024-03-04T15:49:35.742578+0000","last_active":"2024-03-04T15:50:35.210501+0000","last_peered":"2024-03-04T15:50:35.210501+0000","last_clean":"2024-03-04T15:50:35.210501+0000","last_became_active":"2024-03-04T15:49:35.741577+0000","last_became_peered":"2024-03-04T15:49:35.741577+0000","last_unstale":"2024-03-04T15:50:35.210501+0000","last_undegraded":"2024-03-04T15:50:35.210501+0000","last_fullsized":"2024-03-04T15:50:35.210501+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:46:40.777731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"61'1","reported_seq":48,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123358+0000","last_change":"2024-03-04T15:49:33.745052+0000","last_active":"2024-03-04T15:50:36.123358+0000","last_peered":"2024-03-04T15:50:36.123358+0000","last_clean":"2024-03-04T15:50:36.123358+0000","last_became_active":"2024-03-04T15:49:33.744105+0000","last_became_peered":"2024-03-04T15:49:33.744105+0000","last_unstale":"2024-03-04T15:50:36.123358+0000","last_undegraded":"2024-03-04T15:50:36.123358+0000","last_fullsized":"2024-03-04T15:50:36.123358+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:45:27.817273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":398,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.213592+0000","last_change":"2024-03-04T15:49:38.024959+0000","last_active":"2024-03-04T15:50:35.213592+0000","last_peered":"2024-03-04T15:50:35.213592+0000","last_clean":"2024-03-04T15:50:35.213592+0000","last_became_active":"2024-03-04T15:49:38.024138+0000","last_became_peered":"2024-03-04T15:49:38.024138+0000","last_unstale":"2024-03-04T15:50:35.213592+0000","last_undegraded":"2024-03-04T15:50:35.213592+0000","last_fullsized":"2024-03-04T15:50:35.213592+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:23:31.269691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124930+0000","last_change":"2024-03-04T15:49:40.448899+0000","last_active":"2024-03-04T15:50:36.124930+0000","last_peered":"2024-03-04T15:50:36.124930+0000","last_clean":"2024-03-04T15:50:36.124930+0000","last_became_active":"2024-03-04T15:49:40.447962+0000","last_became_peered":"2024-03-04T15:49:40.447962+0000","last_unstale":"2024-03-04T15:50:36.124930+0000","last_undegraded":"2024-03-04T15:50:36.124930+0000","last_fullsized":"2024-03-04T15:50:36.124930+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:05:43.644548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"71'2","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.352325+0000","last_change":"2024-03-04T15:49:48.250908+0000","last_active":"2024-03-04T15:50:35.352325+0000","last_peered":"2024-03-04T15:50:35.352325+0000","last_clean":"2024-03-04T15:50:35.352325+0000","last_became_active":"2024-03-04T15:49:38.000176+0000","last_became_peered":"2024-03-04T15:49:38.000176+0000","last_unstale":"2024-03-04T15:50:35.352325+0000","last_undegraded":"2024-03-04T15:50:35.352325+0000","last_fullsized":"2024-03-04T15:50:35.352325+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:09:44.006918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002567518,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124441+0000","last_change":"2024-03-04T15:49:33.750901+0000","last_active":"2024-03-04T15:50:36.124441+0000","last_peered":"2024-03-04T15:50:36.124441+0000","last_clean":"2024-03-04T15:50:36.124441+0000","last_became_active":"2024-03-04T15:49:33.750070+0000","last_became_peered":"2024-03-04T15:49:33.750070+0000","last_unstale":"2024-03-04T15:50:36.124441+0000","last_undegraded":"2024-03-04T15:50:36.124441+0000","last_fullsized":"2024-03-04T15:50:36.124441+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:43:06.430464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"68'30","reported_seq":122,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:23.958275+0000","last_change":"2024-03-04T15:49:36.019151+0000","last_active":"2024-03-04T15:52:23.958275+0000","last_peered":"2024-03-04T15:52:23.958275+0000","last_clean":"2024-03-04T15:52:23.958275+0000","last_became_active":"2024-03-04T15:49:36.017482+0000","last_became_peered":"2024-03-04T15:49:36.017482+0000","last_unstale":"2024-03-04T15:52:23.958275+0000","last_undegraded":"2024-03-04T15:52:23.958275+0000","last_fullsized":"2024-03-04T15:52:23.958275+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:17:04.424822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":51,"num_read_kb":36,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,5],"acting":[1,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"68'11","reported_seq":78,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:23.958348+0000","last_change":"2024-03-04T15:49:38.154171+0000","last_active":"2024-03-04T15:52:23.958348+0000","last_peered":"2024-03-04T15:52:23.958348+0000","last_clean":"2024-03-04T15:52:23.958348+0000","last_became_active":"2024-03-04T15:49:38.153230+0000","last_became_peered":"2024-03-04T15:49:38.153230+0000","last_unstale":"2024-03-04T15:52:23.958348+0000","last_undegraded":"2024-03-04T15:52:23.958348+0000","last_fullsized":"2024-03-04T15:52:23.958348+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:04:34.338590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.207110+0000","last_change":"2024-03-04T15:49:40.238777+0000","last_active":"2024-03-04T15:50:35.207110+0000","last_peered":"2024-03-04T15:50:35.207110+0000","last_clean":"2024-03-04T15:50:35.207110+0000","last_became_active":"2024-03-04T15:49:40.238132+0000","last_became_peered":"2024-03-04T15:49:40.238132+0000","last_unstale":"2024-03-04T15:50:35.207110+0000","last_undegraded":"2024-03-04T15:50:35.207110+0000","last_fullsized":"2024-03-04T15:50:35.207110+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:17:07.947883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":37,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.206190+0000","last_change":"2024-03-04T15:49:48.245050+0000","last_active":"2024-03-04T15:50:35.206190+0000","last_peered":"2024-03-04T15:50:35.206190+0000","last_clean":"2024-03-04T15:50:35.206190+0000","last_became_active":"2024-03-04T15:49:38.152238+0000","last_became_peered":"2024-03-04T15:49:38.152238+0000","last_unstale":"2024-03-04T15:50:35.206190+0000","last_undegraded":"2024-03-04T15:50:35.206190+0000","last_fullsized":"2024-03-04T15:50:35.206190+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:43:48.329784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052544289999999997,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"61'1","reported_seq":41,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124690+0000","last_change":"2024-03-04T15:49:33.732491+0000","last_active":"2024-03-04T15:50:36.124690+0000","last_peered":"2024-03-04T15:50:36.124690+0000","last_clean":"2024-03-04T15:50:36.124690+0000","last_became_active":"2024-03-04T15:49:33.731911+0000","last_became_peered":"2024-03-04T15:49:33.731911+0000","last_unstale":"2024-03-04T15:50:36.124690+0000","last_undegraded":"2024-03-04T15:50:36.124690+0000","last_fullsized":"2024-03-04T15:50:36.124690+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T03:41:49.855451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"68'13","reported_seq":62,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.210907+0000","last_change":"2024-03-04T15:49:35.761285+0000","last_active":"2024-03-04T15:50:35.210907+0000","last_peered":"2024-03-04T15:50:35.210907+0000","last_clean":"2024-03-04T15:50:35.210907+0000","last_became_active":"2024-03-04T15:49:35.760870+0000","last_became_peered":"2024-03-04T15:49:35.760870+0000","last_unstale":"2024-03-04T15:50:35.210907+0000","last_undegraded":"2024-03-04T15:50:35.210907+0000","last_fullsized":"2024-03-04T15:50:35.210907+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:10:46.714373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"68'11","reported_seq":77,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:23.958466+0000","last_change":"2024-03-04T15:49:38.154931+0000","last_active":"2024-03-04T15:52:23.958466+0000","last_peered":"2024-03-04T15:52:23.958466+0000","last_clean":"2024-03-04T15:52:23.958466+0000","last_became_active":"2024-03-04T15:49:38.153768+0000","last_became_peered":"2024-03-04T15:49:38.153768+0000","last_unstale":"2024-03-04T15:52:23.958466+0000","last_undegraded":"2024-03-04T15:52:23.958466+0000","last_fullsized":"2024-03-04T15:52:23.958466+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T03:13:19.860639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124117+0000","last_change":"2024-03-04T15:49:40.457188+0000","last_active":"2024-03-04T15:50:36.124117+0000","last_peered":"2024-03-04T15:50:36.124117+0000","last_clean":"2024-03-04T15:50:36.124117+0000","last_became_active":"2024-03-04T15:49:40.455923+0000","last_became_peered":"2024-03-04T15:49:40.455923+0000","last_unstale":"2024-03-04T15:50:36.124117+0000","last_undegraded":"2024-03-04T15:50:36.124117+0000","last_fullsized":"2024-03-04T15:50:36.124117+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:50:49.499327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"72'5","reported_seq":194,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:30.078235+0000","last_change":"2024-03-04T15:49:49.018527+0000","last_active":"2024-03-04T15:52:30.078235+0000","last_peered":"2024-03-04T15:52:30.078235+0000","last_clean":"2024-03-04T15:52:30.078235+0000","last_became_active":"2024-03-04T15:49:38.025846+0000","last_became_peered":"2024-03-04T15:49:38.025846+0000","last_unstale":"2024-03-04T15:52:30.078235+0000","last_undegraded":"2024-03-04T15:52:30.078235+0000","last_fullsized":"2024-03-04T15:52:30.078235+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T03:12:49.580578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00097583999999999995,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":154,"num_read_kb":149,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.352956+0000","last_change":"2024-03-04T15:49:33.743893+0000","last_active":"2024-03-04T15:50:35.352956+0000","last_peered":"2024-03-04T15:50:35.352956+0000","last_clean":"2024-03-04T15:50:35.352956+0000","last_became_active":"2024-03-04T15:49:33.743521+0000","last_became_peered":"2024-03-04T15:49:33.743521+0000","last_unstale":"2024-03-04T15:50:35.352956+0000","last_undegraded":"2024-03-04T15:50:35.352956+0000","last_fullsized":"2024-03-04T15:50:35.352956+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T02:53:56.520019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"68'12","reported_seq":54,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.214467+0000","last_change":"2024-03-04T15:49:35.742623+0000","last_active":"2024-03-04T15:50:35.214467+0000","last_peered":"2024-03-04T15:50:35.214467+0000","last_clean":"2024-03-04T15:50:35.214467+0000","last_became_active":"2024-03-04T15:49:35.741672+0000","last_became_peered":"2024-03-04T15:49:35.741672+0000","last_unstale":"2024-03-04T15:50:35.214467+0000","last_undegraded":"2024-03-04T15:50:35.214467+0000","last_fullsized":"2024-03-04T15:50:35.214467+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:14:44.466768+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.206612+0000","last_change":"2024-03-04T15:49:37.997336+0000","last_active":"2024-03-04T15:50:35.206612+0000","last_peered":"2024-03-04T15:50:35.206612+0000","last_clean":"2024-03-04T15:50:35.206612+0000","last_became_active":"2024-03-04T15:49:37.996986+0000","last_became_peered":"2024-03-04T15:49:37.996986+0000","last_unstale":"2024-03-04T15:50:35.206612+0000","last_undegraded":"2024-03-04T15:50:35.206612+0000","last_fullsized":"2024-03-04T15:50:35.206612+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T03:20:51.222665+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.211126+0000","last_change":"2024-03-04T15:49:40.449478+0000","last_active":"2024-03-04T15:50:35.211126+0000","last_peered":"2024-03-04T15:50:35.211126+0000","last_clean":"2024-03-04T15:50:35.211126+0000","last_became_active":"2024-03-04T15:49:40.448321+0000","last_became_peered":"2024-03-04T15:49:40.448321+0000","last_unstale":"2024-03-04T15:50:35.211126+0000","last_undegraded":"2024-03-04T15:50:35.211126+0000","last_fullsized":"2024-03-04T15:50:35.211126+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T03:00:26.668940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.e","version":"68'11","reported_seq":80,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:23.956743+0000","last_change":"2024-03-04T15:49:38.021034+0000","last_active":"2024-03-04T15:52:23.956743+0000","last_peered":"2024-03-04T15:52:23.956743+0000","last_clean":"2024-03-04T15:52:23.956743+0000","last_became_active":"2024-03-04T15:49:38.020598+0000","last_became_peered":"2024-03-04T15:49:38.020598+0000","last_unstale":"2024-03-04T15:52:23.956743+0000","last_undegraded":"2024-03-04T15:52:23.956743+0000","last_fullsized":"2024-03-04T15:52:23.956743+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:08:38.560356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123561+0000","last_change":"2024-03-04T15:49:33.745414+0000","last_active":"2024-03-04T15:50:36.123561+0000","last_peered":"2024-03-04T15:50:36.123561+0000","last_clean":"2024-03-04T15:50:36.123561+0000","last_became_active":"2024-03-04T15:49:33.744726+0000","last_became_peered":"2024-03-04T15:49:33.744726+0000","last_unstale":"2024-03-04T15:50:36.123561+0000","last_undegraded":"2024-03-04T15:50:36.123561+0000","last_fullsized":"2024-03-04T15:50:36.123561+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T20:07:14.994749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"68'12","reported_seq":58,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.206460+0000","last_change":"2024-03-04T15:49:36.018293+0000","last_active":"2024-03-04T15:50:35.206460+0000","last_peered":"2024-03-04T15:50:35.206460+0000","last_clean":"2024-03-04T15:50:35.206460+0000","last_became_active":"2024-03-04T15:49:36.017401+0000","last_became_peered":"2024-03-04T15:49:36.017401+0000","last_unstale":"2024-03-04T15:50:35.206460+0000","last_undegraded":"2024-03-04T15:50:35.206460+0000","last_fullsized":"2024-03-04T15:50:35.206460+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T18:31:35.332038+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,7],"acting":[4,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124320+0000","last_change":"2024-03-04T15:49:40.223056+0000","last_active":"2024-03-04T15:50:36.124320+0000","last_peered":"2024-03-04T15:50:36.124320+0000","last_clean":"2024-03-04T15:50:36.124320+0000","last_became_active":"2024-03-04T15:49:40.222141+0000","last_became_peered":"2024-03-04T15:49:40.222141+0000","last_unstale":"2024-03-04T15:50:36.124320+0000","last_undegraded":"2024-03-04T15:50:36.124320+0000","last_fullsized":"2024-03-04T15:50:36.124320+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:50:29.163434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124722+0000","last_change":"2024-03-04T15:49:38.023066+0000","last_active":"2024-03-04T15:50:36.124722+0000","last_peered":"2024-03-04T15:50:36.124722+0000","last_clean":"2024-03-04T15:50:36.124722+0000","last_became_active":"2024-03-04T15:49:38.022328+0000","last_became_peered":"2024-03-04T15:49:38.022328+0000","last_unstale":"2024-03-04T15:50:36.124722+0000","last_undegraded":"2024-03-04T15:50:36.124722+0000","last_fullsized":"2024-03-04T15:50:36.124722+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:39:42.087976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.125014+0000","last_change":"2024-03-04T15:49:33.731253+0000","last_active":"2024-03-04T15:50:36.125014+0000","last_peered":"2024-03-04T15:50:36.125014+0000","last_clean":"2024-03-04T15:50:36.125014+0000","last_became_active":"2024-03-04T15:49:33.730734+0000","last_became_peered":"2024-03-04T15:49:33.730734+0000","last_unstale":"2024-03-04T15:50:36.125014+0000","last_undegraded":"2024-03-04T15:50:36.125014+0000","last_fullsized":"2024-03-04T15:50:36.125014+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:13:57.535530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"68'15","reported_seq":60,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212490+0000","last_change":"2024-03-04T15:49:35.749494+0000","last_active":"2024-03-04T15:50:35.212490+0000","last_peered":"2024-03-04T15:50:35.212490+0000","last_clean":"2024-03-04T15:50:35.212490+0000","last_became_active":"2024-03-04T15:49:35.748825+0000","last_became_peered":"2024-03-04T15:49:35.748825+0000","last_unstale":"2024-03-04T15:50:35.212490+0000","last_undegraded":"2024-03-04T15:50:35.212490+0000","last_fullsized":"2024-03-04T15:50:35.212490+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:28:12.248198+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212748+0000","last_change":"2024-03-04T15:49:40.234279+0000","last_active":"2024-03-04T15:50:35.212748+0000","last_peered":"2024-03-04T15:50:35.212748+0000","last_clean":"2024-03-04T15:50:35.212748+0000","last_became_active":"2024-03-04T15:49:40.233338+0000","last_became_peered":"2024-03-04T15:49:40.233338+0000","last_unstale":"2024-03-04T15:50:35.212748+0000","last_undegraded":"2024-03-04T15:50:35.212748+0000","last_fullsized":"2024-03-04T15:50:35.212748+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:31:56.525426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.125265+0000","last_change":"2024-03-04T15:49:33.731659+0000","last_active":"2024-03-04T15:50:36.125265+0000","last_peered":"2024-03-04T15:50:36.125265+0000","last_clean":"2024-03-04T15:50:36.125265+0000","last_became_active":"2024-03-04T15:49:33.730899+0000","last_became_peered":"2024-03-04T15:49:33.730899+0000","last_unstale":"2024-03-04T15:50:36.125265+0000","last_undegraded":"2024-03-04T15:50:36.125265+0000","last_fullsized":"2024-03-04T15:50:36.125265+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T02:46:34.770313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"68'9","reported_seq":51,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.210886+0000","last_change":"2024-03-04T15:49:35.746336+0000","last_active":"2024-03-04T15:50:35.210886+0000","last_peered":"2024-03-04T15:50:35.210886+0000","last_clean":"2024-03-04T15:50:35.210886+0000","last_became_active":"2024-03-04T15:49:35.745920+0000","last_became_peered":"2024-03-04T15:49:35.745920+0000","last_unstale":"2024-03-04T15:50:35.210886+0000","last_undegraded":"2024-03-04T15:50:35.210886+0000","last_fullsized":"2024-03-04T15:50:35.210886+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:12:58.978436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,4],"acting":[3,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":31,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.211160+0000","last_change":"2024-03-04T15:49:38.005369+0000","last_active":"2024-03-04T15:50:35.211160+0000","last_peered":"2024-03-04T15:50:35.211160+0000","last_clean":"2024-03-04T15:50:35.211160+0000","last_became_active":"2024-03-04T15:49:38.005061+0000","last_became_peered":"2024-03-04T15:49:38.005061+0000","last_unstale":"2024-03-04T15:50:35.211160+0000","last_undegraded":"2024-03-04T15:50:35.211160+0000","last_fullsized":"2024-03-04T15:50:35.211160+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T17:34:22.171049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.125310+0000","last_change":"2024-03-04T15:49:40.245338+0000","last_active":"2024-03-04T15:50:36.125310+0000","last_peered":"2024-03-04T15:50:36.125310+0000","last_clean":"2024-03-04T15:50:36.125310+0000","last_became_active":"2024-03-04T15:49:40.244547+0000","last_became_peered":"2024-03-04T15:49:40.244547+0000","last_unstale":"2024-03-04T15:50:36.125310+0000","last_undegraded":"2024-03-04T15:50:36.125310+0000","last_fullsized":"2024-03-04T15:50:36.125310+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T00:04:33.291376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.d","version":"68'11","reported_seq":77,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:52:23.956858+0000","last_change":"2024-03-04T15:49:38.021279+0000","last_active":"2024-03-04T15:52:23.956858+0000","last_peered":"2024-03-04T15:52:23.956858+0000","last_clean":"2024-03-04T15:52:23.956858+0000","last_became_active":"2024-03-04T15:49:38.020844+0000","last_became_peered":"2024-03-04T15:49:38.020844+0000","last_unstale":"2024-03-04T15:52:23.956858+0000","last_undegraded":"2024-03-04T15:52:23.956858+0000","last_fullsized":"2024-03-04T15:52:23.956858+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:02:30.542297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124480+0000","last_change":"2024-03-04T15:49:33.748854+0000","last_active":"2024-03-04T15:50:36.124480+0000","last_peered":"2024-03-04T15:50:36.124480+0000","last_clean":"2024-03-04T15:50:36.124480+0000","last_became_active":"2024-03-04T15:49:33.748356+0000","last_became_peered":"2024-03-04T15:49:33.748356+0000","last_unstale":"2024-03-04T15:50:36.124480+0000","last_undegraded":"2024-03-04T15:50:36.124480+0000","last_fullsized":"2024-03-04T15:50:36.124480+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:55:49.935117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"68'19","reported_seq":67,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.352494+0000","last_change":"2024-03-04T15:49:35.753931+0000","last_active":"2024-03-04T15:50:35.352494+0000","last_peered":"2024-03-04T15:50:35.352494+0000","last_clean":"2024-03-04T15:50:35.352494+0000","last_became_active":"2024-03-04T15:49:35.752842+0000","last_became_peered":"2024-03-04T15:49:35.752842+0000","last_unstale":"2024-03-04T15:50:35.352494+0000","last_undegraded":"2024-03-04T15:50:35.352494+0000","last_fullsized":"2024-03-04T15:50:35.352494+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T19:25:43.530434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123180+0000","last_change":"2024-03-04T15:49:40.225692+0000","last_active":"2024-03-04T15:50:36.123180+0000","last_peered":"2024-03-04T15:50:36.123180+0000","last_clean":"2024-03-04T15:50:36.123180+0000","last_became_active":"2024-03-04T15:49:40.224006+0000","last_became_peered":"2024-03-04T15:49:40.224006+0000","last_unstale":"2024-03-04T15:50:36.123180+0000","last_undegraded":"2024-03-04T15:50:36.123180+0000","last_fullsized":"2024-03-04T15:50:36.123180+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T16:55:44.359786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124667+0000","last_change":"2024-03-04T15:49:33.725281+0000","last_active":"2024-03-04T15:50:36.124667+0000","last_peered":"2024-03-04T15:50:36.124667+0000","last_clean":"2024-03-04T15:50:36.124667+0000","last_became_active":"2024-03-04T15:49:33.724481+0000","last_became_peered":"2024-03-04T15:49:33.724481+0000","last_unstale":"2024-03-04T15:50:36.124667+0000","last_undegraded":"2024-03-04T15:50:36.124667+0000","last_fullsized":"2024-03-04T15:50:36.124667+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:33:47.773620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"68'17","reported_seq":64,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124671+0000","last_change":"2024-03-04T15:49:35.764116+0000","last_active":"2024-03-04T15:50:36.124671+0000","last_peered":"2024-03-04T15:50:36.124671+0000","last_clean":"2024-03-04T15:50:36.124671+0000","last_became_active":"2024-03-04T15:49:35.763229+0000","last_became_peered":"2024-03-04T15:49:35.763229+0000","last_unstale":"2024-03-04T15:50:36.124671+0000","last_undegraded":"2024-03-04T15:50:36.124671+0000","last_fullsized":"2024-03-04T15:50:36.124671+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:06:55.820085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,6],"acting":[7,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.352264+0000","last_change":"2024-03-04T15:49:38.148607+0000","last_active":"2024-03-04T15:50:35.352264+0000","last_peered":"2024-03-04T15:50:35.352264+0000","last_clean":"2024-03-04T15:50:35.352264+0000","last_became_active":"2024-03-04T15:49:38.147960+0000","last_became_peered":"2024-03-04T15:49:38.147960+0000","last_unstale":"2024-03-04T15:50:35.352264+0000","last_undegraded":"2024-03-04T15:50:35.352264+0000","last_fullsized":"2024-03-04T15:50:35.352264+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:12:26.050916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.209699+0000","last_change":"2024-03-04T15:49:40.220474+0000","last_active":"2024-03-04T15:50:35.209699+0000","last_peered":"2024-03-04T15:50:35.209699+0000","last_clean":"2024-03-04T15:50:35.209699+0000","last_became_active":"2024-03-04T15:49:40.219620+0000","last_became_peered":"2024-03-04T15:49:40.219620+0000","last_unstale":"2024-03-04T15:50:35.209699+0000","last_undegraded":"2024-03-04T15:50:35.209699+0000","last_fullsized":"2024-03-04T15:50:35.209699+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:05:26.162699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124752+0000","last_change":"2024-03-04T15:49:33.736207+0000","last_active":"2024-03-04T15:50:36.124752+0000","last_peered":"2024-03-04T15:50:36.124752+0000","last_clean":"2024-03-04T15:50:36.124752+0000","last_became_active":"2024-03-04T15:49:33.735658+0000","last_became_peered":"2024-03-04T15:49:33.735658+0000","last_unstale":"2024-03-04T15:50:36.124752+0000","last_undegraded":"2024-03-04T15:50:36.124752+0000","last_fullsized":"2024-03-04T15:50:36.124752+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:39:47.411302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"68'10","reported_seq":51,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123591+0000","last_change":"2024-03-04T15:49:35.769383+0000","last_active":"2024-03-04T15:50:36.123591+0000","last_peered":"2024-03-04T15:50:36.123591+0000","last_clean":"2024-03-04T15:50:36.123591+0000","last_became_active":"2024-03-04T15:49:35.768373+0000","last_became_peered":"2024-03-04T15:49:35.768373+0000","last_unstale":"2024-03-04T15:50:36.123591+0000","last_undegraded":"2024-03-04T15:50:36.123591+0000","last_fullsized":"2024-03-04T15:50:36.123591+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T03:07:08.632363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,6],"acting":[5,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:35.212212+0000","last_change":"2024-03-04T15:49:38.029335+0000","last_active":"2024-03-04T15:50:35.212212+0000","last_peered":"2024-03-04T15:50:35.212212+0000","last_clean":"2024-03-04T15:50:35.212212+0000","last_became_active":"2024-03-04T15:49:38.028509+0000","last_became_peered":"2024-03-04T15:49:38.028509+0000","last_unstale":"2024-03-04T15:50:35.212212+0000","last_undegraded":"2024-03-04T15:50:35.212212+0000","last_fullsized":"2024-03-04T15:50:35.212212+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T21:29:48.184599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":28,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124044+0000","last_change":"2024-03-04T15:49:40.455789+0000","last_active":"2024-03-04T15:50:36.124044+0000","last_peered":"2024-03-04T15:50:36.124044+0000","last_clean":"2024-03-04T15:50:36.124044+0000","last_became_active":"2024-03-04T15:49:40.454939+0000","last_became_peered":"2024-03-04T15:49:40.454939+0000","last_unstale":"2024-03-04T15:50:36.124044+0000","last_undegraded":"2024-03-04T15:50:36.124044+0000","last_fullsized":"2024-03-04T15:50:36.124044+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":66,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:39.167995+0000","last_clean_scrub_stamp":"2024-03-04T15:49:39.167995+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T23:50:23.549656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":40,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.125121+0000","last_change":"2024-03-04T15:49:33.726708+0000","last_active":"2024-03-04T15:50:36.125121+0000","last_peered":"2024-03-04T15:50:36.125121+0000","last_clean":"2024-03-04T15:50:36.125121+0000","last_became_active":"2024-03-04T15:49:33.726158+0000","last_became_peered":"2024-03-04T15:49:33.726158+0000","last_unstale":"2024-03-04T15:50:36.125121+0000","last_undegraded":"2024-03-04T15:50:36.125121+0000","last_fullsized":"2024-03-04T15:50:36.125121+0000","mapping_epoch":60,"log_start":"0'0","ondisk_log_start":"0'0","created":60,"last_epoch_clean":61,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:32.687643+0000","last_clean_scrub_stamp":"2024-03-04T15:49:32.687643+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:58:13.034366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"68'15","reported_seq":61,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.124721+0000","last_change":"2024-03-04T15:49:35.760510+0000","last_active":"2024-03-04T15:50:36.124721+0000","last_peered":"2024-03-04T15:50:36.124721+0000","last_clean":"2024-03-04T15:50:36.124721+0000","last_became_active":"2024-03-04T15:49:35.759479+0000","last_became_peered":"2024-03-04T15:49:35.759479+0000","last_unstale":"2024-03-04T15:50:36.124721+0000","last_undegraded":"2024-03-04T15:50:36.124721+0000","last_fullsized":"2024-03-04T15:50:36.124721+0000","mapping_epoch":62,"log_start":"0'0","ondisk_log_start":"0'0","created":62,"last_epoch_clean":63,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:34.702984+0000","last_clean_scrub_stamp":"2024-03-04T15:49:34.702984+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-06T01:18:40.099689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,0],"acting":[7,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":32,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-03-04T15:50:36.123096+0000","last_change":"2024-03-04T15:49:38.154184+0000","last_active":"2024-03-04T15:50:36.123096+0000","last_peered":"2024-03-04T15:50:36.123096+0000","last_clean":"2024-03-04T15:50:36.123096+0000","last_became_active":"2024-03-04T15:49:38.153478+0000","last_became_peered":"2024-03-04T15:49:38.153478+0000","last_unstale":"2024-03-04T15:50:36.123096+0000","last_undegraded":"2024-03-04T15:50:36.123096+0000","last_fullsized":"2024-03-04T15:50:36.123096+0000","mapping_epoch":64,"log_start":"0'0","ondisk_log_start":"0'0","created":64,"last_epoch_clean":65,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-03-04T15:49:36.959614+0000","last_clean_scrub_stamp":"2024-03-04T15:49:36.959614+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-03-05T22:22:35.525839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":383,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1149,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":154,"num_read_kb":149,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":7,"ondisk_log_size":7,"up":9,"acting":9,"num_store_stats":7},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":88,"ondisk_log_size":88,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":178,"num_object_clones":0,"num_object_copies":534,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":178,"num_whiteouts":0,"num_read":701,"num_read_kb":458,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":395,"ondisk_log_size":395,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":212,"num_read_kb":306,"num_write":179,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1771104,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":128,"ondisk_log_size":128,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1457,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":34,"num_read_kb":34,"num_write":10,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":4371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":96,"acting":96,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":55,"seq":236223201325,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32600,"kb_used_data":1420,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712552,"statfs":{"total":95995035648,"available":95961653248,"internally_reserved":0,"allocated":1454080,"data_stored":948859,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":31914442},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":47,"seq":201863462969,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32572,"kb_used_data":1396,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712580,"statfs":{"total":95995035648,"available":95961681920,"internally_reserved":0,"allocated":1429504,"data_stored":946698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574956,"num_pgs":50,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":828,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":847872,"data_stored":357397,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738486,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32592,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712560,"statfs":{"total":95995035648,"available":95961661440,"internally_reserved":0,"allocated":1445888,"data_stored":947362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411427,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":831488,"data_stored":356758,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182487,"num_pgs":59,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31992,"kb_used_data":812,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713160,"statfs":{"total":95995035648,"available":95962275840,"internally_reserved":0,"allocated":831488,"data_stored":356698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986254,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32032,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713120,"statfs":{"total":95995035648,"available":95962234880,"internally_reserved":0,"allocated":872448,"data_stored":358488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":39,"seq":167503724611,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":31996,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713156,"statfs":{"total":95995035648,"available":95962271744,"internally_reserved":0,"allocated":835584,"data_stored":357542,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":444,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":46,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1365,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1980,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1172,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":1100,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":370,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-03-04T15:52:36.416 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-03-04T15:52:36.416 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-03-04T15:52:36.416 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-03-04T15:52:36.416 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:9ca0b5de35ed230e84387abdd8e619ed8e4f3282 shell --fsid e40b5504-da3c-11ee-95c6-87774f69a715 -- ceph health --format=json 2024-03-04T15:52:36.682 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:52:36 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:52:36.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:36 smithi151 bash[23756]: cluster 2024-03-04T15:52:35.375689+0000 mgr.y (mgr.24428) 108 : cluster 0 pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:52:36.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:36 smithi151 bash[23756]: audit 2024-03-04T15:52:35.526226+0000 mon.c (mon.2) 82 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:52:36.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:36 smithi151 bash[23756]: audit 2024-03-04T15:52:35.752000+0000 mgr.y (mgr.24428) 109 : audit 0 from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T15:52:36.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:36 smithi117 bash[21480]: cluster 2024-03-04T15:52:35.375689+0000 mgr.y (mgr.24428) 108 : cluster 0 pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:52:36.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:36 smithi117 bash[21480]: audit 2024-03-04T15:52:35.526226+0000 mon.c (mon.2) 82 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:52:36.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:36 smithi117 bash[21480]: audit 2024-03-04T15:52:35.752000+0000 mgr.y (mgr.24428) 109 : audit 0 from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T15:52:36.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:36 smithi117 bash[28843]: cluster 2024-03-04T15:52:35.375689+0000 mgr.y (mgr.24428) 108 : cluster 0 pgmap v63: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:52:36.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:36 smithi117 bash[28843]: audit 2024-03-04T15:52:35.526226+0000 mon.c (mon.2) 82 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:52:36.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:36 smithi117 bash[28843]: audit 2024-03-04T15:52:35.752000+0000 mgr.y (mgr.24428) 109 : audit 0 from='client.14718 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-03-04T15:52:37.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:37 smithi117 bash[21480]: audit 2024-03-04T15:52:36.381054+0000 mgr.y (mgr.24428) 110 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:52:37.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:37 smithi117 bash[28843]: audit 2024-03-04T15:52:36.381054+0000 mgr.y (mgr.24428) 110 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:52:37.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:37 smithi151 bash[23756]: audit 2024-03-04T15:52:36.381054+0000 mgr.y (mgr.24428) 110 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:52:38.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:38 smithi117 bash[21480]: cluster 2024-03-04T15:52:37.377104+0000 mgr.y (mgr.24428) 111 : cluster 0 pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:38.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:38 smithi117 bash[28843]: cluster 2024-03-04T15:52:37.377104+0000 mgr.y (mgr.24428) 111 : cluster 0 pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:38 smithi151 bash[23756]: cluster 2024-03-04T15:52:37.377104+0000 mgr.y (mgr.24428) 111 : cluster 0 pgmap v64: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:40.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:40 smithi117 bash[21480]: cluster 2024-03-04T15:52:39.377753+0000 mgr.y (mgr.24428) 112 : cluster 0 pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:40.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:40 smithi117 bash[28843]: cluster 2024-03-04T15:52:39.377753+0000 mgr.y (mgr.24428) 112 : cluster 0 pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:40.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:40 smithi151 bash[23756]: cluster 2024-03-04T15:52:39.377753+0000 mgr.y (mgr.24428) 112 : cluster 0 pgmap v65: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:41.204 INFO:teuthology.orchestra.run.smithi117.stderr:Inferring config /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/mon.a/config 2024-03-04T15:52:42.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:42 smithi117 bash[21480]: cluster 2024-03-04T15:52:41.379135+0000 mgr.y (mgr.24428) 113 : cluster 0 pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:42.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:42 smithi117 bash[28843]: cluster 2024-03-04T15:52:41.379135+0000 mgr.y (mgr.24428) 113 : cluster 0 pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:42.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:42 smithi151 bash[23756]: cluster 2024-03-04T15:52:41.379135+0000 mgr.y (mgr.24428) 113 : cluster 0 pgmap v66: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:43.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff508f59700 1 -- 172.21.15.117:0/2565188581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5041019c0 msgr2=0x7ff50410db40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:43.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff508f59700 1 --2- 172.21.15.117:0/2565188581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5041019c0 0x7ff50410db40 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc00b0d0 tx=0x7ff4fc00b3e0 comp rx=0 tx=0).stop 2024-03-04T15:52:43.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff508f59700 1 -- 172.21.15.117:0/2565188581 shutdown_connections 2024-03-04T15:52:43.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff508f59700 1 --2- 172.21.15.117:0/2565188581 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff5041019c0 0x7ff50410db40 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:43.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff508f59700 1 --2- 172.21.15.117:0/2565188581 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff504106ef0 0x7ff5041072d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:43.747 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff508f59700 1 --2- 172.21.15.117:0/2565188581 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff504101000 0x7ff504101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:43.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff508f59700 1 -- 172.21.15.117:0/2565188581 >> 172.21.15.117:0/2565188581 conn(0x7ff504077a40 msgr2=0x7ff504077e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:43.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff508f59700 1 -- 172.21.15.117:0/2565188581 shutdown_connections 2024-03-04T15:52:43.748 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff508f59700 1 -- 172.21.15.117:0/2565188581 wait complete. 2024-03-04T15:52:43.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff508f59700 1 Processor -- start 2024-03-04T15:52:43.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff508f59700 1 -- start start 2024-03-04T15:52:43.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff508f59700 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff504101000 0x7ff50419d740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:43.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff508f59700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5041019c0 0x7ff50419dc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:43.749 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff508f59700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff504106ef0 0x7ff504197900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:43.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff508f59700 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff5041105b0 con 0x7ff504106ef0 2024-03-04T15:52:43.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff508f59700 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff504110430 con 0x7ff504101000 2024-03-04T15:52:43.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff508f59700 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff504110730 con 0x7ff5041019c0 2024-03-04T15:52:43.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff502d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff504106ef0 0x7ff504197900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:43.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff4f9d9b700 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5041019c0 0x7ff50419dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:43.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff502d9d700 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff504106ef0 0x7ff504197900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:35930/0 (socket says 172.21.15.117:35930) 2024-03-04T15:52:43.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff502d9d700 1 -- 172.21.15.117:0/1415300350 learned_addr learned my addr 172.21.15.117:0/1415300350 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T15:52:43.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff50259c700 1 --2- 172.21.15.117:0/1415300350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff504101000 0x7ff50419d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:43.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff502d9d700 1 -- 172.21.15.117:0/1415300350 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5041019c0 msgr2=0x7ff50419dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:43.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff502d9d700 1 --2- 172.21.15.117:0/1415300350 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5041019c0 0x7ff50419dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:43.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff502d9d700 1 -- 172.21.15.117:0/1415300350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff504101000 msgr2=0x7ff50419d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:43.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff502d9d700 1 --2- 172.21.15.117:0/1415300350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff504101000 0x7ff50419d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:43.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff502d9d700 1 -- 172.21.15.117:0/1415300350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff504198060 con 0x7ff504106ef0 2024-03-04T15:52:43.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff502d9d700 1 --2- 172.21.15.117:0/1415300350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff504106ef0 0x7ff504197900 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc006070 tx=0x7ff4fc007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:43.750 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff4fbfff700 1 -- 172.21.15.117:0/1415300350 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc021020 con 0x7ff504106ef0 2024-03-04T15:52:43.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff4fbfff700 1 -- 172.21.15.117:0/1415300350 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7ff4fc004c50 con 0x7ff504106ef0 2024-03-04T15:52:43.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff4fbfff700 1 -- 172.21.15.117:0/1415300350 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4fc005050 con 0x7ff504106ef0 2024-03-04T15:52:43.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff508f59700 1 -- 172.21.15.117:0/1415300350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff5041982f0 con 0x7ff504106ef0 2024-03-04T15:52:43.751 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff508f59700 1 -- 172.21.15.117:0/1415300350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff5041a4610 con 0x7ff504106ef0 2024-03-04T15:52:43.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.746+0000 7ff508f59700 1 -- 172.21.15.117:0/1415300350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff504102470 con 0x7ff504106ef0 2024-03-04T15:52:43.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.750+0000 7ff4fbfff700 1 -- 172.21.15.117:0/1415300350 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7ff4fc024020 con 0x7ff504106ef0 2024-03-04T15:52:43.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.750+0000 7ff4fbfff700 1 --2- 172.21.15.117:0/1415300350 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7ff4f006d780 0x7ff4f006fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T15:52:43.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.750+0000 7ff4fbfff700 1 -- 172.21.15.117:0/1415300350 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7537+0+0 (secure 0 0 0) 0x7ff4fc00f020 con 0x7ff504106ef0 2024-03-04T15:52:43.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.750+0000 7ff50259c700 1 --2- 172.21.15.117:0/1415300350 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7ff4f006d780 0x7ff4f006fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T15:52:43.752 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.750+0000 7ff50259c700 1 --2- 172.21.15.117:0/1415300350 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7ff4f006d780 0x7ff4f006fc40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4005950 tx=0x7ff4f4005ed0 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T15:52:43.756 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:43.754+0000 7ff4fbfff700 1 -- 172.21.15.117:0/1415300350 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7ff4fc05c310 con 0x7ff504106ef0 2024-03-04T15:52:44.009 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:44.006+0000 7ff508f59700 1 -- 172.21.15.117:0/1415300350 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7ff504198e30 con 0x7ff504106ef0 2024-03-04T15:52:44.010 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:44.006+0000 7ff4fbfff700 1 -- 172.21.15.117:0/1415300350 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7ff4fc05fd60 con 0x7ff504106ef0 2024-03-04T15:52:44.010 INFO:teuthology.orchestra.run.smithi117.stdout: 2024-03-04T15:52:44.010 INFO:teuthology.orchestra.run.smithi117.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-03-04T15:52:44.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:44.010+0000 7ff508f59700 1 -- 172.21.15.117:0/1415300350 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7ff4f006d780 msgr2=0x7ff4f006fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:44.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:44.010+0000 7ff508f59700 1 --2- 172.21.15.117:0/1415300350 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7ff4f006d780 0x7ff4f006fc40 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7ff4f4005950 tx=0x7ff4f4005ed0 comp rx=0 tx=0).stop 2024-03-04T15:52:44.012 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:44.010+0000 7ff508f59700 1 -- 172.21.15.117:0/1415300350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff504106ef0 msgr2=0x7ff504197900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T15:52:44.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:44.010+0000 7ff508f59700 1 --2- 172.21.15.117:0/1415300350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff504106ef0 0x7ff504197900 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff4fc006070 tx=0x7ff4fc007a20 comp rx=0 tx=0).stop 2024-03-04T15:52:44.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:44.010+0000 7ff508f59700 1 -- 172.21.15.117:0/1415300350 shutdown_connections 2024-03-04T15:52:44.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:44.010+0000 7ff508f59700 1 --2- 172.21.15.117:0/1415300350 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7ff4f006d780 0x7ff4f006fc40 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:44.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:44.010+0000 7ff508f59700 1 --2- 172.21.15.117:0/1415300350 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7ff504106ef0 0x7ff504197900 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:44.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:44.010+0000 7ff508f59700 1 --2- 172.21.15.117:0/1415300350 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7ff504101000 0x7ff50419d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:44.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:44.010+0000 7ff508f59700 1 --2- 172.21.15.117:0/1415300350 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7ff5041019c0 0x7ff50419dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T15:52:44.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:44.010+0000 7ff508f59700 1 -- 172.21.15.117:0/1415300350 >> 172.21.15.117:0/1415300350 conn(0x7ff504077a40 msgr2=0x7ff50406a7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T15:52:44.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:44.010+0000 7ff508f59700 1 -- 172.21.15.117:0/1415300350 shutdown_connections 2024-03-04T15:52:44.013 INFO:teuthology.orchestra.run.smithi117.stderr:2024-03-04T15:52:44.010+0000 7ff508f59700 1 -- 172.21.15.117:0/1415300350 wait complete. 2024-03-04T15:52:44.640 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-03-04T15:52:44.641 INFO:tasks.cephadm:Setup complete, yielding 2024-03-04T15:52:44.641 INFO:teuthology.run_tasks:Running task workunit... 2024-03-04T15:52:44.663 INFO:tasks.workunit:Pulling workunits from ref 9ca0b5de35ed230e84387abdd8e619ed8e4f3282 2024-03-04T15:52:44.663 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-03-04T15:52:44.663 DEBUG:teuthology.orchestra.run.smithi117:> stat -- /home/ubuntu/cephtest/mnt.0 2024-03-04T15:52:44.669 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T15:52:44.669 INFO:teuthology.orchestra.run.smithi117.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-03-04T15:52:44.669 DEBUG:teuthology.orchestra.run.smithi117:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-03-04T15:52:44.715 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-03-04T15:52:44.716 DEBUG:teuthology.orchestra.run.smithi117:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-03-04T15:52:44.763 INFO:tasks.workunit:timeout=3h 2024-03-04T15:52:44.763 INFO:tasks.workunit:cleanup=True 2024-03-04T15:52:44.763 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 9ca0b5de35ed230e84387abdd8e619ed8e4f3282 2024-03-04T15:52:44.813 INFO:tasks.workunit.client.0.smithi117.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-03-04T15:52:44.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:44 smithi117 bash[21480]: cluster 2024-03-04T15:52:43.379798+0000 mgr.y (mgr.24428) 114 : cluster 0 pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:44.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:44 smithi117 bash[21480]: audit 2024-03-04T15:52:44.012465+0000 mon.a (mon.0) 843 : audit 0 from='client.? 172.21.15.117:0/1415300350' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-04T15:52:44.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:44 smithi117 bash[28843]: cluster 2024-03-04T15:52:43.379798+0000 mgr.y (mgr.24428) 114 : cluster 0 pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:44.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:44 smithi117 bash[28843]: audit 2024-03-04T15:52:44.012465+0000 mon.a (mon.0) 843 : audit 0 from='client.? 172.21.15.117:0/1415300350' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-04T15:52:44.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:44 smithi151 bash[23756]: cluster 2024-03-04T15:52:43.379798+0000 mgr.y (mgr.24428) 114 : cluster 0 pgmap v67: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:44.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:44 smithi151 bash[23756]: audit 2024-03-04T15:52:44.012465+0000 mon.a (mon.0) 843 : audit 0 from='client.? 172.21.15.117:0/1415300350' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-03-04T15:52:46.462 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:52:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:52:46.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:46 smithi151 bash[23756]: cluster 2024-03-04T15:52:45.380497+0000 mgr.y (mgr.24428) 115 : cluster 0 pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:46.682 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:52:46 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:52:46.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:46 smithi117 bash[21480]: cluster 2024-03-04T15:52:45.380497+0000 mgr.y (mgr.24428) 115 : cluster 0 pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:46.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:46 smithi117 bash[28843]: cluster 2024-03-04T15:52:45.380497+0000 mgr.y (mgr.24428) 115 : cluster 0 pgmap v68: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:47.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:47 smithi117 bash[21480]: audit 2024-03-04T15:52:46.385807+0000 mgr.y (mgr.24428) 116 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:52:47.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:47 smithi117 bash[28843]: audit 2024-03-04T15:52:46.385807+0000 mgr.y (mgr.24428) 116 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:52:47.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:47 smithi151 bash[23756]: audit 2024-03-04T15:52:46.385807+0000 mgr.y (mgr.24428) 116 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:52:48.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:48 smithi117 bash[21480]: cluster 2024-03-04T15:52:47.381971+0000 mgr.y (mgr.24428) 117 : cluster 0 pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:48.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:48 smithi117 bash[28843]: cluster 2024-03-04T15:52:47.381971+0000 mgr.y (mgr.24428) 117 : cluster 0 pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:48.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:48 smithi151 bash[23756]: cluster 2024-03-04T15:52:47.381971+0000 mgr.y (mgr.24428) 117 : cluster 0 pgmap v69: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:50.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:50 smithi117 bash[21480]: cluster 2024-03-04T15:52:49.382610+0000 mgr.y (mgr.24428) 118 : cluster 0 pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:50.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:50 smithi117 bash[28843]: cluster 2024-03-04T15:52:49.382610+0000 mgr.y (mgr.24428) 118 : cluster 0 pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:50 smithi151 bash[23756]: cluster 2024-03-04T15:52:49.382610+0000 mgr.y (mgr.24428) 118 : cluster 0 pgmap v70: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:51.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:51 smithi117 bash[28843]: audit 2024-03-04T15:52:50.543475+0000 mon.c (mon.2) 83 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:52:51.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:51 smithi117 bash[21480]: audit 2024-03-04T15:52:50.543475+0000 mon.c (mon.2) 83 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:52:51.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:51 smithi151 bash[23756]: audit 2024-03-04T15:52:50.543475+0000 mon.c (mon.2) 83 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:52:52.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:52 smithi117 bash[21480]: cluster 2024-03-04T15:52:51.383998+0000 mgr.y (mgr.24428) 119 : cluster 0 pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:52.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:52 smithi117 bash[28843]: cluster 2024-03-04T15:52:51.383998+0000 mgr.y (mgr.24428) 119 : cluster 0 pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:52.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:52 smithi151 bash[23756]: cluster 2024-03-04T15:52:51.383998+0000 mgr.y (mgr.24428) 119 : cluster 0 pgmap v71: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:54.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:54 smithi117 bash[21480]: cluster 2024-03-04T15:52:53.384692+0000 mgr.y (mgr.24428) 120 : cluster 0 pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:54.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:54 smithi117 bash[28843]: cluster 2024-03-04T15:52:53.384692+0000 mgr.y (mgr.24428) 120 : cluster 0 pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:54.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:54 smithi151 bash[23756]: cluster 2024-03-04T15:52:53.384692+0000 mgr.y (mgr.24428) 120 : cluster 0 pgmap v72: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:56.496 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:52:56 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:52:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:52:56.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:56 smithi151 bash[23756]: cluster 2024-03-04T15:52:55.385424+0000 mgr.y (mgr.24428) 121 : cluster 0 pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:56.682 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:52:56 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:52:56.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:56 smithi117 bash[21480]: cluster 2024-03-04T15:52:55.385424+0000 mgr.y (mgr.24428) 121 : cluster 0 pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:56.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:56 smithi117 bash[28843]: cluster 2024-03-04T15:52:55.385424+0000 mgr.y (mgr.24428) 121 : cluster 0 pgmap v73: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:52:57.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:57 smithi117 bash[21480]: audit 2024-03-04T15:52:56.397419+0000 mgr.y (mgr.24428) 122 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:52:57.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:57 smithi117 bash[28843]: audit 2024-03-04T15:52:56.397419+0000 mgr.y (mgr.24428) 122 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:52:57.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:57 smithi151 bash[23756]: audit 2024-03-04T15:52:56.397419+0000 mgr.y (mgr.24428) 122 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:52:58.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:52:58 smithi117 bash[21480]: cluster 2024-03-04T15:52:57.386829+0000 mgr.y (mgr.24428) 123 : cluster 0 pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:58.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:52:58 smithi117 bash[28843]: cluster 2024-03-04T15:52:57.386829+0000 mgr.y (mgr.24428) 123 : cluster 0 pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:52:58.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:52:58 smithi151 bash[23756]: cluster 2024-03-04T15:52:57.386829+0000 mgr.y (mgr.24428) 123 : cluster 0 pgmap v74: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:00.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:00 smithi117 bash[21480]: cluster 2024-03-04T15:52:59.387610+0000 mgr.y (mgr.24428) 124 : cluster 0 pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:00.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:00 smithi117 bash[28843]: cluster 2024-03-04T15:52:59.387610+0000 mgr.y (mgr.24428) 124 : cluster 0 pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:00 smithi151 bash[23756]: cluster 2024-03-04T15:52:59.387610+0000 mgr.y (mgr.24428) 124 : cluster 0 pgmap v75: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:02.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:02 smithi117 bash[21480]: cluster 2024-03-04T15:53:01.389188+0000 mgr.y (mgr.24428) 125 : cluster 0 pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:02.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:02 smithi117 bash[28843]: cluster 2024-03-04T15:53:01.389188+0000 mgr.y (mgr.24428) 125 : cluster 0 pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:02.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:02 smithi151 bash[23756]: cluster 2024-03-04T15:53:01.389188+0000 mgr.y (mgr.24428) 125 : cluster 0 pgmap v76: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:04.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:04 smithi117 bash[21480]: cluster 2024-03-04T15:53:03.389857+0000 mgr.y (mgr.24428) 126 : cluster 0 pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:04.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:04 smithi117 bash[28843]: cluster 2024-03-04T15:53:03.389857+0000 mgr.y (mgr.24428) 126 : cluster 0 pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:04.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:04 smithi151 bash[23756]: cluster 2024-03-04T15:53:03.389857+0000 mgr.y (mgr.24428) 126 : cluster 0 pgmap v77: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:06.537 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:53:06 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:53:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:53:06.682 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:53:06 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:53:06.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:06 smithi151 bash[23756]: cluster 2024-03-04T15:53:05.390521+0000 mgr.y (mgr.24428) 127 : cluster 0 pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:06.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:06 smithi151 bash[23756]: audit 2024-03-04T15:53:05.573557+0000 mon.c (mon.2) 84 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:53:06.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:06 smithi117 bash[21480]: cluster 2024-03-04T15:53:05.390521+0000 mgr.y (mgr.24428) 127 : cluster 0 pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:06.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:06 smithi117 bash[21480]: audit 2024-03-04T15:53:05.573557+0000 mon.c (mon.2) 84 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:53:06.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:06 smithi117 bash[28843]: cluster 2024-03-04T15:53:05.390521+0000 mgr.y (mgr.24428) 127 : cluster 0 pgmap v78: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:06.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:06 smithi117 bash[28843]: audit 2024-03-04T15:53:05.573557+0000 mon.c (mon.2) 84 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:53:07.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:07 smithi117 bash[21480]: audit 2024-03-04T15:53:06.402051+0000 mgr.y (mgr.24428) 128 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:53:07.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:07 smithi117 bash[28843]: audit 2024-03-04T15:53:06.402051+0000 mgr.y (mgr.24428) 128 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:53:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:07 smithi151 bash[23756]: audit 2024-03-04T15:53:06.402051+0000 mgr.y (mgr.24428) 128 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:53:08.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:08 smithi117 bash[28843]: cluster 2024-03-04T15:53:07.391996+0000 mgr.y (mgr.24428) 129 : cluster 0 pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:08.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:08 smithi117 bash[21480]: cluster 2024-03-04T15:53:07.391996+0000 mgr.y (mgr.24428) 129 : cluster 0 pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:08.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:08 smithi151 bash[23756]: cluster 2024-03-04T15:53:07.391996+0000 mgr.y (mgr.24428) 129 : cluster 0 pgmap v79: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:10.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:10 smithi117 bash[21480]: cluster 2024-03-04T15:53:09.392661+0000 mgr.y (mgr.24428) 130 : cluster 0 pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:10.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:10 smithi117 bash[28843]: cluster 2024-03-04T15:53:09.392661+0000 mgr.y (mgr.24428) 130 : cluster 0 pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:10.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:10 smithi151 bash[23756]: cluster 2024-03-04T15:53:09.392661+0000 mgr.y (mgr.24428) 130 : cluster 0 pgmap v80: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:12.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:12 smithi117 bash[21480]: cluster 2024-03-04T15:53:11.394109+0000 mgr.y (mgr.24428) 131 : cluster 0 pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:12.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:12 smithi117 bash[28843]: cluster 2024-03-04T15:53:11.394109+0000 mgr.y (mgr.24428) 131 : cluster 0 pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:12 smithi151 bash[23756]: cluster 2024-03-04T15:53:11.394109+0000 mgr.y (mgr.24428) 131 : cluster 0 pgmap v81: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:14.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:14 smithi117 bash[21480]: cluster 2024-03-04T15:53:13.394824+0000 mgr.y (mgr.24428) 132 : cluster 0 pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:14.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:14 smithi117 bash[28843]: cluster 2024-03-04T15:53:13.394824+0000 mgr.y (mgr.24428) 132 : cluster 0 pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:14.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:14 smithi151 bash[23756]: cluster 2024-03-04T15:53:13.394824+0000 mgr.y (mgr.24428) 132 : cluster 0 pgmap v82: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:16.565 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:53:16 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:53:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:53:16.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:16 smithi151 bash[23756]: cluster 2024-03-04T15:53:15.395507+0000 mgr.y (mgr.24428) 133 : cluster 0 pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:16.682 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:53:16 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:53:16.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:16 smithi117 bash[28843]: cluster 2024-03-04T15:53:15.395507+0000 mgr.y (mgr.24428) 133 : cluster 0 pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:16.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:16 smithi117 bash[21480]: cluster 2024-03-04T15:53:15.395507+0000 mgr.y (mgr.24428) 133 : cluster 0 pgmap v83: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:17.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:17 smithi151 bash[23756]: audit 2024-03-04T15:53:16.413527+0000 mgr.y (mgr.24428) 134 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:53:18.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:17 smithi117 bash[21480]: audit 2024-03-04T15:53:16.413527+0000 mgr.y (mgr.24428) 134 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:53:18.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:17 smithi117 bash[28843]: audit 2024-03-04T15:53:16.413527+0000 mgr.y (mgr.24428) 134 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:53:18.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:18 smithi151 bash[23756]: cluster 2024-03-04T15:53:17.396940+0000 mgr.y (mgr.24428) 135 : cluster 0 pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:19.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:18 smithi117 bash[28843]: cluster 2024-03-04T15:53:17.396940+0000 mgr.y (mgr.24428) 135 : cluster 0 pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:19.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:18 smithi117 bash[21480]: cluster 2024-03-04T15:53:17.396940+0000 mgr.y (mgr.24428) 135 : cluster 0 pgmap v84: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:20.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:20 smithi151 bash[23756]: cluster 2024-03-04T15:53:19.397706+0000 mgr.y (mgr.24428) 136 : cluster 0 pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:21.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:20 smithi117 bash[21480]: cluster 2024-03-04T15:53:19.397706+0000 mgr.y (mgr.24428) 136 : cluster 0 pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:21.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:20 smithi117 bash[28843]: cluster 2024-03-04T15:53:19.397706+0000 mgr.y (mgr.24428) 136 : cluster 0 pgmap v85: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:21.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:21 smithi151 bash[23756]: audit 2024-03-04T15:53:20.604895+0000 mon.c (mon.2) 85 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:53:22.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:21 smithi117 bash[21480]: audit 2024-03-04T15:53:20.604895+0000 mon.c (mon.2) 85 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:53:22.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:21 smithi117 bash[28843]: audit 2024-03-04T15:53:20.604895+0000 mon.c (mon.2) 85 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:53:22.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:22 smithi151 bash[23756]: cluster 2024-03-04T15:53:21.399120+0000 mgr.y (mgr.24428) 137 : cluster 0 pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:23.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:22 smithi117 bash[21480]: cluster 2024-03-04T15:53:21.399120+0000 mgr.y (mgr.24428) 137 : cluster 0 pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:23.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:22 smithi117 bash[28843]: cluster 2024-03-04T15:53:21.399120+0000 mgr.y (mgr.24428) 137 : cluster 0 pgmap v86: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:24.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:24 smithi151 bash[23756]: cluster 2024-03-04T15:53:23.399713+0000 mgr.y (mgr.24428) 138 : cluster 0 pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:25.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:24 smithi117 bash[21480]: cluster 2024-03-04T15:53:23.399713+0000 mgr.y (mgr.24428) 138 : cluster 0 pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:25.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:24 smithi117 bash[28843]: cluster 2024-03-04T15:53:23.399713+0000 mgr.y (mgr.24428) 138 : cluster 0 pgmap v87: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:26.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:53:26 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:53:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:53:26.682 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:53:26 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:53:26.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:26 smithi151 bash[23756]: cluster 2024-03-04T15:53:25.400472+0000 mgr.y (mgr.24428) 139 : cluster 0 pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:27.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:26 smithi117 bash[21480]: cluster 2024-03-04T15:53:25.400472+0000 mgr.y (mgr.24428) 139 : cluster 0 pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:27.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:26 smithi117 bash[28843]: cluster 2024-03-04T15:53:25.400472+0000 mgr.y (mgr.24428) 139 : cluster 0 pgmap v88: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:27.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:27 smithi151 bash[23756]: audit 2024-03-04T15:53:26.425086+0000 mgr.y (mgr.24428) 140 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:53:28.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:27 smithi117 bash[21480]: audit 2024-03-04T15:53:26.425086+0000 mgr.y (mgr.24428) 140 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:53:28.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:27 smithi117 bash[28843]: audit 2024-03-04T15:53:26.425086+0000 mgr.y (mgr.24428) 140 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:53:28.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:28 smithi151 bash[23756]: cluster 2024-03-04T15:53:27.401864+0000 mgr.y (mgr.24428) 141 : cluster 0 pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:29.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:28 smithi117 bash[21480]: cluster 2024-03-04T15:53:27.401864+0000 mgr.y (mgr.24428) 141 : cluster 0 pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:29.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:28 smithi117 bash[28843]: cluster 2024-03-04T15:53:27.401864+0000 mgr.y (mgr.24428) 141 : cluster 0 pgmap v89: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:30.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:30 smithi151 bash[23756]: cluster 2024-03-04T15:53:29.402494+0000 mgr.y (mgr.24428) 142 : cluster 0 pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:31.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:30 smithi117 bash[21480]: cluster 2024-03-04T15:53:29.402494+0000 mgr.y (mgr.24428) 142 : cluster 0 pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:31.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:30 smithi117 bash[28843]: cluster 2024-03-04T15:53:29.402494+0000 mgr.y (mgr.24428) 142 : cluster 0 pgmap v90: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:32.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:32 smithi151 bash[23756]: cluster 2024-03-04T15:53:31.403801+0000 mgr.y (mgr.24428) 143 : cluster 0 pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:32.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:32 smithi151 bash[23756]: audit 2024-03-04T15:53:32.195099+0000 mon.c (mon.2) 86 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:53:33.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:32 smithi117 bash[28843]: cluster 2024-03-04T15:53:31.403801+0000 mgr.y (mgr.24428) 143 : cluster 0 pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:33.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:32 smithi117 bash[28843]: audit 2024-03-04T15:53:32.195099+0000 mon.c (mon.2) 86 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:53:33.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:32 smithi117 bash[21480]: cluster 2024-03-04T15:53:31.403801+0000 mgr.y (mgr.24428) 143 : cluster 0 pgmap v91: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:33.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:32 smithi117 bash[21480]: audit 2024-03-04T15:53:32.195099+0000 mon.c (mon.2) 86 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:53:33.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:33 smithi151 bash[23756]: audit 2024-03-04T15:53:32.921999+0000 mon.c (mon.2) 87 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:53:33.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:33 smithi151 bash[23756]: audit 2024-03-04T15:53:32.923013+0000 mon.c (mon.2) 88 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:53:33.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:33 smithi151 bash[23756]: audit 2024-03-04T15:53:32.928556+0000 mon.a (mon.0) 844 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:53:34.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:33 smithi117 bash[21480]: audit 2024-03-04T15:53:32.921999+0000 mon.c (mon.2) 87 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:53:34.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:33 smithi117 bash[21480]: audit 2024-03-04T15:53:32.923013+0000 mon.c (mon.2) 88 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:53:34.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:33 smithi117 bash[21480]: audit 2024-03-04T15:53:32.928556+0000 mon.a (mon.0) 844 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:53:34.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:33 smithi117 bash[28843]: audit 2024-03-04T15:53:32.921999+0000 mon.c (mon.2) 87 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:53:34.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:33 smithi117 bash[28843]: audit 2024-03-04T15:53:32.923013+0000 mon.c (mon.2) 88 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:53:34.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:33 smithi117 bash[28843]: audit 2024-03-04T15:53:32.928556+0000 mon.a (mon.0) 844 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:53:35.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:34 smithi151 bash[23756]: cluster 2024-03-04T15:53:33.404163+0000 mgr.y (mgr.24428) 144 : cluster 0 pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:35.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:34 smithi117 bash[21480]: cluster 2024-03-04T15:53:33.404163+0000 mgr.y (mgr.24428) 144 : cluster 0 pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:35.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:34 smithi117 bash[28843]: cluster 2024-03-04T15:53:33.404163+0000 mgr.y (mgr.24428) 144 : cluster 0 pgmap v92: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:36.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:35 smithi117 bash[21480]: audit 2024-03-04T15:53:35.634533+0000 mon.c (mon.2) 89 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:53:36.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:35 smithi117 bash[28843]: audit 2024-03-04T15:53:35.634533+0000 mon.c (mon.2) 89 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:53:36.320 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:53:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:53:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:53:36.429 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:35 smithi151 bash[23756]: audit 2024-03-04T15:53:35.634533+0000 mon.c (mon.2) 89 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:53:36.682 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:53:36 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:53:37.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:36 smithi117 bash[28843]: cluster 2024-03-04T15:53:35.404515+0000 mgr.y (mgr.24428) 145 : cluster 0 pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:37.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:36 smithi117 bash[21480]: cluster 2024-03-04T15:53:35.404515+0000 mgr.y (mgr.24428) 145 : cluster 0 pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:37.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:36 smithi151 bash[23756]: cluster 2024-03-04T15:53:35.404515+0000 mgr.y (mgr.24428) 145 : cluster 0 pgmap v93: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:38.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:37 smithi117 bash[21480]: audit 2024-03-04T15:53:36.430244+0000 mgr.y (mgr.24428) 146 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:53:38.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:37 smithi117 bash[28843]: audit 2024-03-04T15:53:36.430244+0000 mgr.y (mgr.24428) 146 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:53:38.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:37 smithi151 bash[23756]: audit 2024-03-04T15:53:36.430244+0000 mgr.y (mgr.24428) 146 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:53:39.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:38 smithi117 bash[28843]: cluster 2024-03-04T15:53:37.405299+0000 mgr.y (mgr.24428) 147 : cluster 0 pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:39.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:38 smithi117 bash[21480]: cluster 2024-03-04T15:53:37.405299+0000 mgr.y (mgr.24428) 147 : cluster 0 pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:39.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:38 smithi151 bash[23756]: cluster 2024-03-04T15:53:37.405299+0000 mgr.y (mgr.24428) 147 : cluster 0 pgmap v94: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:41.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:40 smithi117 bash[28843]: cluster 2024-03-04T15:53:39.405695+0000 mgr.y (mgr.24428) 148 : cluster 0 pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:41.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:40 smithi117 bash[21480]: cluster 2024-03-04T15:53:39.405695+0000 mgr.y (mgr.24428) 148 : cluster 0 pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:41.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:40 smithi151 bash[23756]: cluster 2024-03-04T15:53:39.405695+0000 mgr.y (mgr.24428) 148 : cluster 0 pgmap v95: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T15:53:43.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:42 smithi117 bash[21480]: cluster 2024-03-04T15:53:41.406471+0000 mgr.y (mgr.24428) 149 : cluster 0 pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:43.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:42 smithi117 bash[28843]: cluster 2024-03-04T15:53:41.406471+0000 mgr.y (mgr.24428) 149 : cluster 0 pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:43.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:42 smithi151 bash[23756]: cluster 2024-03-04T15:53:41.406471+0000 mgr.y (mgr.24428) 149 : cluster 0 pgmap v96: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:45.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:44 smithi117 bash[21480]: cluster 2024-03-04T15:53:43.406864+0000 mgr.y (mgr.24428) 150 : cluster 0 pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:53:45.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:44 smithi117 bash[28843]: cluster 2024-03-04T15:53:43.406864+0000 mgr.y (mgr.24428) 150 : cluster 0 pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:53:45.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:44 smithi151 bash[23756]: cluster 2024-03-04T15:53:43.406864+0000 mgr.y (mgr.24428) 150 : cluster 0 pgmap v97: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:53:46.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:53:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:53:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:53:46.932 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:53:46 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:53:47.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:46 smithi117 bash[28843]: cluster 2024-03-04T15:53:45.407258+0000 mgr.y (mgr.24428) 151 : cluster 0 pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:53:47.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:46 smithi117 bash[21480]: cluster 2024-03-04T15:53:45.407258+0000 mgr.y (mgr.24428) 151 : cluster 0 pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:53:47.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:46 smithi151 bash[23756]: cluster 2024-03-04T15:53:45.407258+0000 mgr.y (mgr.24428) 151 : cluster 0 pgmap v98: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:53:48.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:47 smithi117 bash[21480]: audit 2024-03-04T15:53:46.434192+0000 mgr.y (mgr.24428) 152 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:53:48.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:47 smithi117 bash[28843]: audit 2024-03-04T15:53:46.434192+0000 mgr.y (mgr.24428) 152 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:53:48.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:47 smithi151 bash[23756]: audit 2024-03-04T15:53:46.434192+0000 mgr.y (mgr.24428) 152 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:53:49.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:48 smithi117 bash[21480]: cluster 2024-03-04T15:53:47.408042+0000 mgr.y (mgr.24428) 153 : cluster 0 pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:49.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:48 smithi117 bash[28843]: cluster 2024-03-04T15:53:47.408042+0000 mgr.y (mgr.24428) 153 : cluster 0 pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:49.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:48 smithi151 bash[23756]: cluster 2024-03-04T15:53:47.408042+0000 mgr.y (mgr.24428) 153 : cluster 0 pgmap v99: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:51.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:50 smithi117 bash[21480]: cluster 2024-03-04T15:53:49.408450+0000 mgr.y (mgr.24428) 154 : cluster 0 pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:53:51.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:50 smithi117 bash[21480]: audit 2024-03-04T15:53:50.651144+0000 mon.c (mon.2) 90 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:53:51.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:50 smithi117 bash[28843]: cluster 2024-03-04T15:53:49.408450+0000 mgr.y (mgr.24428) 154 : cluster 0 pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:53:51.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:50 smithi117 bash[28843]: audit 2024-03-04T15:53:50.651144+0000 mon.c (mon.2) 90 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:53:51.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:50 smithi151 bash[23756]: cluster 2024-03-04T15:53:49.408450+0000 mgr.y (mgr.24428) 154 : cluster 0 pgmap v100: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:53:51.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:50 smithi151 bash[23756]: audit 2024-03-04T15:53:50.651144+0000 mon.c (mon.2) 90 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:53:53.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:52 smithi117 bash[21480]: cluster 2024-03-04T15:53:51.409239+0000 mgr.y (mgr.24428) 155 : cluster 0 pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:53.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:52 smithi117 bash[28843]: cluster 2024-03-04T15:53:51.409239+0000 mgr.y (mgr.24428) 155 : cluster 0 pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:53.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:52 smithi151 bash[23756]: cluster 2024-03-04T15:53:51.409239+0000 mgr.y (mgr.24428) 155 : cluster 0 pgmap v101: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:55.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:54 smithi117 bash[21480]: cluster 2024-03-04T15:53:53.409659+0000 mgr.y (mgr.24428) 156 : cluster 0 pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:53:55.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:55 smithi117 bash[28843]: cluster 2024-03-04T15:53:53.409659+0000 mgr.y (mgr.24428) 156 : cluster 0 pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:53:55.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:54 smithi151 bash[23756]: cluster 2024-03-04T15:53:53.409659+0000 mgr.y (mgr.24428) 156 : cluster 0 pgmap v102: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:53:56.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:53:56 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:53:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:53:56.932 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:53:56 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:53:57.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:56 smithi117 bash[21480]: cluster 2024-03-04T15:53:55.410063+0000 mgr.y (mgr.24428) 157 : cluster 0 pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:53:57.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:56 smithi117 bash[28843]: cluster 2024-03-04T15:53:55.410063+0000 mgr.y (mgr.24428) 157 : cluster 0 pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:53:57.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:56 smithi151 bash[23756]: cluster 2024-03-04T15:53:55.410063+0000 mgr.y (mgr.24428) 157 : cluster 0 pgmap v103: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:53:58.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:57 smithi117 bash[28843]: audit 2024-03-04T15:53:56.445532+0000 mgr.y (mgr.24428) 158 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:53:58.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:57 smithi117 bash[21480]: audit 2024-03-04T15:53:56.445532+0000 mgr.y (mgr.24428) 158 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:53:58.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:57 smithi151 bash[23756]: audit 2024-03-04T15:53:56.445532+0000 mgr.y (mgr.24428) 158 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:53:58.744 INFO:tasks.workunit.client.0.smithi117.stderr:Note: switching to '9ca0b5de35ed230e84387abdd8e619ed8e4f3282'. 2024-03-04T15:53:58.744 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-03-04T15:53:58.744 INFO:tasks.workunit.client.0.smithi117.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-03-04T15:53:58.744 INFO:tasks.workunit.client.0.smithi117.stderr:changes and commit them, and you can discard any commits you make in this 2024-03-04T15:53:58.744 INFO:tasks.workunit.client.0.smithi117.stderr:state without impacting any branches by switching back to a branch. 2024-03-04T15:53:58.744 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-03-04T15:53:58.744 INFO:tasks.workunit.client.0.smithi117.stderr:If you want to create a new branch to retain commits you create, you may 2024-03-04T15:53:58.744 INFO:tasks.workunit.client.0.smithi117.stderr:do so (now or later) by using -c with the switch command. Example: 2024-03-04T15:53:58.745 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-03-04T15:53:58.745 INFO:tasks.workunit.client.0.smithi117.stderr: git switch -c 2024-03-04T15:53:58.745 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-03-04T15:53:58.745 INFO:tasks.workunit.client.0.smithi117.stderr:Or undo this operation with: 2024-03-04T15:53:58.745 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-03-04T15:53:58.745 INFO:tasks.workunit.client.0.smithi117.stderr: git switch - 2024-03-04T15:53:58.745 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-03-04T15:53:58.745 INFO:tasks.workunit.client.0.smithi117.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-03-04T15:53:58.745 INFO:tasks.workunit.client.0.smithi117.stderr: 2024-03-04T15:53:58.745 INFO:tasks.workunit.client.0.smithi117.stderr:HEAD is now at 9ca0b5de35e Merge branch 'bz2068026-nfs-idmap' of https://github.com/asm0deuz/ceph into wip-asm0deuz-testing-2024-03-04-1211 2024-03-04T15:53:58.748 DEBUG:teuthology.orchestra.run.smithi117:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-03-04T15:53:58.796 INFO:tasks.workunit.client.0.smithi117.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-03-04T15:53:58.800 INFO:tasks.workunit.client.0.smithi117.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T15:53:58.800 INFO:tasks.workunit.client.0.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-03-04T15:53:59.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:53:58 smithi117 bash[21480]: cluster 2024-03-04T15:53:57.411425+0000 mgr.y (mgr.24428) 159 : cluster 0 pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:59.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:53:58 smithi117 bash[28843]: cluster 2024-03-04T15:53:57.411425+0000 mgr.y (mgr.24428) 159 : cluster 0 pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:59.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:53:58 smithi151 bash[23756]: cluster 2024-03-04T15:53:57.411425+0000 mgr.y (mgr.24428) 159 : cluster 0 pgmap v104: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:53:59.561 INFO:tasks.workunit.client.0.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-03-04T15:53:59.644 INFO:tasks.workunit.client.0.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-03-04T15:53:59.701 INFO:tasks.workunit.client.0.smithi117.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-03-04T15:53:59.703 INFO:tasks.workunit.client.0.smithi117.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T15:53:59.703 INFO:tasks.workunit.client.0.smithi117.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-03-04T15:53:59.749 INFO:tasks.workunit.client.0.smithi117.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-03-04T15:53:59.754 DEBUG:teuthology.orchestra.run.smithi117:> set -ex 2024-03-04T15:53:59.754 DEBUG:teuthology.orchestra.run.smithi117:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-03-04T15:53:59.806 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-03-04T15:53:59.808 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-03-04T15:53:59.808 DEBUG:teuthology.orchestra.run.smithi117:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9ca0b5de35ed230e84387abdd8e619ed8e4f3282 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-03-04T15:53:59.861 INFO:tasks.workunit.client.0.smithi117.stderr:+ parallel=1 2024-03-04T15:53:59.861 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' '' = --serial ']' 2024-03-04T15:53:59.861 INFO:tasks.workunit.client.0.smithi117.stderr:+ crimson=0 2024-03-04T15:53:59.861 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' '' = --crimson ']' 2024-03-04T15:53:59.861 INFO:tasks.workunit.client.0.smithi117.stderr:+ color= 2024-03-04T15:53:59.861 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -t 1 ']' 2024-03-04T15:53:59.862 INFO:tasks.workunit.client.0.smithi117.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-03-04T15:53:59.862 INFO:tasks.workunit.client.0.smithi117.stderr:+ GTEST_OUTPUT_DIR=/home/ubuntu/cephtest/archive/unit_test_xml_report 2024-03-04T15:53:59.862 INFO:tasks.workunit.client.0.smithi117.stderr:+ mkdir -p /home/ubuntu/cephtest/archive/unit_test_xml_report 2024-03-04T15:53:59.862 INFO:tasks.workunit.client.0.smithi117.stderr:+ declare -A pids 2024-03-04T15:53:59.863 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:53:59.863 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:53:59.863 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_aio 2024-03-04T15:53:59.863 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_aio' 2024-03-04T15:53:59.864 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_aio 2024-03-04T15:53:59.864 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:53:59.865 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_aio 2024-03-04T15:53:59.866 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=63910 2024-03-04T15:53:59.866 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_aio on pid 63910' 2024-03-04T15:53:59.866 INFO:tasks.workunit.client.0.smithi117.stdout:test api_aio on pid 63910 2024-03-04T15:53:59.867 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=63910 2024-03-04T15:53:59.867 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:53:59.867 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:53:59.867 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-03-04T15:53:59.867 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_aio_pp 2024-03-04T15:53:59.867 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_aio_pp' 2024-03-04T15:53:59.867 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_aio_pp 2024-03-04T15:53:59.867 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:53:59.867 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:53:59.868 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.868 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:53:59.868 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.868 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_aio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio.xml 2024-03-04T15:53:59.869 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_aio.log 2024-03-04T15:53:59.869 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_aio: /' 2024-03-04T15:53:59.869 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_aio_pp 2024-03-04T15:53:59.869 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=63918 2024-03-04T15:53:59.869 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_aio_pp on pid 63918' 2024-03-04T15:53:59.869 INFO:tasks.workunit.client.0.smithi117.stdout:test api_aio_pp on pid 63918 2024-03-04T15:53:59.870 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=63918 2024-03-04T15:53:59.871 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:53:59.871 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:53:59.871 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-03-04T15:53:59.871 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_io 2024-03-04T15:53:59.871 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_io' 2024-03-04T15:53:59.871 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_io 2024-03-04T15:53:59.871 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:53:59.871 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:53:59.871 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.871 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:53:59.871 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.871 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_aio_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_aio_pp.xml 2024-03-04T15:53:59.871 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-03-04T15:53:59.871 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_aio_pp: /' 2024-03-04T15:53:59.872 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_io 2024-03-04T15:53:59.872 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=63926 2024-03-04T15:53:59.872 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_io on pid 63926' 2024-03-04T15:53:59.872 INFO:tasks.workunit.client.0.smithi117.stdout:test api_io on pid 63926 2024-03-04T15:53:59.874 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=63926 2024-03-04T15:53:59.874 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:53:59.874 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:53:59.874 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-03-04T15:53:59.874 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_io_pp 2024-03-04T15:53:59.874 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_io_pp' 2024-03-04T15:53:59.875 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_io_pp 2024-03-04T15:53:59.875 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:53:59.875 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:53:59.875 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.875 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:53:59.875 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.875 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_io --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io.xml 2024-03-04T15:53:59.875 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_io.log 2024-03-04T15:53:59.875 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_io: /' 2024-03-04T15:53:59.875 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_io_pp 2024-03-04T15:53:59.875 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=63934 2024-03-04T15:53:59.875 INFO:tasks.workunit.client.0.smithi117.stdout:test api_io_pp on pid 63934 2024-03-04T15:53:59.877 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_io_pp on pid 63934' 2024-03-04T15:53:59.877 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-03-04T15:53:59.877 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=63934 2024-03-04T15:53:59.877 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:53:59.877 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:53:59.877 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_asio 2024-03-04T15:53:59.877 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_asio' 2024-03-04T15:53:59.877 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:53:59.877 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.877 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:53:59.877 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.878 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_io_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_io_pp.xml 2024-03-04T15:53:59.878 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_asio 2024-03-04T15:53:59.878 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-03-04T15:53:59.878 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:53:59.878 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_io_pp: /' 2024-03-04T15:53:59.878 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_asio 2024-03-04T15:53:59.878 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=63942 2024-03-04T15:53:59.879 INFO:tasks.workunit.client.0.smithi117.stdout:test api_asio on pid 63942 2024-03-04T15:53:59.880 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_asio on pid 63942' 2024-03-04T15:53:59.880 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=63942 2024-03-04T15:53:59.880 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:53:59.881 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-03-04T15:53:59.881 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:53:59.881 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_list 2024-03-04T15:53:59.881 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_list' 2024-03-04T15:53:59.881 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_list 2024-03-04T15:53:59.881 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:53:59.881 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:53:59.881 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.881 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:53:59.881 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.881 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_asio --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_asio.xml 2024-03-04T15:53:59.882 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_asio.log 2024-03-04T15:53:59.882 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_asio: /' 2024-03-04T15:53:59.882 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_list 2024-03-04T15:53:59.883 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=63950 2024-03-04T15:53:59.883 INFO:tasks.workunit.client.0.smithi117.stdout:test api_list on pid 63950 2024-03-04T15:53:59.884 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_list on pid 63950' 2024-03-04T15:53:59.884 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=63950 2024-03-04T15:53:59.884 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:53:59.884 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:53:59.884 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-03-04T15:53:59.884 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_lock 2024-03-04T15:53:59.884 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_lock' 2024-03-04T15:53:59.885 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_lock 2024-03-04T15:53:59.885 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:53:59.885 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:53:59.885 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.885 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:53:59.885 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.885 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_list.log 2024-03-04T15:53:59.887 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_list --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_list.xml 2024-03-04T15:53:59.887 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_list: /' 2024-03-04T15:53:59.887 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_lock 2024-03-04T15:53:59.887 INFO:tasks.workunit.client.0.smithi117.stdout:test api_lock on pid 63958 2024-03-04T15:53:59.887 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=63958 2024-03-04T15:53:59.888 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_lock on pid 63958' 2024-03-04T15:53:59.888 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=63958 2024-03-04T15:53:59.888 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:53:59.888 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:53:59.888 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-03-04T15:53:59.889 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:53:59.889 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.890 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:53:59.890 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.890 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_lock --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock.xml 2024-03-04T15:53:59.890 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_lock.log 2024-03-04T15:53:59.891 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_lock_pp 2024-03-04T15:53:59.891 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_lock_pp' 2024-03-04T15:53:59.891 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_lock: /' 2024-03-04T15:53:59.893 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_lock_pp 2024-03-04T15:53:59.894 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:53:59.896 INFO:tasks.workunit.client.0.smithi117.stdout:test api_lock_pp on pid 63966 2024-03-04T15:53:59.897 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_lock_pp 2024-03-04T15:53:59.897 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=63966 2024-03-04T15:53:59.897 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_lock_pp on pid 63966' 2024-03-04T15:53:59.897 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=63966 2024-03-04T15:53:59.897 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:53:59.897 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:53:59.897 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_misc 2024-03-04T15:53:59.897 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_misc' 2024-03-04T15:53:59.898 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-03-04T15:53:59.898 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_misc 2024-03-04T15:53:59.898 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:53:59.900 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:53:59.901 INFO:tasks.workunit.client.0.smithi117.stdout:test api_misc on pid 63971 2024-03-04T15:53:59.901 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_misc 2024-03-04T15:53:59.902 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=63971 2024-03-04T15:53:59.902 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_misc on pid 63971' 2024-03-04T15:53:59.902 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=63971 2024-03-04T15:53:59.902 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:53:59.902 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:53:59.902 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_misc_pp 2024-03-04T15:53:59.902 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_misc_pp' 2024-03-04T15:53:59.902 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_misc_pp 2024-03-04T15:53:59.903 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-03-04T15:53:59.903 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:53:59.906 INFO:tasks.workunit.client.0.smithi117.stdout:test api_misc_pp on pid 63976 2024-03-04T15:53:59.906 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_misc_pp 2024-03-04T15:53:59.906 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=63976 2024-03-04T15:53:59.906 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_misc_pp on pid 63976' 2024-03-04T15:53:59.906 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=63976 2024-03-04T15:53:59.907 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:53:59.907 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:53:59.907 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-03-04T15:53:59.908 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.909 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:53:59.909 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.909 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:53:59.909 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.909 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:53:59.909 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:53:59.909 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.909 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:53:59.909 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.909 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.909 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_tier_pp 2024-03-04T15:53:59.909 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_misc: /' 2024-03-04T15:53:59.909 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_misc_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc_pp.xml 2024-03-04T15:53:59.909 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_tier_pp' 2024-03-04T15:53:59.910 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_lock_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_lock_pp.xml 2024-03-04T15:53:59.910 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-03-04T15:53:59.910 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_tier_pp 2024-03-04T15:53:59.911 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_lock_pp: /' 2024-03-04T15:53:59.911 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_misc --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_misc.xml 2024-03-04T15:53:59.911 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_misc.log 2024-03-04T15:53:59.912 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:53:59.916 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-03-04T15:53:59.916 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_misc_pp: /' 2024-03-04T15:53:59.921 INFO:tasks.workunit.client.0.smithi117.stdout:test api_tier_pp on pid 64025 2024-03-04T15:53:59.921 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_tier_pp 2024-03-04T15:53:59.922 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64025 2024-03-04T15:53:59.922 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_tier_pp on pid 64025' 2024-03-04T15:53:59.922 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64025 2024-03-04T15:53:59.922 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:53:59.922 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:53:59.926 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-03-04T15:53:59.926 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_pool 2024-03-04T15:53:59.926 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_pool' 2024-03-04T15:53:59.930 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:53:59.931 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.931 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:53:59.931 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.932 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-03-04T15:53:59.935 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_tier_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_tier_pp.xml 2024-03-04T15:53:59.936 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_tier_pp: /' 2024-03-04T15:53:59.941 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:53:59.943 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_pool 2024-03-04T15:53:59.947 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_pool 2024-03-04T15:53:59.948 INFO:tasks.workunit.client.0.smithi117.stdout:test api_pool on pid 64094 2024-03-04T15:53:59.948 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64094 2024-03-04T15:53:59.948 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_pool on pid 64094' 2024-03-04T15:53:59.948 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64094 2024-03-04T15:53:59.948 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:53:59.948 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:53:59.948 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-03-04T15:53:59.949 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:53:59.949 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.949 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:53:59.949 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.950 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_pool --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_pool.xml 2024-03-04T15:53:59.954 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_snapshots 2024-03-04T15:53:59.954 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_snapshots' 2024-03-04T15:53:59.958 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_pool.log 2024-03-04T15:53:59.959 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_pool: /' 2024-03-04T15:53:59.962 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:53:59.966 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_snapshots 2024-03-04T15:53:59.967 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_snapshots 2024-03-04T15:53:59.967 INFO:tasks.workunit.client.0.smithi117.stdout:test api_snapshots on pid 64122 2024-03-04T15:53:59.967 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64122 2024-03-04T15:53:59.967 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_snapshots on pid 64122' 2024-03-04T15:53:59.967 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64122 2024-03-04T15:53:59.967 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:53:59.967 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:53:59.967 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-03-04T15:53:59.968 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_snapshots_pp 2024-03-04T15:53:59.968 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_snapshots_pp' 2024-03-04T15:53:59.969 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:53:59.969 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.969 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:53:59.969 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.969 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-03-04T15:53:59.970 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_snapshots_pp 2024-03-04T15:53:59.970 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:53:59.971 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_snapshots --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots.xml 2024-03-04T15:53:59.972 INFO:tasks.workunit.client.0.smithi117.stdout:test api_snapshots_pp on pid 64144 2024-03-04T15:53:59.972 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_snapshots_pp 2024-03-04T15:53:59.973 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64144 2024-03-04T15:53:59.973 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_snapshots_pp on pid 64144' 2024-03-04T15:53:59.973 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64144 2024-03-04T15:53:59.973 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:53:59.973 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:53:59.973 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-03-04T15:53:59.974 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_stat 2024-03-04T15:53:59.976 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_stat' 2024-03-04T15:53:59.977 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:53:59.977 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.977 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:53:59.977 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.977 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-03-04T15:53:59.978 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_snapshots: /' 2024-03-04T15:53:59.978 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_snapshots_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_snapshots_pp.xml 2024-03-04T15:53:59.980 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:53:59.980 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_stat 2024-03-04T15:53:59.980 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-03-04T15:53:59.983 INFO:tasks.workunit.client.0.smithi117.stdout:test api_stat on pid 64161 2024-03-04T15:53:59.983 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_stat 2024-03-04T15:53:59.983 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64161 2024-03-04T15:53:59.983 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_stat on pid 64161' 2024-03-04T15:53:59.984 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64161 2024-03-04T15:53:59.984 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:53:59.984 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:53:59.984 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-03-04T15:53:59.984 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_stat_pp 2024-03-04T15:53:59.984 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_stat_pp' 2024-03-04T15:53:59.985 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:53:59.985 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:53:59.985 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.985 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:53:59.985 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.985 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_stat.log 2024-03-04T15:53:59.988 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_stat --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat.xml 2024-03-04T15:53:59.988 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_stat: /' 2024-03-04T15:53:59.989 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_stat_pp 2024-03-04T15:53:59.991 INFO:tasks.workunit.client.0.smithi117.stdout:test api_stat_pp on pid 64184 2024-03-04T15:53:59.991 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_stat_pp 2024-03-04T15:53:59.991 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64184 2024-03-04T15:53:59.991 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_stat_pp on pid 64184' 2024-03-04T15:53:59.991 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64184 2024-03-04T15:53:59.991 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:53:59.991 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:53:59.993 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_watch_notify 2024-03-04T15:53:59.993 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-03-04T15:53:59.994 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_watch_notify' 2024-03-04T15:53:59.996 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:53:59.996 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.996 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:53:59.996 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:53:59.997 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:53:59.997 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_stat_pp: /' 2024-03-04T15:53:59.998 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-03-04T15:53:59.999 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_stat_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_stat_pp.xml 2024-03-04T15:54:00.010 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_watch_notify 2024-03-04T15:54:00.011 INFO:tasks.workunit.client.0.smithi117.stdout:test api_watch_notify on pid 64207 2024-03-04T15:54:00.011 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_watch_notify 2024-03-04T15:54:00.011 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64207 2024-03-04T15:54:00.011 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_watch_notify on pid 64207' 2024-03-04T15:54:00.012 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64207 2024-03-04T15:54:00.012 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:54:00.012 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.014 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-03-04T15:54:00.015 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_watch_notify_pp 2024-03-04T15:54:00.015 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_watch_notify_pp' 2024-03-04T15:54:00.018 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_watch_notify_pp 2024-03-04T15:54:00.018 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.020 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.020 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.020 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.020 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.021 INFO:tasks.workunit.client.0.smithi117.stdout:test api_watch_notify_pp on pid 64238 2024-03-04T15:54:00.021 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_watch_notify_pp 2024-03-04T15:54:00.021 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64238 2024-03-04T15:54:00.021 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_watch_notify_pp on pid 64238' 2024-03-04T15:54:00.021 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64238 2024-03-04T15:54:00.021 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:54:00.021 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.021 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-03-04T15:54:00.022 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_cmd 2024-03-04T15:54:00.022 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_watch_notify: /' 2024-03-04T15:54:00.022 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_cmd' 2024-03-04T15:54:00.023 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.023 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.023 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.023 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.023 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_watch_notify --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify.xml 2024-03-04T15:54:00.023 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-03-04T15:54:00.024 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-03-04T15:54:00.024 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-03-04T15:54:00.024 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_watch_notify_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_watch_notify_pp.xml 2024-03-04T15:54:00.030 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_cmd 2024-03-04T15:54:00.031 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.033 INFO:tasks.workunit.client.0.smithi117.stdout:test api_cmd on pid 64267 2024-03-04T15:54:00.033 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_cmd 2024-03-04T15:54:00.033 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64267 2024-03-04T15:54:00.033 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_cmd on pid 64267' 2024-03-04T15:54:00.034 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64267 2024-03-04T15:54:00.034 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:54:00.034 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.034 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-03-04T15:54:00.034 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_cmd_pp 2024-03-04T15:54:00.034 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_cmd_pp' 2024-03-04T15:54:00.035 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_cmd_pp 2024-03-04T15:54:00.035 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.035 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.035 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.035 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.035 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.036 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_cmd --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd.xml 2024-03-04T15:54:00.036 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_cmd: /' 2024-03-04T15:54:00.036 INFO:tasks.workunit.client.0.smithi117.stdout:test api_cmd_pp on pid 64276 2024-03-04T15:54:00.036 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_cmd_pp 2024-03-04T15:54:00.037 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64276 2024-03-04T15:54:00.037 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_cmd_pp on pid 64276' 2024-03-04T15:54:00.037 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64276 2024-03-04T15:54:00.037 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:54:00.037 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.038 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_service 2024-03-04T15:54:00.038 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-03-04T15:54:00.038 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_service' 2024-03-04T15:54:00.038 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_service 2024-03-04T15:54:00.040 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.040 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.040 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.041 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.041 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.041 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-03-04T15:54:00.041 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_cmd_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cmd_pp.xml 2024-03-04T15:54:00.041 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-03-04T15:54:00.042 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_cmd.log 2024-03-04T15:54:00.043 INFO:tasks.workunit.client.0.smithi117.stdout:test api_service on pid 64288 2024-03-04T15:54:00.044 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_service 2024-03-04T15:54:00.044 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64288 2024-03-04T15:54:00.044 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_service on pid 64288' 2024-03-04T15:54:00.044 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64288 2024-03-04T15:54:00.044 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:54:00.044 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.044 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-03-04T15:54:00.044 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_service_pp 2024-03-04T15:54:00.044 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_service_pp' 2024-03-04T15:54:00.045 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.045 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.045 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.045 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.045 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_service --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service.xml 2024-03-04T15:54:00.046 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_service.log 2024-03-04T15:54:00.046 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_service: /' 2024-03-04T15:54:00.047 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.049 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_service_pp 2024-03-04T15:54:00.050 INFO:tasks.workunit.client.0.smithi117.stdout:test api_service_pp on pid 64296 2024-03-04T15:54:00.051 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_service_pp 2024-03-04T15:54:00.051 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64296 2024-03-04T15:54:00.051 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_service_pp on pid 64296' 2024-03-04T15:54:00.051 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64296 2024-03-04T15:54:00.051 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:54:00.051 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.054 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-03-04T15:54:00.054 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_c_write_operations 2024-03-04T15:54:00.054 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_c_write_operations' 2024-03-04T15:54:00.055 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.056 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.056 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.056 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.056 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_service_pp: /' 2024-03-04T15:54:00.062 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-03-04T15:54:00.063 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_service_pp --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_service_pp.xml 2024-03-04T15:54:00.071 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_c_write_operations 2024-03-04T15:54:00.073 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.077 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_c_write_operations 2024-03-04T15:54:00.077 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64338 2024-03-04T15:54:00.077 INFO:tasks.workunit.client.0.smithi117.stdout:test api_c_write_operations on pid 64338 2024-03-04T15:54:00.078 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_c_write_operations on pid 64338' 2024-03-04T15:54:00.078 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64338 2024-03-04T15:54:00.078 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:54:00.078 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.078 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-03-04T15:54:00.078 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_c_read_operations 2024-03-04T15:54:00.079 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.079 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.079 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.079 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.079 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-03-04T15:54:00.080 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-03-04T15:54:00.083 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_c_write_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_write_operations.xml 2024-03-04T15:54:00.084 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_c_read_operations' 2024-03-04T15:54:00.094 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_c_read_operations 2024-03-04T15:54:00.094 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.096 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_c_read_operations 2024-03-04T15:54:00.096 INFO:tasks.workunit.client.0.smithi117.stdout:test api_c_read_operations on pid 64372 2024-03-04T15:54:00.096 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64372 2024-03-04T15:54:00.097 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_c_read_operations on pid 64372' 2024-03-04T15:54:00.097 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64372 2024-03-04T15:54:00.097 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:54:00.097 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.097 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-03-04T15:54:00.098 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.098 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.098 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.098 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.098 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_c_read_operations --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_c_read_operations.xml 2024-03-04T15:54:00.098 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-03-04T15:54:00.102 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-03-04T15:54:00.106 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s api_cls_remote_reads 2024-03-04T15:54:00.106 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' api_cls_remote_reads' 2024-03-04T15:54:00.113 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.115 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo api_cls_remote_reads 2024-03-04T15:54:00.117 INFO:tasks.workunit.client.0.smithi117.stdout:test api_cls_remote_reads on pid 64395 2024-03-04T15:54:00.117 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=api_cls_remote_reads 2024-03-04T15:54:00.117 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64395 2024-03-04T15:54:00.117 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test api_cls_remote_reads on pid 64395' 2024-03-04T15:54:00.117 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64395 2024-03-04T15:54:00.117 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:54:00.117 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.118 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cls_remote_reads.xml 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-03-04T15:54:00.118 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s list_parallel 2024-03-04T15:54:00.118 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' list_parallel' 2024-03-04T15:54:00.119 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.119 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo list_parallel 2024-03-04T15:54:00.120 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.120 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.120 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.120 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.121 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_api_cls_remote_reads --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/api_cls_remote_reads.xml 2024-03-04T15:54:00.121 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-03-04T15:54:00.122 INFO:tasks.workunit.client.0.smithi117.stdout:test list_parallel on pid 64417 2024-03-04T15:54:00.123 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=list_parallel 2024-03-04T15:54:00.123 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64417 2024-03-04T15:54:00.123 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test list_parallel on pid 64417' 2024-03-04T15:54:00.123 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64417 2024-03-04T15:54:00.123 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:54:00.123 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.123 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s open_pools_parallel 2024-03-04T15:54:00.123 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' open_pools_parallel' 2024-03-04T15:54:00.124 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo open_pools_parallel 2024-03-04T15:54:00.124 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.126 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=open_pools_parallel 2024-03-04T15:54:00.126 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-03-04T15:54:00.126 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64422 2024-03-04T15:54:00.126 INFO:tasks.workunit.client.0.smithi117.stdout:test open_pools_parallel on pid 64422 2024-03-04T15:54:00.127 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test open_pools_parallel on pid 64422' 2024-03-04T15:54:00.127 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64422 2024-03-04T15:54:00.127 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-03-04T15:54:00.127 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.127 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-03-04T15:54:00.127 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s delete_pools_parallel 2024-03-04T15:54:00.128 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' delete_pools_parallel' 2024-03-04T15:54:00.138 INFO:tasks.workunit.client.0.smithi117.stdout:test delete_pools_parallel on pid 64434 2024-03-04T15:54:00.138 INFO:tasks.workunit.client.0.smithi117.stdout:test cls on pid 64457 2024-03-04T15:54:00.139 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.140 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo delete_pools_parallel 2024-03-04T15:54:00.140 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.140 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.140 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.140 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.140 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_open_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/open_pools_parallel.xml 2024-03-04T15:54:00.140 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-03-04T15:54:00.140 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-03-04T15:54:00.141 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-03-04T15:54:00.141 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.141 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=delete_pools_parallel 2024-03-04T15:54:00.141 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64434 2024-03-04T15:54:00.141 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test delete_pools_parallel on pid 64434' 2024-03-04T15:54:00.141 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64434 2024-03-04T15:54:00.141 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-04T15:54:00.141 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.141 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-03-04T15:54:00.141 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s cls 2024-03-04T15:54:00.142 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' cls' 2024-03-04T15:54:00.142 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.142 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.142 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.142 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.142 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.142 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_delete_pools_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/delete_pools_parallel.xml 2024-03-04T15:54:00.142 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-03-04T15:54:00.142 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo cls 2024-03-04T15:54:00.142 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.143 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.143 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.143 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=cls 2024-03-04T15:54:00.143 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64457 2024-03-04T15:54:00.143 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test cls on pid 64457' 2024-03-04T15:54:00.143 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64457 2024-03-04T15:54:00.143 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-04T15:54:00.143 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.143 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_rados_list_parallel.log 2024-03-04T15:54:00.143 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s cmd 2024-03-04T15:54:00.144 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' cmd' 2024-03-04T15:54:00.144 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ list_parallel: /' 2024-03-04T15:54:00.144 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cls 2>&1 | tee ceph_test_neorados_cls.log | sed "s/^/ cls: /"' 2024-03-04T15:54:00.146 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.146 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.146 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.146 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-03-04T15:54:00.146 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.146 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ cls: /' 2024-03-04T15:54:00.146 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_neorados_cls.log 2024-03-04T15:54:00.146 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.146 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_neorados_cls 2024-03-04T15:54:00.147 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo cmd 2024-03-04T15:54:00.147 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_rados_list_parallel --gtest_output=xml:/home/ubuntu/cephtest/archive/unit_test_xml_report/list_parallel.xml 2024-03-04T15:54:00.155 INFO:tasks.workunit.client.0.smithi117.stdout:test cmd on pid 64494 2024-03-04T15:54:00.155 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=cmd 2024-03-04T15:54:00.155 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64494 2024-03-04T15:54:00.155 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test cmd on pid 64494' 2024-03-04T15:54:00.155 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64494 2024-03-04T15:54:00.156 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-04T15:54:00.156 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.161 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s handler_error 2024-03-04T15:54:00.161 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' handler_error' 2024-03-04T15:54:00.162 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_cmd 2>&1 | tee ceph_test_neorados_cmd.log | sed "s/^/ cmd: /"' 2024-03-04T15:54:00.169 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.169 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.169 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.169 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.169 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.174 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo handler_error 2024-03-04T15:54:00.174 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ cmd: /' 2024-03-04T15:54:00.175 INFO:tasks.workunit.client.0.smithi117.stdout:test handler_error on pid 64517 2024-03-04T15:54:00.175 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=handler_error 2024-03-04T15:54:00.175 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64517 2024-03-04T15:54:00.176 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test handler_error on pid 64517' 2024-03-04T15:54:00.176 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64517 2024-03-04T15:54:00.176 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-04T15:54:00.176 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.176 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_neorados_cmd 2024-03-04T15:54:00.176 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_neorados_cmd.log 2024-03-04T15:54:00.182 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_handler_error 2>&1 | tee ceph_test_neorados_handler_error.log | sed "s/^/ handler_error: /"' 2024-03-04T15:54:00.183 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.183 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s io 2024-03-04T15:54:00.184 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' io' 2024-03-04T15:54:00.184 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.184 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.184 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.188 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo io 2024-03-04T15:54:00.189 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ handler_error: /' 2024-03-04T15:54:00.189 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.189 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_neorados_handler_error 2024-03-04T15:54:00.190 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_neorados_handler_error.log 2024-03-04T15:54:00.192 INFO:tasks.workunit.client.0.smithi117.stdout:test io on pid 64545 2024-03-04T15:54:00.192 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=io 2024-03-04T15:54:00.192 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64545 2024-03-04T15:54:00.193 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test io on pid 64545' 2024-03-04T15:54:00.193 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64545 2024-03-04T15:54:00.193 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-04T15:54:00.193 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.193 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s ec_io 2024-03-04T15:54:00.193 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' ec_io' 2024-03-04T15:54:00.193 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_io 2>&1 | tee ceph_test_neorados_io.log | sed "s/^/ io: /"' 2024-03-04T15:54:00.194 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.194 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.194 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.194 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.195 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ io: /' 2024-03-04T15:54:00.195 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_neorados_io.log 2024-03-04T15:54:00.196 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_neorados_io 2024-03-04T15:54:00.197 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.198 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo ec_io 2024-03-04T15:54:00.203 INFO:tasks.workunit.client.0.smithi117.stdout:test ec_io on pid 64567 2024-03-04T15:54:00.203 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=ec_io 2024-03-04T15:54:00.203 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64567 2024-03-04T15:54:00.203 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test ec_io on pid 64567' 2024-03-04T15:54:00.203 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64567 2024-03-04T15:54:00.203 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-04T15:54:00.203 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.206 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_io 2>&1 | tee ceph_test_neorados_ec_io.log | sed "s/^/ ec_io: /"' 2024-03-04T15:54:00.206 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s list 2024-03-04T15:54:00.207 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' list' 2024-03-04T15:54:00.208 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.208 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.208 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.208 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.208 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ ec_io: /' 2024-03-04T15:54:00.209 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_neorados_ec_io.log 2024-03-04T15:54:00.210 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_neorados_ec_io 2024-03-04T15:54:00.210 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo list 2024-03-04T15:54:00.210 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.215 INFO:tasks.workunit.client.0.smithi117.stdout:test list on pid 64611 2024-03-04T15:54:00.215 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=list 2024-03-04T15:54:00.215 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64611 2024-03-04T15:54:00.215 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test list on pid 64611' 2024-03-04T15:54:00.215 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64611 2024-03-04T15:54:00.215 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-04T15:54:00.216 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.218 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_list 2>&1 | tee ceph_test_neorados_list.log | sed "s/^/ list: /"' 2024-03-04T15:54:00.219 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s ec_list 2024-03-04T15:54:00.219 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' ec_list' 2024-03-04T15:54:00.221 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.221 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.221 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.221 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.221 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_neorados_list.log 2024-03-04T15:54:00.222 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo ec_list 2024-03-04T15:54:00.222 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ list: /' 2024-03-04T15:54:00.224 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_neorados_list 2024-03-04T15:54:00.225 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.230 INFO:tasks.workunit.client.0.smithi117.stdout:test ec_list on pid 64650 2024-03-04T15:54:00.230 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=ec_list 2024-03-04T15:54:00.230 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64650 2024-03-04T15:54:00.230 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test ec_list on pid 64650' 2024-03-04T15:54:00.230 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64650 2024-03-04T15:54:00.230 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-04T15:54:00.230 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.230 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_ec_list 2>&1 | tee ceph_test_neorados_ec_list.log | sed "s/^/ ec_list: /"' 2024-03-04T15:54:00.232 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.232 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.232 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.232 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.233 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_neorados_ec_list.log 2024-03-04T15:54:00.234 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s misc 2024-03-04T15:54:00.234 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' misc' 2024-03-04T15:54:00.236 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo misc 2024-03-04T15:54:00.238 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ ec_list: /' 2024-03-04T15:54:00.238 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_neorados_ec_list 2024-03-04T15:54:00.239 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.242 INFO:tasks.workunit.client.0.smithi117.stdout:test misc on pid 64681 2024-03-04T15:54:00.242 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=misc 2024-03-04T15:54:00.243 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64681 2024-03-04T15:54:00.243 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test misc on pid 64681' 2024-03-04T15:54:00.243 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64681 2024-03-04T15:54:00.243 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-04T15:54:00.243 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.243 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_misc 2>&1 | tee ceph_test_neorados_misc.log | sed "s/^/ misc: /"' 2024-03-04T15:54:00.243 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s pool 2024-03-04T15:54:00.243 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' pool' 2024-03-04T15:54:00.244 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.244 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.244 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.244 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.244 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo pool 2024-03-04T15:54:00.245 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_neorados_misc 2024-03-04T15:54:00.245 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.256 INFO:tasks.workunit.client.0.smithi117.stdout:test pool on pid 64724 2024-03-04T15:54:00.257 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=pool 2024-03-04T15:54:00.257 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64724 2024-03-04T15:54:00.257 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test pool on pid 64724' 2024-03-04T15:54:00.257 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64724 2024-03-04T15:54:00.257 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-04T15:54:00.257 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.258 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_pool 2>&1 | tee ceph_test_neorados_pool.log | sed "s/^/ pool: /"' 2024-03-04T15:54:00.258 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s read_operations 2024-03-04T15:54:00.258 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' read_operations' 2024-03-04T15:54:00.258 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo read_operations 2024-03-04T15:54:00.258 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.259 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.259 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.259 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.259 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.259 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_neorados_pool 2024-03-04T15:54:00.260 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ misc: /' 2024-03-04T15:54:00.261 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=read_operations 2024-03-04T15:54:00.261 INFO:tasks.workunit.client.0.smithi117.stdout:test read_operations on pid 64738 2024-03-04T15:54:00.262 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64738 2024-03-04T15:54:00.262 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test read_operations on pid 64738' 2024-03-04T15:54:00.262 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64738 2024-03-04T15:54:00.262 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_read_operations 2>&1 | tee ceph_test_neorados_read_operations.log | sed "s/^/ read_operations: /"' 2024-03-04T15:54:00.262 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-04T15:54:00.262 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.262 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ pool: /' 2024-03-04T15:54:00.262 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s snapshots 2024-03-04T15:54:00.263 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' snapshots' 2024-03-04T15:54:00.263 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.263 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.263 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.263 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.263 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_neorados_read_operations 2024-03-04T15:54:00.263 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_neorados_read_operations.log 2024-03-04T15:54:00.264 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_neorados_misc.log 2024-03-04T15:54:00.264 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo snapshots 2024-03-04T15:54:00.268 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_neorados_pool.log 2024-03-04T15:54:00.268 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.268 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ read_operations: /' 2024-03-04T15:54:00.271 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=snapshots 2024-03-04T15:54:00.271 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64750 2024-03-04T15:54:00.271 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test snapshots on pid 64750' 2024-03-04T15:54:00.272 INFO:tasks.workunit.client.0.smithi117.stdout:test snapshots on pid 64750 2024-03-04T15:54:00.272 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64750 2024-03-04T15:54:00.272 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_snapshots 2>&1 | tee ceph_test_neorados_snapshots.log | sed "s/^/ snapshots: /"' 2024-03-04T15:54:00.272 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-04T15:54:00.273 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.273 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s watch_notify 2024-03-04T15:54:00.273 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' watch_notify' 2024-03-04T15:54:00.274 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.274 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.274 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.274 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.274 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo watch_notify 2024-03-04T15:54:00.274 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_neorados_snapshots 2024-03-04T15:54:00.274 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_neorados_snapshots.log 2024-03-04T15:54:00.274 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ snapshots: /' 2024-03-04T15:54:00.274 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.277 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=watch_notify 2024-03-04T15:54:00.277 INFO:tasks.workunit.client.0.smithi117.stdout:test watch_notify on pid 64767 2024-03-04T15:54:00.277 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64767 2024-03-04T15:54:00.277 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test watch_notify on pid 64767' 2024-03-04T15:54:00.277 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64767 2024-03-04T15:54:00.278 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_watch_notify 2>&1 | tee ceph_test_neorados_watch_notify.log | sed "s/^/ watch_notify: /"' 2024-03-04T15:54:00.278 INFO:tasks.workunit.client.0.smithi117.stderr:+ for f in cls cmd handler_error io ec_io list ec_list misc pool read_operations snapshots watch_notify write_operations 2024-03-04T15:54:00.278 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.279 INFO:tasks.workunit.client.0.smithi117.stderr:++ printf %25s write_operations 2024-03-04T15:54:00.279 INFO:tasks.workunit.client.0.smithi117.stderr:+ r=' write_operations' 2024-03-04T15:54:00.279 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.279 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.279 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.279 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.279 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_neorados_watch_notify 2024-03-04T15:54:00.279 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_neorados_watch_notify.log 2024-03-04T15:54:00.281 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ watch_notify: /' 2024-03-04T15:54:00.283 INFO:tasks.workunit.client.0.smithi117.stderr:++ echo write_operations 2024-03-04T15:54:00.283 INFO:tasks.workunit.client.0.smithi117.stderr:++ awk '{print $1}' 2024-03-04T15:54:00.287 INFO:tasks.workunit.client.0.smithi117.stdout:test write_operations on pid 64793 2024-03-04T15:54:00.287 INFO:tasks.workunit.client.0.smithi117.stderr:+ ff=write_operations 2024-03-04T15:54:00.287 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64793 2024-03-04T15:54:00.287 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo 'test write_operations on pid 64793' 2024-03-04T15:54:00.287 INFO:tasks.workunit.client.0.smithi117.stderr:+ pids[$f]=64793 2024-03-04T15:54:00.287 INFO:tasks.workunit.client.0.smithi117.stderr:+ ret=0 2024-03-04T15:54:00.287 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' 1 -eq 1 ']' 2024-03-04T15:54:00.287 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T15:54:00.287 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64025 2024-03-04T15:54:00.288 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64025 2024-03-04T15:54:00.290 INFO:tasks.workunit.client.0.smithi117.stderr:+ bash -o pipefail -exc 'ceph_test_neorados_write_operations 2>&1 | tee ceph_test_neorados_write_operations.log | sed "s/^/ write_operations: /"' 2024-03-04T15:54:00.292 INFO:tasks.workunit.client.0.smithi117.stderr:+ '[' -z '' ']' 2024-03-04T15:54:00.292 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.292 INFO:tasks.workunit.client.0.smithi117.stderr:+ case $- in 2024-03-04T15:54:00.292 INFO:tasks.workunit.client.0.smithi117.stderr:+ return 2024-03-04T15:54:00.292 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph_test_neorados_write_operations 2024-03-04T15:54:00.292 INFO:tasks.workunit.client.0.smithi117.stderr:+ tee ceph_test_neorados_write_operations.log 2024-03-04T15:54:00.293 INFO:tasks.workunit.client.0.smithi117.stderr:+ sed 's/^/ write_operations: /' 2024-03-04T15:54:01.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:00 smithi151 bash[23756]: audit 2024-03-04T15:54:00.091947+0000 mon.c (mon.2) 91 : audit 1 from='client.? 172.21.15.117:0/3140935506' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:01.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:00 smithi151 bash[23756]: audit 2024-03-04T15:54:00.105659+0000 mon.b (mon.1) 33 : audit 0 from='client.? 172.21.15.117:0/1797993668' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-03-04T15:54:01.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:00 smithi151 bash[23756]: audit 2024-03-04T15:54:00.270319+0000 mon.c (mon.2) 92 : audit 1 from='client.? 172.21.15.117:0/705951852' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi117-64587-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:01.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:00 smithi117 bash[21480]: audit 2024-03-04T15:54:00.091947+0000 mon.c (mon.2) 91 : audit 1 from='client.? 172.21.15.117:0/3140935506' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:01.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:00 smithi117 bash[21480]: audit 2024-03-04T15:54:00.105659+0000 mon.b (mon.1) 33 : audit 0 from='client.? 172.21.15.117:0/1797993668' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-03-04T15:54:01.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:00 smithi117 bash[21480]: audit 2024-03-04T15:54:00.270319+0000 mon.c (mon.2) 92 : audit 1 from='client.? 172.21.15.117:0/705951852' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi117-64587-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:01.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:00 smithi117 bash[28843]: audit 2024-03-04T15:54:00.091947+0000 mon.c (mon.2) 91 : audit 1 from='client.? 172.21.15.117:0/3140935506' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:01.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:00 smithi117 bash[28843]: audit 2024-03-04T15:54:00.105659+0000 mon.b (mon.1) 33 : audit 0 from='client.? 172.21.15.117:0/1797993668' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-03-04T15:54:01.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:00 smithi117 bash[28843]: audit 2024-03-04T15:54:00.270319+0000 mon.c (mon.2) 92 : audit 1 from='client.? 172.21.15.117:0/705951852' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi117-64587-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:06.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:54:06 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:54:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:54:06.932 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:54:06 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:54:12.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: cluster 2024-03-04T15:53:59.412137+0000 mgr.y (mgr.24428) 160 : cluster 0 pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:54:12.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: audit 2024-03-04T15:54:00.841068+0000 mon.a (mon.0) 845 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:12.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: audit 2024-03-04T15:54:00.841661+0000 mon.a (mon.0) 846 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi117-64587-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:12.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: audit 2024-03-04T15:54:00.870933+0000 mon.b (mon.1) 34 : audit 1 from='client.? 172.21.15.117:0/636335385' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi117-63915-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: cluster 2024-03-04T15:54:01.413962+0000 mgr.y (mgr.24428) 161 : cluster 0 pgmap v107: 196 pgs: 49 creating+peering, 132 active+clean, 15 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T15:54:12.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: audit 2024-03-04T15:54:04.148329+0000 mon.c (mon.2) 93 : audit 1 from='client.? 172.21.15.117:0/3140935506' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:12.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: audit 2024-03-04T15:54:05.668204+0000 mon.c (mon.2) 94 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:54:12.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: cluster 2024-03-04T15:54:06.283055+0000 mon.a (mon.0) 847 : cluster 0 osdmap e76: 8 total, 8 up, 8 in 2024-03-04T15:54:12.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: audit 2024-03-04T15:54:06.292271+0000 mon.a (mon.0) 848 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi117-64656-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:12.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: audit 2024-03-04T15:54:06.307443+0000 mon.b (mon.1) 35 : audit 1 from='client.? 172.21.15.117:0/2329301318' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi117-63960-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: audit 2024-03-04T15:54:06.308748+0000 mon.b (mon.1) 36 : audit 1 from='client.? 172.21.15.117:0/3915996147' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi117-64169-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: audit 2024-03-04T15:54:06.309510+0000 mon.b (mon.1) 37 : audit 1 from='client.? 172.21.15.117:0/3419787529' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi117-64374-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: audit 2024-03-04T15:54:06.310715+0000 mon.b (mon.1) 38 : audit 1 from='client.? 172.21.15.117:0/1133986764' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi117-63931-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: audit 2024-03-04T15:54:06.314667+0000 mon.b (mon.1) 39 : audit 1 from='client.? 172.21.15.117:0/2288841152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi117-64189-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: cluster 2024-03-04T15:53:59.412137+0000 mgr.y (mgr.24428) 160 : cluster 0 pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:54:12.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: audit 2024-03-04T15:54:00.841068+0000 mon.a (mon.0) 845 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:12.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: audit 2024-03-04T15:54:00.841661+0000 mon.a (mon.0) 846 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi117-64587-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:12.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: audit 2024-03-04T15:54:00.870933+0000 mon.b (mon.1) 34 : audit 1 from='client.? 172.21.15.117:0/636335385' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi117-63915-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: cluster 2024-03-04T15:54:01.413962+0000 mgr.y (mgr.24428) 161 : cluster 0 pgmap v107: 196 pgs: 49 creating+peering, 132 active+clean, 15 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T15:54:12.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: audit 2024-03-04T15:54:04.148329+0000 mon.c (mon.2) 93 : audit 1 from='client.? 172.21.15.117:0/3140935506' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:12.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: audit 2024-03-04T15:54:05.668204+0000 mon.c (mon.2) 94 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:54:12.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: cluster 2024-03-04T15:54:06.283055+0000 mon.a (mon.0) 847 : cluster 0 osdmap e76: 8 total, 8 up, 8 in 2024-03-04T15:54:12.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: audit 2024-03-04T15:54:06.292271+0000 mon.a (mon.0) 848 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi117-64656-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:12.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: audit 2024-03-04T15:54:06.307443+0000 mon.b (mon.1) 35 : audit 1 from='client.? 172.21.15.117:0/2329301318' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi117-63960-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: audit 2024-03-04T15:54:06.308748+0000 mon.b (mon.1) 36 : audit 1 from='client.? 172.21.15.117:0/3915996147' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi117-64169-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: audit 2024-03-04T15:54:06.309510+0000 mon.b (mon.1) 37 : audit 1 from='client.? 172.21.15.117:0/3419787529' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi117-64374-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: audit 2024-03-04T15:54:06.310715+0000 mon.b (mon.1) 38 : audit 1 from='client.? 172.21.15.117:0/1133986764' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi117-63931-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: audit 2024-03-04T15:54:06.314667+0000 mon.b (mon.1) 39 : audit 1 from='client.? 172.21.15.117:0/2288841152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi117-64189-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: audit 2024-03-04T15:54:06.317819+0000 mon.c (mon.2) 95 : audit 1 from='client.? 172.21.15.117:0/2176842115' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: audit 2024-03-04T15:54:06.323547+0000 mon.c (mon.2) 96 : audit 1 from='client.? 172.21.15.117:0/2576661355' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi117-64156-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: audit 2024-03-04T15:54:06.326453+0000 mon.c (mon.2) 97 : audit 1 from='client.? 172.21.15.117:0/65928035' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi117-63986-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: audit 2024-03-04T15:54:06.328697+0000 mon.b (mon.1) 40 : audit 1 from='client.? 172.21.15.117:0/2861621207' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi117-64235-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: audit 2024-03-04T15:54:06.329176+0000 mon.c (mon.2) 98 : audit 1 from='client.? 172.21.15.117:0/4221982738' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi117-63937-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: cluster 2024-03-04T15:54:07.122914+0000 mon.a (mon.0) 849 : cluster 0 osdmap e77: 8 total, 8 up, 8 in 2024-03-04T15:54:12.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: audit 2024-03-04T15:54:09.298049+0000 mon.a (mon.0) 850 : audit 1 from='client.? 172.21.15.117:0/3918041939' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi117-63923-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[21480]: audit 2024-03-04T15:54:09.307132+0000 mon.a (mon.0) 851 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi117-63915-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: audit 2024-03-04T15:54:06.317819+0000 mon.c (mon.2) 95 : audit 1 from='client.? 172.21.15.117:0/2176842115' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: audit 2024-03-04T15:54:06.323547+0000 mon.c (mon.2) 96 : audit 1 from='client.? 172.21.15.117:0/2576661355' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi117-64156-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: audit 2024-03-04T15:54:06.326453+0000 mon.c (mon.2) 97 : audit 1 from='client.? 172.21.15.117:0/65928035' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi117-63986-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: audit 2024-03-04T15:54:06.328697+0000 mon.b (mon.1) 40 : audit 1 from='client.? 172.21.15.117:0/2861621207' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi117-64235-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: audit 2024-03-04T15:54:06.329176+0000 mon.c (mon.2) 98 : audit 1 from='client.? 172.21.15.117:0/4221982738' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi117-63937-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: cluster 2024-03-04T15:54:07.122914+0000 mon.a (mon.0) 849 : cluster 0 osdmap e77: 8 total, 8 up, 8 in 2024-03-04T15:54:12.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: audit 2024-03-04T15:54:09.298049+0000 mon.a (mon.0) 850 : audit 1 from='client.? 172.21.15.117:0/3918041939' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi117-63923-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:12 smithi117 bash[28843]: audit 2024-03-04T15:54:09.307132+0000 mon.a (mon.0) 851 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi117-63915-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: cluster 2024-03-04T15:53:59.412137+0000 mgr.y (mgr.24428) 160 : cluster 0 pgmap v105: 132 pgs: 132 active+clean; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-03-04T15:54:12.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: audit 2024-03-04T15:54:00.841068+0000 mon.a (mon.0) 845 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:12.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: audit 2024-03-04T15:54:00.841661+0000 mon.a (mon.0) 846 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi117-64587-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:12.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: audit 2024-03-04T15:54:00.870933+0000 mon.b (mon.1) 34 : audit 1 from='client.? 172.21.15.117:0/636335385' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi117-63915-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: cluster 2024-03-04T15:54:01.413962+0000 mgr.y (mgr.24428) 161 : cluster 0 pgmap v107: 196 pgs: 49 creating+peering, 132 active+clean, 15 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T15:54:12.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: audit 2024-03-04T15:54:04.148329+0000 mon.c (mon.2) 93 : audit 1 from='client.? 172.21.15.117:0/3140935506' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:12.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: audit 2024-03-04T15:54:05.668204+0000 mon.c (mon.2) 94 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:54:12.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: cluster 2024-03-04T15:54:06.283055+0000 mon.a (mon.0) 847 : cluster 0 osdmap e76: 8 total, 8 up, 8 in 2024-03-04T15:54:12.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: audit 2024-03-04T15:54:06.292271+0000 mon.a (mon.0) 848 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi117-64656-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:12.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: audit 2024-03-04T15:54:06.307443+0000 mon.b (mon.1) 35 : audit 1 from='client.? 172.21.15.117:0/2329301318' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi117-63960-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: audit 2024-03-04T15:54:06.308748+0000 mon.b (mon.1) 36 : audit 1 from='client.? 172.21.15.117:0/3915996147' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi117-64169-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: audit 2024-03-04T15:54:06.309510+0000 mon.b (mon.1) 37 : audit 1 from='client.? 172.21.15.117:0/3419787529' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi117-64374-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: audit 2024-03-04T15:54:06.310715+0000 mon.b (mon.1) 38 : audit 1 from='client.? 172.21.15.117:0/1133986764' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi117-63931-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: audit 2024-03-04T15:54:06.314667+0000 mon.b (mon.1) 39 : audit 1 from='client.? 172.21.15.117:0/2288841152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi117-64189-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: audit 2024-03-04T15:54:06.317819+0000 mon.c (mon.2) 95 : audit 1 from='client.? 172.21.15.117:0/2176842115' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: audit 2024-03-04T15:54:06.323547+0000 mon.c (mon.2) 96 : audit 1 from='client.? 172.21.15.117:0/2576661355' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi117-64156-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: audit 2024-03-04T15:54:06.326453+0000 mon.c (mon.2) 97 : audit 1 from='client.? 172.21.15.117:0/65928035' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi117-63986-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: audit 2024-03-04T15:54:06.328697+0000 mon.b (mon.1) 40 : audit 1 from='client.? 172.21.15.117:0/2861621207' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi117-64235-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.686 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: audit 2024-03-04T15:54:06.329176+0000 mon.c (mon.2) 98 : audit 1 from='client.? 172.21.15.117:0/4221982738' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi117-63937-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.686 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: cluster 2024-03-04T15:54:07.122914+0000 mon.a (mon.0) 849 : cluster 0 osdmap e77: 8 total, 8 up, 8 in 2024-03-04T15:54:12.686 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: audit 2024-03-04T15:54:09.298049+0000 mon.a (mon.0) 850 : audit 1 from='client.? 172.21.15.117:0/3918041939' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi117-63923-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:12.686 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:12 smithi151 bash[23756]: audit 2024-03-04T15:54:09.307132+0000 mon.a (mon.0) 851 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi117-63915-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: cluster 2024-03-04T15:54:03.414862+0000 mgr.y (mgr.24428) 162 : cluster 0 pgmap v108: 196 pgs: 49 creating+peering, 132 active+clean, 15 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T15:54:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: cluster 2024-03-04T15:54:05.415658+0000 mgr.y (mgr.24428) 163 : cluster 0 pgmap v109: 196 pgs: 49 creating+peering, 132 active+clean, 15 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T15:54:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:06.457049+0000 mgr.y (mgr.24428) 164 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:54:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: cluster 2024-03-04T15:54:07.417814+0000 mgr.y (mgr.24428) 165 : cluster 0 pgmap v111: 612 pgs: 66 creating+peering, 132 active+clean, 414 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:54:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:09.307294+0000 mon.a (mon.0) 852 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:09.308441+0000 mon.a (mon.0) 853 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi117-64656-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:09.308519+0000 mon.a (mon.0) 854 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi117-64587-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:09.329607+0000 mon.c (mon.2) 99 : audit 1 from='client.? 172.21.15.117:0/705951852' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi117-64587-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi117-64587-1"}]: dispatch 2024-03-04T15:54:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: cluster 2024-03-04T15:54:09.419274+0000 mgr.y (mgr.24428) 166 : cluster 0 pgmap v113: 836 pgs: 66 creating+peering, 132 active+clean, 638 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-04T15:54:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: cluster 2024-03-04T15:54:11.423665+0000 mgr.y (mgr.24428) 167 : cluster 0 pgmap v114: 836 pgs: 610 creating+peering, 167 active+clean, 59 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:54:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: cluster 2024-03-04T15:54:12.261477+0000 mon.a (mon.0) 855 : cluster 0 osdmap e78: 8 total, 8 up, 8 in 2024-03-04T15:54:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:12.262366+0000 mon.c (mon.2) 100 : audit 1 from='client.? 172.21.15.117:0/3140935506' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: cluster 2024-03-04T15:54:12.262639+0000 mon.a (mon.0) 856 : cluster 3 Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:12.288694+0000 mon.a (mon.0) 857 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi117-64656-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi117-64656-1"}]: dispatch 2024-03-04T15:54:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:12.289938+0000 mon.a (mon.0) 858 : audit 1 from='client.? 172.21.15.117:0/3532420059' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64273-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:12.291108+0000 mon.a (mon.0) 859 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi117-63960-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:12.292533+0000 mon.a (mon.0) 860 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi117-64169-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:12.293070+0000 mon.a (mon.0) 861 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi117-64374-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:12.293359+0000 mon.a (mon.0) 862 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:12.293738+0000 mon.a (mon.0) 863 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi117-63931-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:12.294472+0000 mon.a (mon.0) 864 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi117-64189-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:12.294667+0000 mon.a (mon.0) 865 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi117-64156-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:12.295176+0000 mon.a (mon.0) 866 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi117-63986-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:12.295548+0000 mon.a (mon.0) 867 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi117-64235-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:12.296202+0000 mon.a (mon.0) 868 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi117-63937-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:12.298089+0000 mon.a (mon.0) 869 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi117-64587-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi117-64587-1"}]: dispatch 2024-03-04T15:54:15.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:12.311290+0000 mon.a (mon.0) 870 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:15.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[28843]: audit 2024-03-04T15:54:12.312799+0000 mon.a (mon.0) 871 : audit 1 from='client.? 172.21.15.117:0/3918041939' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi117-63923-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:15.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: cluster 2024-03-04T15:54:03.414862+0000 mgr.y (mgr.24428) 162 : cluster 0 pgmap v108: 196 pgs: 49 creating+peering, 132 active+clean, 15 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T15:54:15.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: cluster 2024-03-04T15:54:05.415658+0000 mgr.y (mgr.24428) 163 : cluster 0 pgmap v109: 196 pgs: 49 creating+peering, 132 active+clean, 15 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T15:54:15.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:06.457049+0000 mgr.y (mgr.24428) 164 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:54:15.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: cluster 2024-03-04T15:54:07.417814+0000 mgr.y (mgr.24428) 165 : cluster 0 pgmap v111: 612 pgs: 66 creating+peering, 132 active+clean, 414 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:54:15.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:09.307294+0000 mon.a (mon.0) 852 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:15.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:09.308441+0000 mon.a (mon.0) 853 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi117-64656-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:15.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:09.308519+0000 mon.a (mon.0) 854 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi117-64587-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:15.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:09.329607+0000 mon.c (mon.2) 99 : audit 1 from='client.? 172.21.15.117:0/705951852' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi117-64587-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi117-64587-1"}]: dispatch 2024-03-04T15:54:15.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: cluster 2024-03-04T15:54:09.419274+0000 mgr.y (mgr.24428) 166 : cluster 0 pgmap v113: 836 pgs: 66 creating+peering, 132 active+clean, 638 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-04T15:54:15.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: cluster 2024-03-04T15:54:11.423665+0000 mgr.y (mgr.24428) 167 : cluster 0 pgmap v114: 836 pgs: 610 creating+peering, 167 active+clean, 59 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:54:15.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: cluster 2024-03-04T15:54:12.261477+0000 mon.a (mon.0) 855 : cluster 0 osdmap e78: 8 total, 8 up, 8 in 2024-03-04T15:54:15.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:12.262366+0000 mon.c (mon.2) 100 : audit 1 from='client.? 172.21.15.117:0/3140935506' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:15.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: cluster 2024-03-04T15:54:12.262639+0000 mon.a (mon.0) 856 : cluster 3 Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:15.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:12.288694+0000 mon.a (mon.0) 857 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi117-64656-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi117-64656-1"}]: dispatch 2024-03-04T15:54:15.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:12.289938+0000 mon.a (mon.0) 858 : audit 1 from='client.? 172.21.15.117:0/3532420059' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64273-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:12.291108+0000 mon.a (mon.0) 859 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi117-63960-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:12.292533+0000 mon.a (mon.0) 860 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi117-64169-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:12.293070+0000 mon.a (mon.0) 861 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi117-64374-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:12.293359+0000 mon.a (mon.0) 862 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:12.293738+0000 mon.a (mon.0) 863 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi117-63931-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:12.294472+0000 mon.a (mon.0) 864 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi117-64189-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:12.294667+0000 mon.a (mon.0) 865 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi117-64156-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:12.295176+0000 mon.a (mon.0) 866 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi117-63986-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:12.295548+0000 mon.a (mon.0) 867 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi117-64235-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:12.296202+0000 mon.a (mon.0) 868 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi117-63937-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:12.298089+0000 mon.a (mon.0) 869 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi117-64587-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi117-64587-1"}]: dispatch 2024-03-04T15:54:15.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:12.311290+0000 mon.a (mon.0) 870 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:15.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:15 smithi117 bash[21480]: audit 2024-03-04T15:54:12.312799+0000 mon.a (mon.0) 871 : audit 1 from='client.? 172.21.15.117:0/3918041939' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi117-63923-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:15.656 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-03-04T15:54:15.656 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [----------] Global test environment set-up. 2024-03-04T15:54:15.656 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [----------] 12 tests from AsioRados 2024-03-04T15:54:15.656 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-03-04T15:54:15.657 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (1 ms) 2024-03-04T15:54:15.657 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-03-04T15:54:15.657 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (1 ms) 2024-03-04T15:54:15.657 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-03-04T15:54:15.657 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (0 ms) 2024-03-04T15:54:15.657 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-03-04T15:54:15.657 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (6 ms) 2024-03-04T15:54:15.657 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-03-04T15:54:15.657 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (5 ms) 2024-03-04T15:54:15.657 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-03-04T15:54:15.657 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (5 ms) 2024-03-04T15:54:15.657 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-03-04T15:54:15.657 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (1 ms) 2024-03-04T15:54:15.657 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-03-04T15:54:15.657 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (0 ms) 2024-03-04T15:54:15.657 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-03-04T15:54:15.658 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (1 ms) 2024-03-04T15:54:15.658 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-03-04T15:54:15.658 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (1 ms) 2024-03-04T15:54:15.658 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-03-04T15:54:15.658 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (2 ms) 2024-03-04T15:54:15.658 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-03-04T15:54:15.658 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (2 ms) 2024-03-04T15:54:15.658 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [----------] 12 tests from AsioRados (25 ms total) 2024-03-04T15:54:15.658 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: 2024-03-04T15:54:15.658 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [----------] Global test environment tear-down 2024-03-04T15:54:15.658 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (15697 ms total) 2024-03-04T15:54:15.658 INFO:tasks.workunit.client.0.smithi117.stdout: api_asio: [ PASSED ] 12 tests. 2024-03-04T15:54:15.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: cluster 2024-03-04T15:54:03.414862+0000 mgr.y (mgr.24428) 162 : cluster 0 pgmap v108: 196 pgs: 49 creating+peering, 132 active+clean, 15 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T15:54:15.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: cluster 2024-03-04T15:54:05.415658+0000 mgr.y (mgr.24428) 163 : cluster 0 pgmap v109: 196 pgs: 49 creating+peering, 132 active+clean, 15 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T15:54:15.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:06.457049+0000 mgr.y (mgr.24428) 164 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:54:15.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: cluster 2024-03-04T15:54:07.417814+0000 mgr.y (mgr.24428) 165 : cluster 0 pgmap v111: 612 pgs: 66 creating+peering, 132 active+clean, 414 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:54:15.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:09.307294+0000 mon.a (mon.0) 852 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:15.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:09.308441+0000 mon.a (mon.0) 853 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectssmithi117-64656-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:15.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:09.308519+0000 mon.a (mon.0) 854 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritesmithi117-64587-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:15.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:09.329607+0000 mon.c (mon.2) 99 : audit 1 from='client.? 172.21.15.117:0/705951852' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi117-64587-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi117-64587-1"}]: dispatch 2024-03-04T15:54:15.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: cluster 2024-03-04T15:54:09.419274+0000 mgr.y (mgr.24428) 166 : cluster 0 pgmap v113: 836 pgs: 66 creating+peering, 132 active+clean, 638 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-04T15:54:15.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: cluster 2024-03-04T15:54:11.423665+0000 mgr.y (mgr.24428) 167 : cluster 0 pgmap v114: 836 pgs: 610 creating+peering, 167 active+clean, 59 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:54:15.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: cluster 2024-03-04T15:54:12.261477+0000 mon.a (mon.0) 855 : cluster 0 osdmap e78: 8 total, 8 up, 8 in 2024-03-04T15:54:15.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:12.262366+0000 mon.c (mon.2) 100 : audit 1 from='client.? 172.21.15.117:0/3140935506' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:15.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: cluster 2024-03-04T15:54:12.262639+0000 mon.a (mon.0) 856 : cluster 3 Health check failed: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:15.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:12.288694+0000 mon.a (mon.0) 857 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectssmithi117-64656-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi117-64656-1"}]: dispatch 2024-03-04T15:54:15.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:12.289938+0000 mon.a (mon.0) 858 : audit 1 from='client.? 172.21.15.117:0/3532420059' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64273-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:12.291108+0000 mon.a (mon.0) 859 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi117-63960-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:12.292533+0000 mon.a (mon.0) 860 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi117-64169-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:12.293070+0000 mon.a (mon.0) 861 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi117-64374-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:12.293359+0000 mon.a (mon.0) 862 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:12.293738+0000 mon.a (mon.0) 863 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi117-63931-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:12.294472+0000 mon.a (mon.0) 864 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi117-64189-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:12.294667+0000 mon.a (mon.0) 865 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi117-64156-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:12.295176+0000 mon.a (mon.0) 866 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi117-63986-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:12.295548+0000 mon.a (mon.0) 867 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi117-64235-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:12.296202+0000 mon.a (mon.0) 868 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi117-63937-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:15.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:12.298089+0000 mon.a (mon.0) 869 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritesmithi117-64587-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi117-64587-1"}]: dispatch 2024-03-04T15:54:15.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:12.311290+0000 mon.a (mon.0) 870 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:15.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:15 smithi151 bash[23756]: audit 2024-03-04T15:54:12.312799+0000 mon.a (mon.0) 871 : audit 1 from='client.? 172.21.15.117:0/3918041939' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi117-63923-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:16.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:54:16 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:54:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:54:16.932 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:54:16 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:54:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:16 smithi151 bash[23756]: audit 2024-03-04T15:54:12.312924+0000 mon.a (mon.0) 872 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi117-63915-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:16 smithi151 bash[23756]: audit 2024-03-04T15:54:12.336831+0000 mon.b (mon.1) 41 : audit 1 from='client.? 172.21.15.117:0/3558721679' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64411-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:16 smithi151 bash[23756]: audit 2024-03-04T15:54:12.337811+0000 mon.b (mon.1) 42 : audit 1 from='client.? 172.21.15.117:0/875595284' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi117-64133-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:16 smithi151 bash[23756]: audit 2024-03-04T15:54:12.343632+0000 mon.b (mon.1) 43 : audit 1 from='client.? 172.21.15.117:0/1113997168' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:16 smithi151 bash[23756]: cluster 2024-03-04T15:54:15.261672+0000 mon.a (mon.0) 873 : cluster 0 osdmap e79: 8 total, 8 up, 8 in 2024-03-04T15:54:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:16 smithi151 bash[23756]: audit 2024-03-04T15:54:15.298694+0000 mon.a (mon.0) 874 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64411-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:16.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:16 smithi151 bash[23756]: audit 2024-03-04T15:54:15.309344+0000 mon.a (mon.0) 875 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi117-64133-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:16.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:16 smithi151 bash[23756]: audit 2024-03-04T15:54:15.309808+0000 mon.a (mon.0) 876 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:17.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:16 smithi117 bash[28843]: audit 2024-03-04T15:54:12.312924+0000 mon.a (mon.0) 872 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi117-63915-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:17.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:16 smithi117 bash[28843]: audit 2024-03-04T15:54:12.336831+0000 mon.b (mon.1) 41 : audit 1 from='client.? 172.21.15.117:0/3558721679' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64411-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:17.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:16 smithi117 bash[28843]: audit 2024-03-04T15:54:12.337811+0000 mon.b (mon.1) 42 : audit 1 from='client.? 172.21.15.117:0/875595284' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi117-64133-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:17.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:16 smithi117 bash[28843]: audit 2024-03-04T15:54:12.343632+0000 mon.b (mon.1) 43 : audit 1 from='client.? 172.21.15.117:0/1113997168' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:17.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:16 smithi117 bash[28843]: cluster 2024-03-04T15:54:15.261672+0000 mon.a (mon.0) 873 : cluster 0 osdmap e79: 8 total, 8 up, 8 in 2024-03-04T15:54:17.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:16 smithi117 bash[28843]: audit 2024-03-04T15:54:15.298694+0000 mon.a (mon.0) 874 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64411-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:17.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:16 smithi117 bash[28843]: audit 2024-03-04T15:54:15.309344+0000 mon.a (mon.0) 875 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi117-64133-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:17.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:16 smithi117 bash[28843]: audit 2024-03-04T15:54:15.309808+0000 mon.a (mon.0) 876 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:17.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:16 smithi117 bash[21480]: audit 2024-03-04T15:54:12.312924+0000 mon.a (mon.0) 872 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi117-63915-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:17.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:16 smithi117 bash[21480]: audit 2024-03-04T15:54:12.336831+0000 mon.b (mon.1) 41 : audit 1 from='client.? 172.21.15.117:0/3558721679' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64411-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:17.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:16 smithi117 bash[21480]: audit 2024-03-04T15:54:12.337811+0000 mon.b (mon.1) 42 : audit 1 from='client.? 172.21.15.117:0/875595284' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi117-64133-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:17.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:16 smithi117 bash[21480]: audit 2024-03-04T15:54:12.343632+0000 mon.b (mon.1) 43 : audit 1 from='client.? 172.21.15.117:0/1113997168' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:17.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:16 smithi117 bash[21480]: cluster 2024-03-04T15:54:15.261672+0000 mon.a (mon.0) 873 : cluster 0 osdmap e79: 8 total, 8 up, 8 in 2024-03-04T15:54:17.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:16 smithi117 bash[21480]: audit 2024-03-04T15:54:15.298694+0000 mon.a (mon.0) 874 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64411-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:17.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:16 smithi117 bash[21480]: audit 2024-03-04T15:54:15.309344+0000 mon.a (mon.0) 875 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi117-64133-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:17.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:16 smithi117 bash[21480]: audit 2024-03-04T15:54:15.309808+0000 mon.a (mon.0) 876 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:17.777 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: cluster 2024-03-04T15:54:13.425466+0000 mgr.y (mgr.24428) 168 : cluster 0 pgmap v116: 1060 pgs: 610 creating+peering, 167 active+clean, 283 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-04T15:54:17.777 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:15.312723+0000 mon.c (mon.2) 101 : audit 0 from='client.? 172.21.15.117:0/1852782749' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-04T15:54:17.777 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:15.312875+0000 mon.a (mon.0) 877 : audit 1 from='client.? 172.21.15.117:0/3532420059' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64273-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:17.778 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:15.312954+0000 mon.a (mon.0) 878 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi117-63960-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:17.778 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:15.313004+0000 mon.a (mon.0) 879 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi117-64169-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:17.778 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:15.313081+0000 mon.a (mon.0) 880 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi117-64374-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:17.778 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:15.313284+0000 mon.a (mon.0) 881 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:17.778 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:15.313397+0000 mon.a (mon.0) 882 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi117-63931-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:17.778 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:15.313497+0000 mon.a (mon.0) 883 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi117-64189-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:17.778 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:15.313602+0000 mon.a (mon.0) 884 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi117-64156-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:17.778 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:15.313713+0000 mon.a (mon.0) 885 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi117-63986-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:17.778 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:15.313801+0000 mon.a (mon.0) 886 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi117-64235-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:17.778 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:15.313904+0000 mon.a (mon.0) 887 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi117-63937-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:17.778 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:15.314000+0000 mon.a (mon.0) 888 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:17.778 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:15.337131+0000 mon.c (mon.2) 102 : audit 1 from='client.? 172.21.15.117:0/3140935506' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi117-64254-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:17.778 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: cluster 2024-03-04T15:54:15.426654+0000 mgr.y (mgr.24428) 169 : cluster 0 pgmap v118: 1252 pgs: 578 creating+peering, 167 active+clean, 507 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 837 B/s rd, 0 op/s 2024-03-04T15:54:17.779 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:16.223901+0000 mon.c (mon.2) 103 : audit 1 from='client.? 172.21.15.117:0/871738455' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64285-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:17.779 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: cluster 2024-03-04T15:54:16.644925+0000 mon.a (mon.0) 889 : cluster 0 osdmap e80: 8 total, 8 up, 8 in 2024-03-04T15:54:17.779 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:16.690017+0000 mon.a (mon.0) 890 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:17.779 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:16.690234+0000 mon.a (mon.0) 891 : audit 1 from='client.? 172.21.15.117:0/2485526553' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi117-64061-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:17.779 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:16.690491+0000 mon.a (mon.0) 892 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi117-63955-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:17.779 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:16.691746+0000 mon.a (mon.0) 893 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi117-64254-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:17.779 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:16.694876+0000 mon.a (mon.0) 894 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi117-64656-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi117-64656-1"}]': finished 2024-03-04T15:54:17.779 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:16.695002+0000 mon.a (mon.0) 895 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi117-64587-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi117-64587-1"}]': finished 2024-03-04T15:54:17.779 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:16.695093+0000 mon.a (mon.0) 896 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64411-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:17.779 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:16.695183+0000 mon.a (mon.0) 897 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi117-64133-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:17.779 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:16.695250+0000 mon.a (mon.0) 898 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:17.779 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: cluster 2024-03-04T15:54:16.702759+0000 mon.a (mon.0) 899 : cluster 0 osdmap e81: 8 total, 8 up, 8 in 2024-03-04T15:54:17.779 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: cluster 2024-03-04T15:54:17.290750+0000 mon.a (mon.0) 900 : cluster 3 Health check update: 22 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:17.779 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:17 smithi151 bash[23756]: audit 2024-03-04T15:54:17.293157+0000 mon.a (mon.0) 901 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64285-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:18.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: cluster 2024-03-04T15:54:13.425466+0000 mgr.y (mgr.24428) 168 : cluster 0 pgmap v116: 1060 pgs: 610 creating+peering, 167 active+clean, 283 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-04T15:54:18.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:15.312723+0000 mon.c (mon.2) 101 : audit 0 from='client.? 172.21.15.117:0/1852782749' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-04T15:54:18.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:15.312875+0000 mon.a (mon.0) 877 : audit 1 from='client.? 172.21.15.117:0/3532420059' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64273-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:15.312954+0000 mon.a (mon.0) 878 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi117-63960-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:15.313004+0000 mon.a (mon.0) 879 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi117-64169-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:15.313081+0000 mon.a (mon.0) 880 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi117-64374-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:15.313284+0000 mon.a (mon.0) 881 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:15.313397+0000 mon.a (mon.0) 882 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi117-63931-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:15.313497+0000 mon.a (mon.0) 883 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi117-64189-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:15.313602+0000 mon.a (mon.0) 884 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi117-64156-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:15.313713+0000 mon.a (mon.0) 885 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi117-63986-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:15.313801+0000 mon.a (mon.0) 886 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi117-64235-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:15.313904+0000 mon.a (mon.0) 887 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi117-63937-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:15.314000+0000 mon.a (mon.0) 888 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:18.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: cluster 2024-03-04T15:54:13.425466+0000 mgr.y (mgr.24428) 168 : cluster 0 pgmap v116: 1060 pgs: 610 creating+peering, 167 active+clean, 283 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-04T15:54:18.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:15.312723+0000 mon.c (mon.2) 101 : audit 0 from='client.? 172.21.15.117:0/1852782749' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-04T15:54:18.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:15.312875+0000 mon.a (mon.0) 877 : audit 1 from='client.? 172.21.15.117:0/3532420059' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64273-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:15.312954+0000 mon.a (mon.0) 878 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi117-63960-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:15.313004+0000 mon.a (mon.0) 879 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi117-64169-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:15.313081+0000 mon.a (mon.0) 880 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi117-64374-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:15.313284+0000 mon.a (mon.0) 881 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:15.313397+0000 mon.a (mon.0) 882 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi117-63931-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:15.313497+0000 mon.a (mon.0) 883 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi117-64189-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:15.313602+0000 mon.a (mon.0) 884 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi117-64156-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:15.313713+0000 mon.a (mon.0) 885 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi117-63986-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:15.313801+0000 mon.a (mon.0) 886 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi117-64235-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:15.313904+0000 mon.a (mon.0) 887 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi117-63937-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:15.314000+0000 mon.a (mon.0) 888 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:18.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:15.337131+0000 mon.c (mon.2) 102 : audit 1 from='client.? 172.21.15.117:0/3140935506' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi117-64254-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:18.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: cluster 2024-03-04T15:54:15.426654+0000 mgr.y (mgr.24428) 169 : cluster 0 pgmap v118: 1252 pgs: 578 creating+peering, 167 active+clean, 507 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 837 B/s rd, 0 op/s 2024-03-04T15:54:18.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:16.223901+0000 mon.c (mon.2) 103 : audit 1 from='client.? 172.21.15.117:0/871738455' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64285-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:18.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: cluster 2024-03-04T15:54:16.644925+0000 mon.a (mon.0) 889 : cluster 0 osdmap e80: 8 total, 8 up, 8 in 2024-03-04T15:54:18.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:16.690017+0000 mon.a (mon.0) 890 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:18.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:16.690234+0000 mon.a (mon.0) 891 : audit 1 from='client.? 172.21.15.117:0/2485526553' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi117-64061-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:18.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:16.690491+0000 mon.a (mon.0) 892 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi117-63955-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:18.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:16.691746+0000 mon.a (mon.0) 893 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi117-64254-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:18.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:16.694876+0000 mon.a (mon.0) 894 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi117-64656-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi117-64656-1"}]': finished 2024-03-04T15:54:18.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:16.695002+0000 mon.a (mon.0) 895 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi117-64587-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi117-64587-1"}]': finished 2024-03-04T15:54:18.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:16.695093+0000 mon.a (mon.0) 896 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64411-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:16.695183+0000 mon.a (mon.0) 897 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi117-64133-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:16.695250+0000 mon.a (mon.0) 898 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: cluster 2024-03-04T15:54:16.702759+0000 mon.a (mon.0) 899 : cluster 0 osdmap e81: 8 total, 8 up, 8 in 2024-03-04T15:54:18.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: cluster 2024-03-04T15:54:17.290750+0000 mon.a (mon.0) 900 : cluster 3 Health check update: 22 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:18.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[28843]: audit 2024-03-04T15:54:17.293157+0000 mon.a (mon.0) 901 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64285-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:18.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:15.337131+0000 mon.c (mon.2) 102 : audit 1 from='client.? 172.21.15.117:0/3140935506' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi117-64254-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:18.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: cluster 2024-03-04T15:54:15.426654+0000 mgr.y (mgr.24428) 169 : cluster 0 pgmap v118: 1252 pgs: 578 creating+peering, 167 active+clean, 507 unknown; 582 KiB data, 248 MiB used, 715 GiB / 715 GiB avail; 837 B/s rd, 0 op/s 2024-03-04T15:54:18.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:16.223901+0000 mon.c (mon.2) 103 : audit 1 from='client.? 172.21.15.117:0/871738455' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64285-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:18.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: cluster 2024-03-04T15:54:16.644925+0000 mon.a (mon.0) 889 : cluster 0 osdmap e80: 8 total, 8 up, 8 in 2024-03-04T15:54:18.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:16.690017+0000 mon.a (mon.0) 890 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:18.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:16.690234+0000 mon.a (mon.0) 891 : audit 1 from='client.? 172.21.15.117:0/2485526553' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi117-64061-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:18.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:16.690491+0000 mon.a (mon.0) 892 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi117-63955-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:18.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:16.691746+0000 mon.a (mon.0) 893 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi117-64254-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:18.075 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:16.694876+0000 mon.a (mon.0) 894 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi117-64656-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi117-64656-1"}]': finished 2024-03-04T15:54:18.075 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:16.695002+0000 mon.a (mon.0) 895 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi117-64587-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi117-64587-1"}]': finished 2024-03-04T15:54:18.075 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:16.695093+0000 mon.a (mon.0) 896 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64411-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.075 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:16.695183+0000 mon.a (mon.0) 897 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi117-64133-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.075 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:16.695250+0000 mon.a (mon.0) 898 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.075 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: cluster 2024-03-04T15:54:16.702759+0000 mon.a (mon.0) 899 : cluster 0 osdmap e81: 8 total, 8 up, 8 in 2024-03-04T15:54:18.075 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: cluster 2024-03-04T15:54:17.290750+0000 mon.a (mon.0) 900 : cluster 3 Health check update: 22 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:18.075 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:17 smithi117 bash[21480]: audit 2024-03-04T15:54:17.293157+0000 mon.a (mon.0) 901 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64285-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:18.113 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_1_[64497]: starting. 2024-03-04T15:54:18.113 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_1_[64497]: creating pool ceph_test_rados_list_parallel.smithi117-64456 2024-03-04T15:54:18.113 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_1_[64497]: created object 0... 2024-03-04T15:54:18.113 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_1_[64497]: created object 25... 2024-03-04T15:54:18.113 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_1_[64497]: created object 49... 2024-03-04T15:54:18.113 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_1_[64497]: finishing. 2024-03-04T15:54:18.113 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_1_[64497]: shutting down. 2024-03-04T15:54:18.113 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_2_[64498]: starting. 2024-03-04T15:54:18.113 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_2_[64498]: listing objects. 2024-03-04T15:54:18.114 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_2_[64498]: listed object 0... 2024-03-04T15:54:18.114 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_2_[64498]: listed object 25... 2024-03-04T15:54:18.114 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_2_[64498]: saw 50 objects 2024-03-04T15:54:18.114 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_2_[64498]: shutting down. 2024-03-04T15:54:18.114 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.114 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_3_[65419]: starting. 2024-03-04T15:54:18.114 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_3_[65419]: creating pool ceph_test_rados_list_parallel.smithi117-64456 2024-03-04T15:54:18.114 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_3_[65419]: created object 0... 2024-03-04T15:54:18.114 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_3_[65419]: created object 25... 2024-03-04T15:54:18.114 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_3_[65419]: created object 49... 2024-03-04T15:54:18.114 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_3_[65419]: finishing. 2024-03-04T15:54:18.114 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_3_[65419]: shutting down. 2024-03-04T15:54:18.114 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.114 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_4_[65420]: starting. 2024-03-04T15:54:18.114 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_4_[65420]: listing objects. 2024-03-04T15:54:18.114 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_4_[65420]: listed object 0... 2024-03-04T15:54:18.115 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_4_[65420]: listed object 25... 2024-03-04T15:54:18.115 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_4_[65420]: saw 42 objects 2024-03-04T15:54:18.115 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_4_[65420]: shutting down. 2024-03-04T15:54:18.115 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.115 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_5_[65421]: starting. 2024-03-04T15:54:18.115 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_5_[65421]: removed 25 objects... 2024-03-04T15:54:18.115 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_5_[65421]: removed half of the objects 2024-03-04T15:54:18.115 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_5_[65421]: removed 50 objects... 2024-03-04T15:54:18.115 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_5_[65421]: removed 50 objects 2024-03-04T15:54:18.115 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_5_[65421]: shutting down. 2024-03-04T15:54:18.115 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.115 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.115 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_6_[65529]: starting. 2024-03-04T15:54:18.115 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_6_[65529]: creating pool ceph_test_rados_list_parallel.smithi117-64456 2024-03-04T15:54:18.115 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_6_[65529]: created object 0... 2024-03-04T15:54:18.116 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_6_[65529]: created object 25... 2024-03-04T15:54:18.116 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_6_[65529]: created object 49... 2024-03-04T15:54:18.116 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_6_[65529]: finishing. 2024-03-04T15:54:18.116 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_6_[65529]: shutting down. 2024-03-04T15:54:18.116 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.116 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.116 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_7_[65530]: starting. 2024-03-04T15:54:18.116 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_7_[65530]: listing objects. 2024-03-04T15:54:18.116 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_7_[65530]: listed object 0... 2024-03-04T15:54:18.116 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_7_[65530]: listed object 25... 2024-03-04T15:54:18.116 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_7_[65530]: listed object 50... 2024-03-04T15:54:18.116 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_7_[65530]: saw 55 objects 2024-03-04T15:54:18.116 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_7_[65530]: shutting down. 2024-03-04T15:54:18.116 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.117 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.117 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_8_[65531]: starting. 2024-03-04T15:54:18.117 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_8_[65531]: added 25 objects... 2024-03-04T15:54:18.117 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_8_[65531]: added half of the objects 2024-03-04T15:54:18.117 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_8_[65531]: added 50 objects... 2024-03-04T15:54:18.117 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_8_[65531]: added 50 objects 2024-03-04T15:54:18.117 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_8_[65531]: shutting down. 2024-03-04T15:54:18.117 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.117 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.729 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ********************** api_cmd: Running main() from gmock_main.cc 2024-03-04T15:54:18.729 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-03-04T15:54:18.729 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: [----------] Global test environment set-up. 2024-03-04T15:54:18.729 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-03-04T15:54:18.729 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-03-04T15:54:18.729 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (57 ms) 2024-03-04T15:54:18.729 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-03-04T15:54:18.730 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (65 ms) 2024-03-04T15:54:18.730 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-03-04T15:54:18.730 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (18523 ms) 2024-03-04T15:54:18.730 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-03-04T15:54:18.730 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:15.312875+0000 mon.a [INF] from='client.? 172.21.15.117:0/3532420059' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64273-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.730 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:15.312954+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi117-63960-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.730 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:15.313004+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi117-64169-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.730 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:15.313081+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi117-64374-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.730 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:15.313284+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.730 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:15.313397+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi117-63931-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.730 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:15.313497+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi117-64189-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.730 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:15.313602+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi117-64156-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.730 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:15.313713+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi117-63986-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.730 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:15.313801+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi117-64235-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.731 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:15.313904+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi117-63937-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:18.731 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:15.314000+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:18.942 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:15.337131+0000 mon.c [INF] from='client.? 172.21.15.117:0/3140935506' entity='client.admin' cmd=[{"prefix": "osd pool cre********* 2024-03-04T15:54:18.942 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_9_[65601]: starting. 2024-03-04T15:54:18.942 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_9_[65601]: creating pool ceph_test_rados_list_parallel.smithi117-64456 2024-03-04T15:54:18.942 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_9_[65601]: created object 0... 2024-03-04T15:54:18.942 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_9_[65601]: created object 25... 2024-03-04T15:54:18.942 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_9_[65601]: created object 49... 2024-03-04T15:54:18.942 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_9_[65601]: finishing. 2024-03-04T15:54:18.942 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_9_[65601]: shutting down. 2024-03-04T15:54:18.942 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.943 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.943 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.943 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_10_[65602]: starting. 2024-03-04T15:54:18.943 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_10_[65602]: listing objects. 2024-03-04T15:54:18.943 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_10_[65602]: listed object 0... 2024-03-04T15:54:18.943 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_10_[65602]: listed object 25... 2024-03-04T15:54:18.943 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_10_[65602]: listed object 50... 2024-03-04T15:54:18.943 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_10_[65602]: listed object 75... 2024-03-04T15:54:18.943 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_10_[65602]: saw 98 objects 2024-03-04T15:54:18.943 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_10_[65602]: shutting down. 2024-03-04T15:54:18.943 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.943 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.943 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.943 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_13_[65605]: starting. 2024-03-04T15:54:18.943 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_13_[65605]: removed 25 objects... 2024-03-04T15:54:18.944 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_13_[65605]: removed half of the objects 2024-03-04T15:54:18.944 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_13_[65605]: removed 50 objects... 2024-03-04T15:54:18.944 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_13_[65605]: removed 50 objects 2024-03-04T15:54:18.944 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_13_[65605]: shutting down. 2024-03-04T15:54:18.944 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.944 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.944 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.944 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_11_[65603]: starting. 2024-03-04T15:54:18.944 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_11_[65603]: added 25 objects... 2024-03-04T15:54:18.944 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_11_[65603]: added half of the objects 2024-03-04T15:54:18.944 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_11_[65603]: added 50 objects... 2024-03-04T15:54:18.944 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_11_[65603]: added 50 objects 2024-03-04T15:54:18.944 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_11_[65603]: shutting down. 2024-03-04T15:54:18.944 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.944 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.944 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.945 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_12_[65604]: starting. 2024-03-04T15:54:18.945 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_12_[65604]: added 25 objects... 2024-03-04T15:54:18.945 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_12_[65604]: added half of the objects 2024-03-04T15:54:18.945 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_12_[65604]: added 50 objects... 2024-03-04T15:54:18.945 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_12_[65604]: added 50 objects 2024-03-04T15:54:18.945 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_12_[65604]: shutting down. 2024-03-04T15:54:18.945 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.945 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.945 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.945 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.945 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_14_[65691]: starting. 2024-03-04T15:54:18.945 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_14_[65691]: creating pool ceph_test_rados_list_parallel.smithi117-64456 2024-03-04T15:54:18.945 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_14_[65691]: created object 0... 2024-03-04T15:54:18.945 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_14_[65691]: created object 25... 2024-03-04T15:54:18.945 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_14_[65691]: created object 49... 2024-03-04T15:54:18.945 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_14_[65691]: finishing. 2024-03-04T15:54:18.946 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_14_[65691]: shutting down. 2024-03-04T15:54:18.946 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.946 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.946 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.946 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:18.946 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_16_[65693]: starting. 2024-03-04T15:54:18.946 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_16_[65693]: added 25 objects... 2024-03-04T15:54:18.946 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_16_[65693]: added half of the objects 2024-03-04T15:54:19.120 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_16_[65693]: added api_list: [==========] Running 11 tests from 3 test suites. 2024-03-04T15:54:19.120 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [----------] Global test environment set-up. 2024-03-04T15:54:19.121 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [----------] 7 tests from LibRadosList 2024-03-04T15:54:19.121 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-03-04T15:54:19.121 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ OK ] LibRadosList.ListObjects (62 ms) 2024-03-04T15:54:19.121 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-03-04T15:54:19.121 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (33 ms) 2024-03-04T15:54:19.121 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-03-04T15:54:19.121 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: myset foo1,foo2,foo3 2024-03-04T15:54:19.121 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: foo1 2024-03-04T15:54:19.121 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: foo2 2024-03-04T15:54:19.121 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: foo3 2024-03-04T15:54:19.121 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: myset foo1,foo4,foo5 2024-03-04T15:54:19.121 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: foo4 2024-03-04T15:54:19.121 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: foo5 2024-03-04T15:54:19.121 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: foo1 2024-03-04T15:54:19.121 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: myset foo6,foo7 2024-03-04T15:54:19.122 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: foo7 2024-03-04T15:54:19.122 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: foo6 2024-03-04T15:54:19.122 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-03-04T15:54:19.122 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: ns1:foo4 2024-03-04T15:54:19.122 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: ns1:foo5 2024-03-04T15:54:19.122 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: ns2:foo7 2024-03-04T15:54:19.122 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: ns2:foo6 2024-03-04T15:54:19.122 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: ns1:foo1 2024-03-04T15:54:19.122 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: :foo1 2024-03-04T15:54:19.122 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: :foo2 2024-03-04T15:54:19.122 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: :foo3 2024-03-04T15:54:19.122 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (107 ms) 2024-03-04T15:54:19.122 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-03-04T15:54:19.122 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 1 0 2024-03-04T15:54:19.122 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 10 0 2024-03-04T15:54:19.122 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 13 0 2024-03-04T15:54:19.123 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 7 0 2024-03-04T15:54:19.123 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 14 0 2024-03-04T15:54:19.123 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 0 0 2024-03-04T15:54:19.123 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 15 0 2024-03-04T15:54:19.123 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 11 0 2024-03-04T15:54:19.123 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 5 0 2024-03-04T15:54:19.123 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 8 0 2024-03-04T15:54:19.123 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 6 0 2024-03-04T15:54:19.123 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 3 0 2024-03-04T15:54:19.123 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 4 0 2024-03-04T15:54:19.123 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 12 0 2024-03-04T15:54:19.123 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 9 0 2024-03-04T15:54:19.123 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 2 0 2024-03-04T15:54:19.123 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-03-04T15:54:19.123 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (103 ms) 2024-03-04T15:54:19.123 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-03-04T15:54:19.124 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: x cursor=MIN 2024-03-04T15:54:19.124 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > oid=1 cursor=39:02547ec2:::1:head 2024-03-04T15:54:19.124 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > oid=10 cursor=39:52ea6a34:::10:head 2024-03-04T15:54:19.124 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > oid=13 cursor=39:566253c9:::13:head 2024-03-04T15:54:19.124 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > oid=7 cursor=39:5c6b0b28:::7:head 2024-03-04T15:54:19.124 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > oid=14 cursor=39:62a1935d:::14:head 2024-03-04T15:54:19.124 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > oid=0 cursor=39:6cac518f:::0:head 2024-03-04T15:54:19.124 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > oid=15 cursor=39:863748b0:::15:head 2024-03-04T15:54:19.124 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > oid=11 cursor=39:89d3ae78:::11:head 2024-03-04T15:54:19.124 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > oid=5 cursor=39:b29083e3:::5:head 2024-03-04T15:54:19.125 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > oid=8 cursor=39:bd63b0f1:::8:head 2024-03-04T15:54:19.125 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > oid=6 cursor=39:c4fdafeb:::6:head 2024-03-04T15:54:19.125 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > oid=3 cursor=39:cfc208b3:::3:head 2024-03-04T15:54:19.125 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > oid=4 cursor=39:d83876eb:::4:head 2024-03-04T15:54:19.125 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > oid=12 cursor=39:de5d7c5f:::12:head 2024-03-04T15:54:19.125 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > oid=9 cursor=39:e960b815:::9:head 2024-03-04T15:54:19.125 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > oid=2 cursor=39:f905c69b:::2:head 2024-03-04T15:54:19.125 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: FIRST> seek to MIN oid=1 2024-03-04T15:54:19.125 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : oid=1 cursor=39:02547ec2:::1:head 2024-03-04T15:54:19.125 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:02547ec2:::1:head 2024-03-04T15:54:19.125 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:02547ec2:::1:head -> 1 2024-03-04T15:54:19.125 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : oid=10 cursor=39:52ea6a34:::10:head 2024-03-04T15:54:19.125 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:52ea6a34:::10:head 2024-03-04T15:54:19.125 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:52ea6a34:::10:head -> 10 2024-03-04T15:54:19.125 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : oid=13 cursor=39:566253c9:::13:head 2024-03-04T15:54:19.126 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:566253c9:::13:head 2024-03-04T15:54:19.126 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:566253c9:::13:head -> 13 2024-03-04T15:54:19.126 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : oid=7 cursor=39:5c6b0b28:::7:head 2024-03-04T15:54:19.126 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:5c6b0b28:::7:head 2024-03-04T15:54:19.126 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:5c6b0b28:::7:head -> 7 2024-03-04T15:54:19.158 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : oid=14 cuate", "pool": "LibRadosWatchNotifyECPP_smithi117-64254-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:19.158 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:16.223901+0000 mon.c [INF] from='client.? 172.21.15.117:0/871738455' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64285-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:19.158 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:16.690017+0000 mon.a [INF] from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:19.158 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:16.690234+0000 mon.a [INF] from='client.? 172.21.15.117:0/2485526553' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi117-64061-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:19.158 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:16.690491+0000 mon.a [INF] from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi117-63955-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:19.158 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:16.691746+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi117-64254-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:19.158 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:16.694876+0000 mon.a [INF] from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectssmithi117-64656-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectssmithi117-64656-1"}]': finished 2024-03-04T15:54:19.158 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:16.695002+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritesmithi117-64587-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritesmithi117-64587-1"}]': finished 2024-03-04T15:54:19.158 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:16.695093+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64411-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:19.158 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:16.695183+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi117-64133-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:19.159 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:16.695250+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:19.159 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:17.290750+0000 mon.a [WRN] Health check update: 22 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:19.159 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:17.293157+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64285-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:19.159 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:17.783650+0000 mon.a [INF] from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:19.193 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:17.783736+0000 mon.a [INF] from='client.? 172.21.15.117:0/2485526553' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi117-64061-1","app": "rados","yes_i_ api_c_read_operations: Running main() from gmock_main.cc 2024-03-04T15:54:19.194 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [==========] Running 17 tests from 1 test suite. 2024-03-04T15:54:19.194 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-03-04T15:54:19.194 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest 2024-03-04T15:54:19.194 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-03-04T15:54:19.194 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-03-04T15:54:19.194 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-03-04T15:54:19.194 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (272 ms) 2024-03-04T15:54:19.194 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-03-04T15:54:19.194 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (14 ms) 2024-03-04T15:54:19.194 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-03-04T15:54:19.194 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (22 ms) 2024-03-04T15:54:19.194 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-03-04T15:54:19.194 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (20 ms) 2024-03-04T15:54:19.194 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-03-04T15:54:19.194 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (8 ms) 2024-03-04T15:54:19.194 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-03-04T15:54:19.195 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (8 ms) 2024-03-04T15:54:19.195 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-03-04T15:54:19.195 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (5 ms) 2024-03-04T15:54:19.195 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-03-04T15:54:19.195 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (5 ms) 2024-03-04T15:54:19.195 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-03-04T15:54:19.195 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (4 ms) 2024-03-04T15:54:19.195 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-03-04T15:54:19.195 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (5 ms) 2024-03-04T15:54:19.195 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-03-04T15:54:19.195 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (5 ms) 2024-03-04T15:54:19.195 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat2 2024-03-04T15:54:19.195 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat2 (10 ms) 2024-03-04T15:54:19.195 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-03-04T15:54:19.196 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (11 ms) 2024-03-04T15:54:19.196 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-03-04T15:54:19.196 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (9 ms) 2024-03-04T15:54:19.196 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-03-04T15:54:19.196 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (13 ms) 2024-03-04T15:54:19.196 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-03-04T15:54:19.196 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (7 ms) 2024-03-04T15:54:19.196 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [----------] 17 tests from CReadOpsTest (418 ms total) 2024-03-04T15:54:19.196 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: 2024-03-04T15:54:19.196 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-03-04T15:54:19.196 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [==========] 17 tests from 1 test suite ran. (19078 ms total) 2024-03-04T15:54:19.196 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_read_operations: [ PASSED ] 17 tests. 2024-03-04T15:54:19.303 INFO:tasks.workunit.client.0.smithi117.stdout:rsor=39:62a1935d:::14:head 2024-03-04T15:54:19.303 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:62a1935d:::14:head 2024-03-04T15:54:19.303 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:62a1935d:::14:head -> 14 2024-03-04T15:54:19.303 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : oid=0 cursor=39:6cac518f:::0:head 2024-03-04T15:54:19.303 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:6cac518f:::0:head 2024-03-04T15:54:19.303 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:6cac518f:::0:head -> 0 2024-03-04T15:54:19.303 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : oid=15 cursor=39:863748b0:::15:head 2024-03-04T15:54:19.303 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:863748b0:::15:head 2024-03-04T15:54:19.303 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:863748b0:::15:head -> 15 2024-03-04T15:54:19.303 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : oid=11 cursor=39:89d3ae78:::11:head 2024-03-04T15:54:19.304 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:89d3ae78:::11:head 2024-03-04T15:54:19.304 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:89d3ae78:::11:head -> 11 2024-03-04T15:54:19.304 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : oid=5 cursor=39:b29083e3:::5:head 2024-03-04T15:54:19.304 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:b29083e3:::5:head 2024-03-04T15:54:19.304 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:b29083e3:::5:head -> 5 2024-03-04T15:54:19.304 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : oid=8 cursor=39:bd63b0f1:::8:head 2024-03-04T15:54:19.304 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:bd63b0f1:::8:head 2024-03-04T15:54:19.304 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:bd63b0f1:::8:head -> 8 2024-03-04T15:54:19.304 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : oid=6 cursor=39:c4fdafeb:::6:head 2024-03-04T15:54:19.304 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:c4fdafeb:::6:head 2024-03-04T15:54:19.304 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:c4fdafeb:::6:head -> 6 2024-03-04T15:54:19.304 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : oid=3 cursor=39:cfc208b3:::3:head 2024-03-04T15:54:19.304 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:cfc208b3:::3:head 2024-03-04T15:54:19.304 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:cfc208b3:::3:head -> 3 2024-03-04T15:54:19.304 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : oid=4 cursor=39:d83876eb:::4:head 2024-03-04T15:54:19.305 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:d83876eb:::4:head 2024-03-04T15:54:19.305 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:d83876eb:::4:head -> 4 2024-03-04T15:54:19.305 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : oid=12 cursor=39:de5d7c5f:::12:head 2024-03-04T15:54:19.305 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:de5d7c5f:::12:head 2024-03-04T15:54:19.305 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:de5d7c5f:::12:head -> 12 2024-03-04T15:54:19.305 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : oid=9 cursor=39:e960b815:::9:head 2024-03-04T15:54:19.305 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:e960b815:::9:head 2024-03-04T15:54:19.305 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:e960b815:::9:head -> 9 2024-03-04T15:54:19.305 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : oid=2 cursor=39:f905c69b:::2:head 2024-03-04T15:54:19.305 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:f905c69b:::2:head 2024-03-04T15:54:19.305 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:f905c69b:::2:head -> 2 2024-03-04T15:54:19.305 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:c4fdafeb:::6:head 2024-03-04T15:54:19.305 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : cursor()=39:c4fdafeb:::6:head expected=39:c4fdafeb:::6:head 2024-03-04T15:54:19.305 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:c4fdafeb:::6:head -> 6 2024-03-04T15:54:19.305 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : entry=6 expected=6 2024-03-04T15:54:19.306 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:bd63b0f1:::8:head 2024-03-04T15:54:19.306 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : cursor()=39:bd63b0f1:::8:head expected=39:bd63b0f1:::8:head 2024-03-04T15:54:19.306 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:bd63b0f1:::8:head -> 8 2024-03-04T15:54:19.306 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : entry=8 expected=8 2024-03-04T15:54:19.306 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:f905c69b:::2:head 2024-03-04T15:54:19.306 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : cursor()=39:f905c69b:::2:head expected=39:f905c69b:::2:head 2024-03-04T15:54:19.306 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:f905c69b:::2:head -> 2 2024-03-04T15:54:19.306 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : entry=2 expected=2 2024-03-04T15:54:19.306 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:e960b815:::9:head 2024-03-04T15:54:19.306 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : cursor()=39:e960b815:::9:head expected=39:e960b815:::9:head 2024-03-04T15:54:19.306 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:e960b815:::9:head -> 9 2024-03-04T15:54:19.306 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : entry=9 expected=9 2024-03-04T15:54:19.306 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:de5d7c5f:::12:head 2024-03-04T15:54:19.306 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : cursor()=39:de5d7c5f:::12:head expected=39:de5d7c5f:::12:head 2024-03-04T15:54:19.306 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:de5d7c5f:::12:head -> 12 2024-03-04T15:54:19.307 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : entry=12 expected=12 2024-03-04T15:54:19.307 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:d83876eb:::4:head 2024-03-04T15:54:19.307 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : cursor()=39:d83876eb:::4:head expected=39:d83876eb:::4:head 2024-03-04T15:54:19.307 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:d83876eb:::4:head -> 4 2024-03-04T15:54:19.307 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : entry=4 expected=4 2024-03-04T15:54:19.307 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:cfc208b3:::3:head 2024-03-04T15:54:19.307 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : cursor()=39:cfc208b3:::3:head expected=39:cfc208b3:::3:head 2024-03-04T15:54:19.307 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:cfc208b3:::3:head -> 3 2024-03-04T15:54:19.307 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : entry=3 expected=3 2024-03-04T15:54:19.307 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:5c6b0b28:::7:head 2024-03-04T15:54:19.307 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : cursor()=39:5c6b0b28:::7:head expected=39:5c6b0b28:::7:head 2024-03-04T15:54:19.307 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:5c6b0b28:::7:head -> 7 2024-03-04T15:54:19.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:19 smithi151 bash[23756]: audit 2024-03-04T15:54:17.332766+0000 mon.c (mon.2) 104 : audit 0 from='client.? 172.21.15.117:0/4239499071' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-04T15:54:19.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:19 smithi151 bash[23756]: audit 2024-03-04T15:54:17.783650+0000 mon.a (mon.0) 902 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:19.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:19 smithi151 bash[23756]: audit 2024-03-04T15:54:17.783736+0000 mon.a (mon.0) 903 : audit 1 from='client.? 172.21.15.117:0/2485526553' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi117-64061-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:19.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:19 smithi151 bash[23756]: audit 2024-03-04T15:54:17.783810+0000 mon.a (mon.0) 904 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi117-63955-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:19.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:19 smithi151 bash[23756]: audit 2024-03-04T15:54:17.783896+0000 mon.a (mon.0) 905 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64285-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:19.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:19 smithi151 bash[23756]: audit 2024-03-04T15:54:18.272066+0000 mon.c (mon.2) 105 : audit 1 from='client.? 172.21.15.117:0/705951852' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi117-64587-1"}]: dispatch 2024-03-04T15:54:19.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:19 smithi151 bash[23756]: audit 2024-03-04T15:54:18.312275+0000 mon.c (mon.2) 106 : audit 1 from='client.? 172.21.15.117:0/3022824111' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:19.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:19 smithi151 bash[23756]: audit 2024-03-04T15:54:18.313174+0000 mon.b (mon.1) 44 : audit 1 from='client.? 172.21.15.117:0/807910892' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:19.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:19 smithi151 bash[23756]: audit 2024-03-04T15:54:18.323413+0000 mon.c (mon.2) 107 : audit 1 from='client.? 172.21.15.117:0/2687162500' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:19.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:19 smithi151 bash[23756]: cluster 2024-03-04T15:54:18.692837+0000 mon.a (mon.0) 906 : cluster 0 osdmap e82: 8 total, 8 up, 8 in 2024-03-04T15:54:19.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:19 smithi117 bash[21480]: audit 2024-03-04T15:54:17.332766+0000 mon.c (mon.2) 104 : audit 0 from='client.? 172.21.15.117:0/4239499071' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-04T15:54:19.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:19 smithi117 bash[21480]: audit 2024-03-04T15:54:17.783650+0000 mon.a (mon.0) 902 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:19.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:19 smithi117 bash[21480]: audit 2024-03-04T15:54:17.783736+0000 mon.a (mon.0) 903 : audit 1 from='client.? 172.21.15.117:0/2485526553' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi117-64061-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:19.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:19 smithi117 bash[21480]: audit 2024-03-04T15:54:17.783810+0000 mon.a (mon.0) 904 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi117-63955-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:19.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:19 smithi117 bash[21480]: audit 2024-03-04T15:54:17.783896+0000 mon.a (mon.0) 905 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64285-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:19.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:19 smithi117 bash[21480]: audit 2024-03-04T15:54:18.272066+0000 mon.c (mon.2) 105 : audit 1 from='client.? 172.21.15.117:0/705951852' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi117-64587-1"}]: dispatch 2024-03-04T15:54:19.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:19 smithi117 bash[21480]: audit 2024-03-04T15:54:18.312275+0000 mon.c (mon.2) 106 : audit 1 from='client.? 172.21.15.117:0/3022824111' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:19.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:19 smithi117 bash[21480]: audit 2024-03-04T15:54:18.313174+0000 mon.b (mon.1) 44 : audit 1 from='client.? 172.21.15.117:0/807910892' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:19.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:19 smithi117 bash[21480]: audit 2024-03-04T15:54:18.323413+0000 mon.c (mon.2) 107 : audit 1 from='client.? 172.21.15.117:0/2687162500' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:19.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:19 smithi117 bash[21480]: cluster 2024-03-04T15:54:18.692837+0000 mon.a (mon.0) 906 : cluster 0 osdmap e82: 8 total, 8 up, 8 in 2024-03-04T15:54:19.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:19 smithi117 bash[28843]: audit 2024-03-04T15:54:17.332766+0000 mon.c (mon.2) 104 : audit 0 from='client.? 172.21.15.117:0/4239499071' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-03-04T15:54:19.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:19 smithi117 bash[28843]: audit 2024-03-04T15:54:17.783650+0000 mon.a (mon.0) 902 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:19.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:19 smithi117 bash[28843]: audit 2024-03-04T15:54:17.783736+0000 mon.a (mon.0) 903 : audit 1 from='client.? 172.21.15.117:0/2485526553' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi117-64061-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:19.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:19 smithi117 bash[28843]: audit 2024-03-04T15:54:17.783810+0000 mon.a (mon.0) 904 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi117-63955-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:19.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:19 smithi117 bash[28843]: audit 2024-03-04T15:54:17.783896+0000 mon.a (mon.0) 905 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64285-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:19.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:19 smithi117 bash[28843]: audit 2024-03-04T15:54:18.272066+0000 mon.c (mon.2) 105 : audit 1 from='client.? 172.21.15.117:0/705951852' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi117-64587-1"}]: dispatch 2024-03-04T15:54:19.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:19 smithi117 bash[28843]: audit 2024-03-04T15:54:18.312275+0000 mon.c (mon.2) 106 : audit 1 from='client.? 172.21.15.117:0/3022824111' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:19.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:19 smithi117 bash[28843]: audit 2024-03-04T15:54:18.313174+0000 mon.b (mon.1) 44 : audit 1 from='client.? 172.21.15.117:0/807910892' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:19.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:19 smithi117 bash[28843]: audit 2024-03-04T15:54:18.323413+0000 mon.c (mon.2) 107 : audit 1 from='client.? 172.21.15.117:0/2687162500' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:19.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:19 smithi117 bash[28843]: cluster 2024-03-04T15:54:18.692837+0000 mon.a (mon.0) 906 : cluster 0 osdmap e82: 8 total, 8 up, 8 in 2024-03-04T15:54:20.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:20 smithi117 bash[28843]: audit 2024-03-04T15:54:16.468676+0000 mgr.y (mgr.24428) 170 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:54:20.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:20 smithi117 bash[28843]: cluster 2024-03-04T15:54:17.427543+0000 mgr.y (mgr.24428) 171 : cluster 0 pgmap v120: 1204 pgs: 306 creating+peering, 740 active+clean, 158 unknown; 601 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 32 KiB/s wr, 65 op/s 2024-03-04T15:54:20.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:20 smithi117 bash[28843]: audit 2024-03-04T15:54:18.732364+0000 mon.c (mon.2) 108 : audit 1 from='client.? 172.21.15.117:0/1028889172' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-04T15:54:20.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:20 smithi117 bash[28843]: audit 2024-03-04T15:54:19.173057+0000 mon.a (mon.0) 907 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi117-64254-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]': finished 2024-03-04T15:54:20.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:20 smithi117 bash[28843]: audit 2024-03-04T15:54:19.191649+0000 mon.c (mon.2) 109 : audit 1 from='client.? 172.21.15.117:0/3699730053' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi117-63915-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:20.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:20 smithi117 bash[28843]: audit 2024-03-04T15:54:19.192901+0000 mon.c (mon.2) 110 : audit 1 from='client.? 172.21.15.117:0/3134094376' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:20.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:20 smithi117 bash[28843]: cluster 2024-03-04T15:54:19.649078+0000 mon.a (mon.0) 908 : cluster 0 osdmap e83: 8 total, 8 up, 8 in 2024-03-04T15:54:20.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:20 smithi117 bash[28843]: audit 2024-03-04T15:54:19.649997+0000 mon.a (mon.0) 909 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:20.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:20 smithi117 bash[28843]: audit 2024-03-04T15:54:19.658314+0000 mon.c (mon.2) 111 : audit 1 from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:20.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:20 smithi117 bash[28843]: audit 2024-03-04T15:54:19.668197+0000 mon.a (mon.0) 910 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi117-64587-1"}]: dispatch 2024-03-04T15:54:20.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:20 smithi117 bash[21480]: audit 2024-03-04T15:54:16.468676+0000 mgr.y (mgr.24428) 170 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:54:20.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:20 smithi117 bash[21480]: cluster 2024-03-04T15:54:17.427543+0000 mgr.y (mgr.24428) 171 : cluster 0 pgmap v120: 1204 pgs: 306 creating+peering, 740 active+clean, 158 unknown; 601 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 32 KiB/s wr, 65 op/s 2024-03-04T15:54:20.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:20 smithi117 bash[21480]: audit 2024-03-04T15:54:18.732364+0000 mon.c (mon.2) 108 : audit 1 from='client.? 172.21.15.117:0/1028889172' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-04T15:54:20.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:20 smithi117 bash[21480]: audit 2024-03-04T15:54:19.173057+0000 mon.a (mon.0) 907 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi117-64254-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]': finished 2024-03-04T15:54:20.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:20 smithi117 bash[21480]: audit 2024-03-04T15:54:19.191649+0000 mon.c (mon.2) 109 : audit 1 from='client.? 172.21.15.117:0/3699730053' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi117-63915-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:20.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:20 smithi117 bash[21480]: audit 2024-03-04T15:54:19.192901+0000 mon.c (mon.2) 110 : audit 1 from='client.? 172.21.15.117:0/3134094376' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:20.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:20 smithi117 bash[21480]: cluster 2024-03-04T15:54:19.649078+0000 mon.a (mon.0) 908 : cluster 0 osdmap e83: 8 total, 8 up, 8 in 2024-03-04T15:54:20.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:20 smithi117 bash[21480]: audit 2024-03-04T15:54:19.649997+0000 mon.a (mon.0) 909 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:20.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:20 smithi117 bash[21480]: audit 2024-03-04T15:54:19.658314+0000 mon.c (mon.2) 111 : audit 1 from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:20.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:20 smithi117 bash[21480]: audit 2024-03-04T15:54:19.668197+0000 mon.a (mon.0) 910 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi117-64587-1"}]: dispatch 2024-03-04T15:54:20.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:20 smithi151 bash[23756]: audit 2024-03-04T15:54:16.468676+0000 mgr.y (mgr.24428) 170 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:54:20.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:20 smithi151 bash[23756]: cluster 2024-03-04T15:54:17.427543+0000 mgr.y (mgr.24428) 171 : cluster 0 pgmap v120: 1204 pgs: 306 creating+peering, 740 active+clean, 158 unknown; 601 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 19 KiB/s rd, 32 KiB/s wr, 65 op/s 2024-03-04T15:54:20.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:20 smithi151 bash[23756]: audit 2024-03-04T15:54:18.732364+0000 mon.c (mon.2) 108 : audit 1 from='client.? 172.21.15.117:0/1028889172' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-04T15:54:20.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:20 smithi151 bash[23756]: audit 2024-03-04T15:54:19.173057+0000 mon.a (mon.0) 907 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi117-64254-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]': finished 2024-03-04T15:54:20.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:20 smithi151 bash[23756]: audit 2024-03-04T15:54:19.191649+0000 mon.c (mon.2) 109 : audit 1 from='client.? 172.21.15.117:0/3699730053' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi117-63915-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:20.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:20 smithi151 bash[23756]: audit 2024-03-04T15:54:19.192901+0000 mon.c (mon.2) 110 : audit 1 from='client.? 172.21.15.117:0/3134094376' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:20.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:20 smithi151 bash[23756]: cluster 2024-03-04T15:54:19.649078+0000 mon.a (mon.0) 908 : cluster 0 osdmap e83: 8 total, 8 up, 8 in 2024-03-04T15:54:20.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:20 smithi151 bash[23756]: audit 2024-03-04T15:54:19.649997+0000 mon.a (mon.0) 909 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:20.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:20 smithi151 bash[23756]: audit 2024-03-04T15:54:19.658314+0000 mon.c (mon.2) 111 : audit 1 from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:20.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:20 smithi151 bash[23756]: audit 2024-03-04T15:54:19.668197+0000 mon.a (mon.0) 910 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi117-64587-1"}]: dispatch 2024-03-04T15:54:22.417 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : entry=7 expectreally_mean_it": true}]': finished 2024-03-04T15:54:22.417 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:17.783810+0000 mon.a [INF] from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi117-63955-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:22.417 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:17.783896+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64285-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:22.417 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:18.272066+0000 mon.c [INF] from='client.? 172.21.15.117:0/705951852' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi117-64587-1"}]: dispatch 2024-03-04T15:54:22.417 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:18.312275+0000 mon.c [INF] from='client.? 172.21.15.117:0/3022824111' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:22.418 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:18.313174+0000 mon.b [INF] from='client.? 172.21.15.117:0/807910892' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:22.418 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:18.323413+0000 mon.c [INF] from='client.? 172.21.15.117:0/2687162500' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:22.418 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:18.732364+0000 mon.c [INF] from='client.? 172.21.15.117:0/1028889172' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-04T15:54:22.418 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:19.173057+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi117-64254-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]': finished 2024-03-04T15:54:22.418 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:19.191649+0000 mon.c [INF] from='client.? 172.21.15.117:0/3699730053' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi117-63915-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:22.418 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:19.192901+0000 mon.c [INF] from='client.? 172.21.15.117:0/3134094376' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:22.418 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:19.658314+0000 mon.c [INF] from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:22.418 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:19.668197+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi117-64587-1"}]: dispatch 2024-03-04T15:54:22.419 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:18.732663+0000 client.admin [INF] onexx 2024-03-04T15:54:22.419 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:20.551747+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:22.419 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:20.559699+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:22.419 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:20.560357+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-04T15:54:22.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:22 smithi151 bash[23756]: cluster 2024-03-04T15:54:18.732663+0000 client.admin (client.?) 0 : cluster 1 onexx 2024-03-04T15:54:22.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:22 smithi151 bash[23756]: audit 2024-03-04T15:54:20.551747+0000 mon.a (mon.0) 911 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:22.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:22 smithi151 bash[23756]: audit 2024-03-04T15:54:20.559699+0000 mon.a (mon.0) 912 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:22.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:22 smithi151 bash[23756]: audit 2024-03-04T15:54:20.560357+0000 mon.a (mon.0) 913 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-04T15:54:22.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:22 smithi151 bash[23756]: audit 2024-03-04T15:54:20.562530+0000 mon.a (mon.0) 914 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi117-63915-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:22.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:22 smithi151 bash[23756]: audit 2024-03-04T15:54:20.562720+0000 mon.a (mon.0) 915 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:22.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:22 smithi117 bash[21480]: cluster 2024-03-04T15:54:18.732663+0000 client.admin (client.?) 0 : cluster 1 onexx 2024-03-04T15:54:22.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:22 smithi117 bash[21480]: audit 2024-03-04T15:54:20.551747+0000 mon.a (mon.0) 911 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:22.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:22 smithi117 bash[21480]: audit 2024-03-04T15:54:20.559699+0000 mon.a (mon.0) 912 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:22.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:22 smithi117 bash[21480]: audit 2024-03-04T15:54:20.560357+0000 mon.a (mon.0) 913 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-04T15:54:22.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:22 smithi117 bash[21480]: audit 2024-03-04T15:54:20.562530+0000 mon.a (mon.0) 914 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi117-63915-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:22.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:22 smithi117 bash[21480]: audit 2024-03-04T15:54:20.562720+0000 mon.a (mon.0) 915 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:22.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:22 smithi117 bash[28843]: cluster 2024-03-04T15:54:18.732663+0000 client.admin (client.?) 0 : cluster 1 onexx 2024-03-04T15:54:22.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:22 smithi117 bash[28843]: audit 2024-03-04T15:54:20.551747+0000 mon.a (mon.0) 911 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:22.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:22 smithi117 bash[28843]: audit 2024-03-04T15:54:20.559699+0000 mon.a (mon.0) 912 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:22.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:22 smithi117 bash[28843]: audit 2024-03-04T15:54:20.560357+0000 mon.a (mon.0) 913 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-03-04T15:54:22.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:22 smithi117 bash[28843]: audit 2024-03-04T15:54:20.562530+0000 mon.a (mon.0) 914 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi117-63915-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:22.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:22 smithi117 bash[28843]: audit 2024-03-04T15:54:20.562720+0000 mon.a (mon.0) 915 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:23.854 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:20.562530+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi117-63 50 objects... 2024-03-04T15:54:23.854 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_16_[65693]: added 50 objects 2024-03-04T15:54:23.854 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_16_[65693]: shutting down. 2024-03-04T15:54:23.854 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:23.854 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:23.855 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:23.855 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:23.855 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_15_[65692]: starting. 2024-03-04T15:54:23.855 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_15_[65692]: listing objects. 2024-03-04T15:54:23.855 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_15_[65692]: listed object 0... 2024-03-04T15:54:23.855 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_15_[65692]: listed object 25... 2024-03-04T15:54:23.855 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_15_[65692]: listed object 50... 2024-03-04T15:54:23.855 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_15_[65692]: listed object 75... 2024-03-04T15:54:23.855 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_15_[65692]: listed object 100... 2024-03-04T15:54:23.855 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_15_[65692]: listed object 125... 2024-03-04T15:54:23.855 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_15_[65692]: saw 150 objects 2024-03-04T15:54:23.855 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: process_15_[65692]: shutting down. 2024-03-04T15:54:23.855 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:23.855 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:23.856 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:23.856 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:23.856 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******************************* 2024-03-04T15:54:23.856 INFO:tasks.workunit.client.0.smithi117.stdout: list_parallel: ******* SUCCESS ********** 2024-03-04T15:54:24.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: cluster 2024-03-04T15:54:19.428386+0000 mgr.y (mgr.24428) 172 : cluster 0 pgmap v123: 916 pgs: 284 creating+peering, 356 active+clean, 276 unknown; 590 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 16 KiB/s wr, 21 op/s 2024-03-04T15:54:24.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:20.563061+0000 mon.a (mon.0) 916 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi117-64587-1"}]': finished 2024-03-04T15:54:24.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:20.572809+0000 mon.c (mon.2) 112 : audit 1 from='client.? 172.21.15.117:0/705951852' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi117-64587-1"}]: dispatch 2024-03-04T15:54:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:20.574118+0000 mon.c (mon.2) 113 : audit 1 from='client.? 172.21.15.117:0/3022824111' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:20.656440+0000 mon.c (mon.2) 114 : audit 1 from='client.? 172.21.15.117:0/1728754262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:21.065474+0000 mon.c (mon.2) 115 : audit 1 from='client.? 172.21.15.117:0/2687162500' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: cluster 2024-03-04T15:54:22.405427+0000 mon.a (mon.0) 917 : cluster 0 osdmap e84: 8 total, 8 up, 8 in 2024-03-04T15:54:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: cluster 2024-03-04T15:54:22.411610+0000 mon.a (mon.0) 918 : cluster 3 Health check update: 21 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:22.418227+0000 mon.a (mon.0) 919 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-03-04T15:54:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:22.423026+0000 mon.c (mon.2) 116 : audit 1 from='client.? 172.21.15.117:0/1028889172' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-04T15:54:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:22.429035+0000 mon.a (mon.0) 920 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:22.433503+0000 mon.a (mon.0) 921 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:22.434681+0000 mon.a (mon.0) 922 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi117-63915-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:22.434918+0000 mon.a (mon.0) 923 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: cluster 2024-03-04T15:54:22.453731+0000 mon.a (mon.0) 924 : cluster 0 osdmap e85: 8 total, 8 up, 8 in 2024-03-04T15:54:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:22.455171+0000 mon.c (mon.2) 117 : audit 1 from='client.? 172.21.15.117:0/64238658' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:24.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: cluster 2024-03-04T15:54:19.428386+0000 mgr.y (mgr.24428) 172 : cluster 0 pgmap v123: 916 pgs: 284 creating+peering, 356 active+clean, 276 unknown; 590 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 16 KiB/s wr, 21 op/s 2024-03-04T15:54:24.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:20.563061+0000 mon.a (mon.0) 916 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi117-64587-1"}]': finished 2024-03-04T15:54:24.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:20.572809+0000 mon.c (mon.2) 112 : audit 1 from='client.? 172.21.15.117:0/705951852' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi117-64587-1"}]: dispatch 2024-03-04T15:54:24.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:20.574118+0000 mon.c (mon.2) 113 : audit 1 from='client.? 172.21.15.117:0/3022824111' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:24.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:20.656440+0000 mon.c (mon.2) 114 : audit 1 from='client.? 172.21.15.117:0/1728754262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:24.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:21.065474+0000 mon.c (mon.2) 115 : audit 1 from='client.? 172.21.15.117:0/2687162500' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:24.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: cluster 2024-03-04T15:54:22.405427+0000 mon.a (mon.0) 917 : cluster 0 osdmap e84: 8 total, 8 up, 8 in 2024-03-04T15:54:24.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: cluster 2024-03-04T15:54:22.411610+0000 mon.a (mon.0) 918 : cluster 3 Health check update: 21 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:24.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:22.418227+0000 mon.a (mon.0) 919 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-03-04T15:54:24.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:22.423026+0000 mon.c (mon.2) 116 : audit 1 from='client.? 172.21.15.117:0/1028889172' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-04T15:54:24.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:22.429035+0000 mon.a (mon.0) 920 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:24.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:22.433503+0000 mon.a (mon.0) 921 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:24.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:22.434681+0000 mon.a (mon.0) 922 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi117-63915-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:24.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:22.434918+0000 mon.a (mon.0) 923 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:24.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: cluster 2024-03-04T15:54:22.453731+0000 mon.a (mon.0) 924 : cluster 0 osdmap e85: 8 total, 8 up, 8 in 2024-03-04T15:54:24.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:22.455171+0000 mon.c (mon.2) 117 : audit 1 from='client.? 172.21.15.117:0/64238658' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:24.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:22.902293+0000 mon.a (mon.0) 925 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.0"}]: dispatch 2024-03-04T15:54:24.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:22.904797+0000 mon.a (mon.0) 926 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:24.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:22.905308+0000 mon.a (mon.0) 927 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi117-64587-1"}]: dispatch 2024-03-04T15:54:24.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:22.905526+0000 mon.a (mon.0) 928 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.1"}]: dispatch 2024-03-04T15:54:24.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:22.905800+0000 mon.c (mon.2) 118 : audit 1 from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:24.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:22.905880+0000 mon.a (mon.0) 929 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:24.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:22.906786+0000 mon.c (mon.2) 119 : audit 1 from='client.? 172.21.15.117:0/3022824111' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi117-64169-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:24.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:23.343434+0000 mon.a (mon.0) 930 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.2"}]: dispatch 2024-03-04T15:54:24.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:23.344454+0000 mon.a (mon.0) 931 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:24.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:23.345176+0000 mon.a (mon.0) 932 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.3"}]: dispatch 2024-03-04T15:54:24.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:23.345738+0000 mon.c (mon.2) 120 : audit 1 from='client.? 172.21.15.117:0/1728754262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:24.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:23.346920+0000 mon.a (mon.0) 933 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.4"}]: dispatch 2024-03-04T15:54:24.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:23.348342+0000 mon.a (mon.0) 934 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.5"}]: dispatch 2024-03-04T15:54:24.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:23.349865+0000 mon.a (mon.0) 935 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.6"}]: dispatch 2024-03-04T15:54:24.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:23.351218+0000 mon.a (mon.0) 936 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.7"}]: dispatch 2024-03-04T15:54:24.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:23.352584+0000 mon.a (mon.0) 937 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.8"}]: dispatch 2024-03-04T15:54:24.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[28843]: audit 2024-03-04T15:54:23.353760+0000 mon.a (mon.0) 938 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.9"}]: dispatch 2024-03-04T15:54:24.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:22.902293+0000 mon.a (mon.0) 925 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.0"}]: dispatch 2024-03-04T15:54:24.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:22.904797+0000 mon.a (mon.0) 926 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:24.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:22.905308+0000 mon.a (mon.0) 927 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi117-64587-1"}]: dispatch 2024-03-04T15:54:24.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:22.905526+0000 mon.a (mon.0) 928 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.1"}]: dispatch 2024-03-04T15:54:24.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:22.905800+0000 mon.c (mon.2) 118 : audit 1 from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:24.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:22.905880+0000 mon.a (mon.0) 929 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:24.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:22.906786+0000 mon.c (mon.2) 119 : audit 1 from='client.? 172.21.15.117:0/3022824111' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi117-64169-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:24.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:23.343434+0000 mon.a (mon.0) 930 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.2"}]: dispatch 2024-03-04T15:54:24.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:23.344454+0000 mon.a (mon.0) 931 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:24.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:23.345176+0000 mon.a (mon.0) 932 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.3"}]: dispatch 2024-03-04T15:54:24.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:23.345738+0000 mon.c (mon.2) 120 : audit 1 from='client.? 172.21.15.117:0/1728754262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:24.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:23.346920+0000 mon.a (mon.0) 933 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.4"}]: dispatch 2024-03-04T15:54:24.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:23.348342+0000 mon.a (mon.0) 934 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.5"}]: dispatch 2024-03-04T15:54:24.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:23.349865+0000 mon.a (mon.0) 935 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.6"}]: dispatch 2024-03-04T15:54:24.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:23.351218+0000 mon.a (mon.0) 936 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.7"}]: dispatch 2024-03-04T15:54:24.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:23.352584+0000 mon.a (mon.0) 937 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.8"}]: dispatch 2024-03-04T15:54:24.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:23 smithi117 bash[21480]: audit 2024-03-04T15:54:23.353760+0000 mon.a (mon.0) 938 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.9"}]: dispatch 2024-03-04T15:54:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: cluster 2024-03-04T15:54:19.428386+0000 mgr.y (mgr.24428) 172 : cluster 0 pgmap v123: 916 pgs: 284 creating+peering, 356 active+clean, 276 unknown; 590 KiB data, 257 MiB used, 715 GiB / 715 GiB avail; 2.4 KiB/s rd, 16 KiB/s wr, 21 op/s 2024-03-04T15:54:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:20.563061+0000 mon.a (mon.0) 916 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritesmithi117-64587-1"}]': finished 2024-03-04T15:54:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:20.572809+0000 mon.c (mon.2) 112 : audit 1 from='client.? 172.21.15.117:0/705951852' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi117-64587-1"}]: dispatch 2024-03-04T15:54:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:20.574118+0000 mon.c (mon.2) 113 : audit 1 from='client.? 172.21.15.117:0/3022824111' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:20.656440+0000 mon.c (mon.2) 114 : audit 1 from='client.? 172.21.15.117:0/1728754262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:21.065474+0000 mon.c (mon.2) 115 : audit 1 from='client.? 172.21.15.117:0/2687162500' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: cluster 2024-03-04T15:54:22.405427+0000 mon.a (mon.0) 917 : cluster 0 osdmap e84: 8 total, 8 up, 8 in 2024-03-04T15:54:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: cluster 2024-03-04T15:54:22.411610+0000 mon.a (mon.0) 918 : cluster 3 Health check update: 21 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:22.418227+0000 mon.a (mon.0) 919 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-03-04T15:54:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:22.423026+0000 mon.c (mon.2) 116 : audit 1 from='client.? 172.21.15.117:0/1028889172' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-04T15:54:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:22.429035+0000 mon.a (mon.0) 920 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:22.433503+0000 mon.a (mon.0) 921 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:22.434681+0000 mon.a (mon.0) 922 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi117-63915-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:22.434918+0000 mon.a (mon.0) 923 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:24.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: cluster 2024-03-04T15:54:22.453731+0000 mon.a (mon.0) 924 : cluster 0 osdmap e85: 8 total, 8 up, 8 in 2024-03-04T15:54:24.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:22.455171+0000 mon.c (mon.2) 117 : audit 1 from='client.? 172.21.15.117:0/64238658' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:24.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:22.902293+0000 mon.a (mon.0) 925 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.0"}]: dispatch 2024-03-04T15:54:24.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:22.904797+0000 mon.a (mon.0) 926 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:24.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:22.905308+0000 mon.a (mon.0) 927 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi117-64587-1"}]: dispatch 2024-03-04T15:54:24.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:22.905526+0000 mon.a (mon.0) 928 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.1"}]: dispatch 2024-03-04T15:54:24.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:22.905800+0000 mon.c (mon.2) 118 : audit 1 from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:24.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:22.905880+0000 mon.a (mon.0) 929 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:24.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:22.906786+0000 mon.c (mon.2) 119 : audit 1 from='client.? 172.21.15.117:0/3022824111' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi117-64169-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:24.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:23.343434+0000 mon.a (mon.0) 930 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.2"}]: dispatch 2024-03-04T15:54:24.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:23.344454+0000 mon.a (mon.0) 931 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:24.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:23.345176+0000 mon.a (mon.0) 932 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.3"}]: dispatch 2024-03-04T15:54:24.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:23.345738+0000 mon.c (mon.2) 120 : audit 1 from='client.? 172.21.15.117:0/1728754262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:24.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:23.346920+0000 mon.a (mon.0) 933 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.4"}]: dispatch 2024-03-04T15:54:24.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:23.348342+0000 mon.a (mon.0) 934 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.5"}]: dispatch 2024-03-04T15:54:24.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:23.349865+0000 mon.a (mon.0) 935 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.6"}]: dispatch 2024-03-04T15:54:24.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:23.351218+0000 mon.a (mon.0) 936 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.7"}]: dispatch 2024-03-04T15:54:24.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:23.352584+0000 mon.a (mon.0) 937 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.8"}]: dispatch 2024-03-04T15:54:24.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:23 smithi151 bash[23756]: audit 2024-03-04T15:54:23.353760+0000 mon.a (mon.0) 938 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.9"}]: dispatch 2024-03-04T15:54:25.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[28843]: cluster 2024-03-04T15:54:21.432930+0000 mgr.y (mgr.24428) 173 : cluster 0 pgmap v125: 820 pgs: 152 creating+peering, 647 active+clean, 21 unknown; 634 KiB data, 297 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 79 KiB/s wr, 476 op/s 2024-03-04T15:54:25.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[28843]: cluster 2024-03-04T15:54:22.427202+0000 client.admin (client.?) 0 : cluster 1 twoxx 2024-03-04T15:54:25.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[28843]: audit 2024-03-04T15:54:23.820249+0000 mon.a (mon.0) 939 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:25.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[28843]: audit 2024-03-04T15:54:23.821931+0000 mon.a (mon.0) 940 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:25.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[28843]: audit 2024-03-04T15:54:23.822550+0000 mon.a (mon.0) 941 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-04T15:54:25.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[28843]: audit 2024-03-04T15:54:23.823534+0000 mon.a (mon.0) 942 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:25.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[28843]: audit 2024-03-04T15:54:23.823669+0000 mon.a (mon.0) 943 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi117-64587-1"}]': finished 2024-03-04T15:54:25.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[28843]: audit 2024-03-04T15:54:23.843488+0000 mon.a (mon.0) 944 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:54:25.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[28843]: audit 2024-03-04T15:54:23.847185+0000 mon.c (mon.2) 121 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:54:25.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[28843]: audit 2024-03-04T15:54:23.850895+0000 mon.c (mon.2) 122 : audit 1 from='client.? 172.21.15.117:0/2687162500' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:25.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[28843]: audit 2024-03-04T15:54:23.852744+0000 mon.c (mon.2) 123 : audit 1 from='client.? 172.21.15.117:0/2968425926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:25.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[28843]: audit 2024-03-04T15:54:23.888467+0000 mon.c (mon.2) 124 : audit 1 from='client.? 172.21.15.117:0/3837496039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi117-64587-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:25.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[28843]: cluster 2024-03-04T15:54:24.307448+0000 mon.a (mon.0) 945 : cluster 0 osdmap e86: 8 total, 8 up, 8 in 2024-03-04T15:54:25.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[28843]: audit 2024-03-04T15:54:24.309534+0000 mon.a (mon.0) 946 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:25.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[21480]: cluster 2024-03-04T15:54:21.432930+0000 mgr.y (mgr.24428) 173 : cluster 0 pgmap v125: 820 pgs: 152 creating+peering, 647 active+clean, 21 unknown; 634 KiB data, 297 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 79 KiB/s wr, 476 op/s 2024-03-04T15:54:25.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[21480]: cluster 2024-03-04T15:54:22.427202+0000 client.admin (client.?) 0 : cluster 1 twoxx 2024-03-04T15:54:25.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[21480]: audit 2024-03-04T15:54:23.820249+0000 mon.a (mon.0) 939 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:25.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[21480]: audit 2024-03-04T15:54:23.821931+0000 mon.a (mon.0) 940 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:25.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[21480]: audit 2024-03-04T15:54:23.822550+0000 mon.a (mon.0) 941 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-04T15:54:25.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[21480]: audit 2024-03-04T15:54:23.823534+0000 mon.a (mon.0) 942 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:25.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[21480]: audit 2024-03-04T15:54:23.823669+0000 mon.a (mon.0) 943 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi117-64587-1"}]': finished 2024-03-04T15:54:25.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[21480]: audit 2024-03-04T15:54:23.843488+0000 mon.a (mon.0) 944 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:54:25.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[21480]: audit 2024-03-04T15:54:23.847185+0000 mon.c (mon.2) 121 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:54:25.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[21480]: audit 2024-03-04T15:54:23.850895+0000 mon.c (mon.2) 122 : audit 1 from='client.? 172.21.15.117:0/2687162500' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:25.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[21480]: audit 2024-03-04T15:54:23.852744+0000 mon.c (mon.2) 123 : audit 1 from='client.? 172.21.15.117:0/2968425926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:25.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[21480]: audit 2024-03-04T15:54:23.888467+0000 mon.c (mon.2) 124 : audit 1 from='client.? 172.21.15.117:0/3837496039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi117-64587-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:25.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[21480]: cluster 2024-03-04T15:54:24.307448+0000 mon.a (mon.0) 945 : cluster 0 osdmap e86: 8 total, 8 up, 8 in 2024-03-04T15:54:25.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:25 smithi117 bash[21480]: audit 2024-03-04T15:54:24.309534+0000 mon.a (mon.0) 946 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:25.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:25 smithi151 bash[23756]: cluster 2024-03-04T15:54:21.432930+0000 mgr.y (mgr.24428) 173 : cluster 0 pgmap v125: 820 pgs: 152 creating+peering, 647 active+clean, 21 unknown; 634 KiB data, 297 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 79 KiB/s wr, 476 op/s 2024-03-04T15:54:25.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:25 smithi151 bash[23756]: cluster 2024-03-04T15:54:22.427202+0000 client.admin (client.?) 0 : cluster 1 twoxx 2024-03-04T15:54:25.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:25 smithi151 bash[23756]: audit 2024-03-04T15:54:23.820249+0000 mon.a (mon.0) 939 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:25.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:25 smithi151 bash[23756]: audit 2024-03-04T15:54:23.821931+0000 mon.a (mon.0) 940 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:25.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:25 smithi151 bash[23756]: audit 2024-03-04T15:54:23.822550+0000 mon.a (mon.0) 941 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-03-04T15:54:25.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:25 smithi151 bash[23756]: audit 2024-03-04T15:54:23.823534+0000 mon.a (mon.0) 942 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:25.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:25 smithi151 bash[23756]: audit 2024-03-04T15:54:23.823669+0000 mon.a (mon.0) 943 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritesmithi117-64587-1"}]': finished 2024-03-04T15:54:25.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:25 smithi151 bash[23756]: audit 2024-03-04T15:54:23.843488+0000 mon.a (mon.0) 944 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:54:25.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:25 smithi151 bash[23756]: audit 2024-03-04T15:54:23.847185+0000 mon.c (mon.2) 121 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:54:25.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:25 smithi151 bash[23756]: audit 2024-03-04T15:54:23.850895+0000 mon.c (mon.2) 122 : audit 1 from='client.? 172.21.15.117:0/2687162500' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:25.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:25 smithi151 bash[23756]: audit 2024-03-04T15:54:23.852744+0000 mon.c (mon.2) 123 : audit 1 from='client.? 172.21.15.117:0/2968425926' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:25.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:25 smithi151 bash[23756]: audit 2024-03-04T15:54:23.888467+0000 mon.c (mon.2) 124 : audit 1 from='client.? 172.21.15.117:0/3837496039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi117-64587-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:25.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:25 smithi151 bash[23756]: cluster 2024-03-04T15:54:24.307448+0000 mon.a (mon.0) 945 : cluster 0 osdmap e86: 8 total, 8 up, 8 in 2024-03-04T15:54:25.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:25 smithi151 bash[23756]: audit 2024-03-04T15:54:24.309534+0000 mon.a (mon.0) 946 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:26.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:54:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:54:26.679 INFO:tasks.workunit.client.0.smithi117.stdout: handler_error: Running main() from gmock_main.cc 2024-03-04T15:54:26.679 INFO:tasks.workunit.client.0.smithi117.stdout: handler_error: [==========] Running 1 test from 1 test suite. 2024-03-04T15:54:26.679 INFO:tasks.workunit.client.0.smithi117.stdout: handler_error: [----------] Global test environment set-up. 2024-03-04T15:54:26.679 INFO:tasks.workunit.client.0.smithi117.stdout: handler_error: [----------] 1 test from neocls_handler_error 2024-03-04T15:54:26.679 INFO:tasks.workunit.client.0.smithi117.stdout: handler_error: [ RUN ] neocls_handler_error.test_handler_error 2024-03-04T15:54:26.679 INFO:tasks.workunit.client.0.smithi117.stdout: handler_error: [ OK ] neocls_handler_error.test_handler_error (26467 ms) 2024-03-04T15:54:26.680 INFO:tasks.workunit.client.0.smithi117.stdout: handler_error: [----------] 1 test from neocls_handler_error (26467 ms total) 2024-03-04T15:54:26.680 INFO:tasks.workunit.client.0.smithi117.stdout: handler_error: 2024-03-04T15:54:26.680 INFO:tasks.workunit.client.0.smithi117.stdout: handler_error: [----------] Global test environment tear-down 2024-03-04T15:54:26.680 INFO:tasks.workunit.client.0.smithi117.stdout: handler_error: [==========] 1 test from 1 test suite ran. (26467 ms total) 2024-03-04T15:54:26.680 INFO:tasks.workunit.client.0.smithi117.stdout: handler_error: [ PASSED ] 1 test. 2024-03-04T15:54:26.685 INFO:tasks.workunit.client.0.smithi117.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-03-04T15:54:26.685 INFO:tasks.workunit.client.0.smithi117.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-03-04T15:54:26.685 INFO:tasks.workunit.client.0.smithi117.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-03-04T15:54:26.685 INFO:tasks.workunit.client.0.smithi117.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-03-04T15:54:26.685 INFO:tasks.workunit.client.0.smithi117.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (26510 ms) 2024-03-04T15:54:26.685 INFO:tasks.workunit.client.0.smithi117.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (26511 ms total) 2024-03-04T15:54:26.685 INFO:tasks.workunit.client.0.smithi117.stdout: api_cls_remote_reads: 2024-03-04T15:54:26.685 INFO:tasks.workunit.client.0.smithi117.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-03-04T15:54:26.685 INFO:tasks.workunit.client.0.smithi117.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (26511 ms total) 2024-03-04T15:54:26.685 INFO:tasks.workunit.client.0.smithi117.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-03-04T15:54:26.690 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd_pp: Running main() from gmock_main.cc 2024-03-04T15:54:26.690 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-03-04T15:54:26.690 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-03-04T15:54:26.691 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-03-04T15:54:26.691 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-03-04T15:54:26.691 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (78 ms) 2024-03-04T15:54:26.691 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-03-04T15:54:26.691 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (735 ms) 2024-03-04T15:54:26.691 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-03-04T15:54:26.691 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (25813 ms) 2024-03-04T15:54:26.691 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (26626 ms total) 2024-03-04T15:54:26.691 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd_pp: 2024-03-04T15:54:26.691 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-03-04T15:54:26.691 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (26626 ms total) 2024-03-04T15:54:26.691 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-03-04T15:54:26.932 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:54:26.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:22.902612+0000 mgr.y (mgr.24428) 174 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.0"}]: dispatch 2024-03-04T15:54:26.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:22.905720+0000 mgr.y (mgr.24428) 175 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.1"}]: dispatch 2024-03-04T15:54:26.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:23.343808+0000 mgr.y (mgr.24428) 176 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.2"}]: dispatch 2024-03-04T15:54:26.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:23.345465+0000 mgr.y (mgr.24428) 177 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.3"}]: dispatch 2024-03-04T15:54:26.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:23.347150+0000 mgr.y (mgr.24428) 178 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.4"}]: dispatch 2024-03-04T15:54:26.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:23.348611+0000 mgr.y (mgr.24428) 179 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.5"}]: dispatch 2024-03-04T15:54:26.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:23.350080+0000 mgr.y (mgr.24428) 180 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.6"}]: dispatch 2024-03-04T15:54:26.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:23.351425+0000 mgr.y (mgr.24428) 181 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.7"}]: dispatch 2024-03-04T15:54:26.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:23.352787+0000 mgr.y (mgr.24428) 182 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.8"}]: dispatch 2024-03-04T15:54:26.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:23.353967+0000 mgr.y (mgr.24428) 183 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.9"}]: dispatch 2024-03-04T15:54:26.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: cluster 2024-03-04T15:54:23.364279+0000 osd.3 (osd.3) 3 : cluster 0 37.6 deep-scrub starts 2024-03-04T15:54:26.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: cluster 2024-03-04T15:54:23.365591+0000 osd.3 (osd.3) 4 : cluster 0 37.6 deep-scrub ok 2024-03-04T15:54:26.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: cluster 2024-03-04T15:54:23.391633+0000 osd.1 (osd.1) 3 : cluster 0 37.0 deep-scrub starts 2024-03-04T15:54:26.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: cluster 2024-03-04T15:54:23.394162+0000 osd.1 (osd.1) 4 : cluster 0 37.0 deep-scrub ok 2024-03-04T15:54:26.934 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: cluster 2024-03-04T15:54:23.434415+0000 mgr.y (mgr.24428) 184 : cluster 0 pgmap v127: 916 pgs: 152 creating+peering, 615 active+clean, 149 unknown; 634 KiB data, 297 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 82 KiB/s wr, 424 op/s 2024-03-04T15:54:26.934 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: cluster 2024-03-04T15:54:23.456964+0000 osd.2 (osd.2) 3 : cluster 0 37.3 deep-scrub starts 2024-03-04T15:54:26.934 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: cluster 2024-03-04T15:54:23.459681+0000 osd.2 (osd.2) 4 : cluster 0 37.3 deep-scrub ok 2024-03-04T15:54:26.934 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: cluster 2024-03-04T15:54:24.364505+0000 osd.0 (osd.0) 3 : cluster 0 37.2 deep-scrub starts 2024-03-04T15:54:26.934 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: cluster 2024-03-04T15:54:24.368274+0000 osd.0 (osd.0) 4 : cluster 0 37.2 deep-scrub ok 2024-03-04T15:54:26.934 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: cluster 2024-03-04T15:54:24.416547+0000 osd.1 (osd.1) 5 : cluster 0 37.8 deep-scrub starts 2024-03-04T15:54:26.934 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: cluster 2024-03-04T15:54:24.419052+0000 osd.1 (osd.1) 6 : cluster 0 37.8 deep-scrub ok 2024-03-04T15:54:26.934 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.207591+0000 mon.a (mon.0) 947 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:26.934 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.208614+0000 mon.a (mon.0) 948 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-04T15:54:26.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.221650+0000 mon.a (mon.0) 949 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:26.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: cluster 2024-03-04T15:54:25.231653+0000 mon.a (mon.0) 950 : cluster 0 osdmap e87: 8 total, 8 up, 8 in 2024-03-04T15:54:26.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.248401+0000 mon.a (mon.0) 951 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi117-64656-1"}]: dispatch 2024-03-04T15:54:26.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.248970+0000 mon.a (mon.0) 952 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:26.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.252817+0000 mon.a (mon.0) 953 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:26.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.263417+0000 mon.a (mon.0) 954 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi117-64169-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:26.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.264362+0000 mon.a (mon.0) 955 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:26.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.269683+0000 mon.c (mon.2) 125 : audit 1 from='client.? 172.21.15.117:0/3134094376' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:26.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.271342+0000 mon.a (mon.0) 956 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:26.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.272060+0000 mon.a (mon.0) 957 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:26.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.277414+0000 mon.a (mon.0) 958 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi117-64587-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:26.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.281337+0000 mon.c (mon.2) 126 : audit 1 from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi117-63931-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:26.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.281622+0000 mon.c (mon.2) 127 : audit 1 from='client.? 172.21.15.117:0/1728754262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:26.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.290688+0000 mon.c (mon.2) 128 : audit 1 from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:26.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.305978+0000 mon.a (mon.0) 959 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:26.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.311288+0000 mon.a (mon.0) 960 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi117-63931-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:26.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.312672+0000 mon.a (mon.0) 961 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:26.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.321629+0000 mon.a (mon.0) 962 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:26.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.322481+0000 mon.c (mon.2) 129 : audit 1 from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:26.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.323369+0000 mon.a (mon.0) 963 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:26.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.326437+0000 mon.c (mon.2) 130 : audit 1 from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi117-63960-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:26.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.327684+0000 mon.a (mon.0) 964 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:26.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.330859+0000 mon.b (mon.1) 45 : audit 1 from='client.? 172.21.15.117:0/807910892' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:26.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: cluster 2024-03-04T15:54:25.443021+0000 osd.1 (osd.1) 7 : cluster 0 37.9 deep-scrub starts 2024-03-04T15:54:26.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: cluster 2024-03-04T15:54:25.499224+0000 osd.1 (osd.1) 8 : cluster 0 37.9 deep-scrub ok 2024-03-04T15:54:26.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:26 smithi151 bash[23756]: audit 2024-03-04T15:54:25.751801+0000 mon.a (mon.0) 965 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi117-63960-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:22.902612+0000 mgr.y (mgr.24428) 174 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.0"}]: dispatch 2024-03-04T15:54:27.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:22.905720+0000 mgr.y (mgr.24428) 175 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.1"}]: dispatch 2024-03-04T15:54:27.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:23.343808+0000 mgr.y (mgr.24428) 176 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.2"}]: dispatch 2024-03-04T15:54:27.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:23.345465+0000 mgr.y (mgr.24428) 177 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.3"}]: dispatch 2024-03-04T15:54:27.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:23.347150+0000 mgr.y (mgr.24428) 178 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.4"}]: dispatch 2024-03-04T15:54:27.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:23.348611+0000 mgr.y (mgr.24428) 179 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.5"}]: dispatch 2024-03-04T15:54:27.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:23.350080+0000 mgr.y (mgr.24428) 180 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.6"}]: dispatch 2024-03-04T15:54:27.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:23.351425+0000 mgr.y (mgr.24428) 181 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.7"}]: dispatch 2024-03-04T15:54:27.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:23.352787+0000 mgr.y (mgr.24428) 182 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.8"}]: dispatch 2024-03-04T15:54:27.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:23.353967+0000 mgr.y (mgr.24428) 183 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.9"}]: dispatch 2024-03-04T15:54:27.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: cluster 2024-03-04T15:54:23.364279+0000 osd.3 (osd.3) 3 : cluster 0 37.6 deep-scrub starts 2024-03-04T15:54:27.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: cluster 2024-03-04T15:54:23.365591+0000 osd.3 (osd.3) 4 : cluster 0 37.6 deep-scrub ok 2024-03-04T15:54:27.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: cluster 2024-03-04T15:54:23.391633+0000 osd.1 (osd.1) 3 : cluster 0 37.0 deep-scrub starts 2024-03-04T15:54:27.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: cluster 2024-03-04T15:54:23.394162+0000 osd.1 (osd.1) 4 : cluster 0 37.0 deep-scrub ok 2024-03-04T15:54:27.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: cluster 2024-03-04T15:54:23.434415+0000 mgr.y (mgr.24428) 184 : cluster 0 pgmap v127: 916 pgs: 152 creating+peering, 615 active+clean, 149 unknown; 634 KiB data, 297 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 82 KiB/s wr, 424 op/s 2024-03-04T15:54:27.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: cluster 2024-03-04T15:54:23.456964+0000 osd.2 (osd.2) 3 : cluster 0 37.3 deep-scrub starts 2024-03-04T15:54:27.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: cluster 2024-03-04T15:54:23.459681+0000 osd.2 (osd.2) 4 : cluster 0 37.3 deep-scrub ok 2024-03-04T15:54:27.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: cluster 2024-03-04T15:54:24.364505+0000 osd.0 (osd.0) 3 : cluster 0 37.2 deep-scrub starts 2024-03-04T15:54:27.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: cluster 2024-03-04T15:54:24.368274+0000 osd.0 (osd.0) 4 : cluster 0 37.2 deep-scrub ok 2024-03-04T15:54:27.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: cluster 2024-03-04T15:54:24.416547+0000 osd.1 (osd.1) 5 : cluster 0 37.8 deep-scrub starts 2024-03-04T15:54:27.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: cluster 2024-03-04T15:54:24.419052+0000 osd.1 (osd.1) 6 : cluster 0 37.8 deep-scrub ok 2024-03-04T15:54:27.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.207591+0000 mon.a (mon.0) 947 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:27.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.208614+0000 mon.a (mon.0) 948 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-04T15:54:27.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:22.902612+0000 mgr.y (mgr.24428) 174 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.0"}]: dispatch 2024-03-04T15:54:27.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:22.905720+0000 mgr.y (mgr.24428) 175 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.1"}]: dispatch 2024-03-04T15:54:27.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:23.343808+0000 mgr.y (mgr.24428) 176 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.2"}]: dispatch 2024-03-04T15:54:27.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:23.345465+0000 mgr.y (mgr.24428) 177 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.3"}]: dispatch 2024-03-04T15:54:27.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:23.347150+0000 mgr.y (mgr.24428) 178 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.4"}]: dispatch 2024-03-04T15:54:27.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:23.348611+0000 mgr.y (mgr.24428) 179 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.5"}]: dispatch 2024-03-04T15:54:27.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:23.350080+0000 mgr.y (mgr.24428) 180 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.6"}]: dispatch 2024-03-04T15:54:27.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:23.351425+0000 mgr.y (mgr.24428) 181 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.7"}]: dispatch 2024-03-04T15:54:27.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:23.352787+0000 mgr.y (mgr.24428) 182 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.8"}]: dispatch 2024-03-04T15:54:27.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:23.353967+0000 mgr.y (mgr.24428) 183 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.9"}]: dispatch 2024-03-04T15:54:27.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: cluster 2024-03-04T15:54:23.364279+0000 osd.3 (osd.3) 3 : cluster 0 37.6 deep-scrub starts 2024-03-04T15:54:27.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: cluster 2024-03-04T15:54:23.365591+0000 osd.3 (osd.3) 4 : cluster 0 37.6 deep-scrub ok 2024-03-04T15:54:27.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: cluster 2024-03-04T15:54:23.391633+0000 osd.1 (osd.1) 3 : cluster 0 37.0 deep-scrub starts 2024-03-04T15:54:27.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: cluster 2024-03-04T15:54:23.394162+0000 osd.1 (osd.1) 4 : cluster 0 37.0 deep-scrub ok 2024-03-04T15:54:27.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: cluster 2024-03-04T15:54:23.434415+0000 mgr.y (mgr.24428) 184 : cluster 0 pgmap v127: 916 pgs: 152 creating+peering, 615 active+clean, 149 unknown; 634 KiB data, 297 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 82 KiB/s wr, 424 op/s 2024-03-04T15:54:27.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: cluster 2024-03-04T15:54:23.456964+0000 osd.2 (osd.2) 3 : cluster 0 37.3 deep-scrub starts 2024-03-04T15:54:27.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: cluster 2024-03-04T15:54:23.459681+0000 osd.2 (osd.2) 4 : cluster 0 37.3 deep-scrub ok 2024-03-04T15:54:27.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: cluster 2024-03-04T15:54:24.364505+0000 osd.0 (osd.0) 3 : cluster 0 37.2 deep-scrub starts 2024-03-04T15:54:27.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: cluster 2024-03-04T15:54:24.368274+0000 osd.0 (osd.0) 4 : cluster 0 37.2 deep-scrub ok 2024-03-04T15:54:27.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: cluster 2024-03-04T15:54:24.416547+0000 osd.1 (osd.1) 5 : cluster 0 37.8 deep-scrub starts 2024-03-04T15:54:27.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: cluster 2024-03-04T15:54:24.419052+0000 osd.1 (osd.1) 6 : cluster 0 37.8 deep-scrub ok 2024-03-04T15:54:27.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.207591+0000 mon.a (mon.0) 947 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:27.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.208614+0000 mon.a (mon.0) 948 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-04T15:54:27.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.221650+0000 mon.a (mon.0) 949 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:27.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: cluster 2024-03-04T15:54:25.231653+0000 mon.a (mon.0) 950 : cluster 0 osdmap e87: 8 total, 8 up, 8 in 2024-03-04T15:54:27.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.248401+0000 mon.a (mon.0) 951 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi117-64656-1"}]: dispatch 2024-03-04T15:54:27.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.248970+0000 mon.a (mon.0) 952 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:27.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.252817+0000 mon.a (mon.0) 953 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:27.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.263417+0000 mon.a (mon.0) 954 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi117-64169-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.264362+0000 mon.a (mon.0) 955 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:27.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.269683+0000 mon.c (mon.2) 125 : audit 1 from='client.? 172.21.15.117:0/3134094376' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:27.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.271342+0000 mon.a (mon.0) 956 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.272060+0000 mon.a (mon.0) 957 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:27.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.277414+0000 mon.a (mon.0) 958 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi117-64587-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.281337+0000 mon.c (mon.2) 126 : audit 1 from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi117-63931-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.281622+0000 mon.c (mon.2) 127 : audit 1 from='client.? 172.21.15.117:0/1728754262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.074 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.290688+0000 mon.c (mon.2) 128 : audit 1 from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:27.074 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.221650+0000 mon.a (mon.0) 949 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:27.074 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: cluster 2024-03-04T15:54:25.231653+0000 mon.a (mon.0) 950 : cluster 0 osdmap e87: 8 total, 8 up, 8 in 2024-03-04T15:54:27.074 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.248401+0000 mon.a (mon.0) 951 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi117-64656-1"}]: dispatch 2024-03-04T15:54:27.074 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.248970+0000 mon.a (mon.0) 952 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:27.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.252817+0000 mon.a (mon.0) 953 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:27.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.263417+0000 mon.a (mon.0) 954 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi117-64169-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.264362+0000 mon.a (mon.0) 955 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:27.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.269683+0000 mon.c (mon.2) 125 : audit 1 from='client.? 172.21.15.117:0/3134094376' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:27.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.271342+0000 mon.a (mon.0) 956 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.272060+0000 mon.a (mon.0) 957 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:27.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.277414+0000 mon.a (mon.0) 958 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi117-64587-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.281337+0000 mon.c (mon.2) 126 : audit 1 from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi117-63931-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.281622+0000 mon.c (mon.2) 127 : audit 1 from='client.? 172.21.15.117:0/1728754262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.290688+0000 mon.c (mon.2) 128 : audit 1 from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:27.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.305978+0000 mon.a (mon.0) 959 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:27.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.311288+0000 mon.a (mon.0) 960 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi117-63931-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.312672+0000 mon.a (mon.0) 961 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.076 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.321629+0000 mon.a (mon.0) 962 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:27.076 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.322481+0000 mon.c (mon.2) 129 : audit 1 from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:27.076 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.323369+0000 mon.a (mon.0) 963 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:27.076 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.326437+0000 mon.c (mon.2) 130 : audit 1 from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi117-63960-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.076 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.327684+0000 mon.a (mon.0) 964 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:27.076 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.330859+0000 mon.b (mon.1) 45 : audit 1 from='client.? 172.21.15.117:0/807910892' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:27.076 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: cluster 2024-03-04T15:54:25.443021+0000 osd.1 (osd.1) 7 : cluster 0 37.9 deep-scrub starts 2024-03-04T15:54:27.076 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: cluster 2024-03-04T15:54:25.499224+0000 osd.1 (osd.1) 8 : cluster 0 37.9 deep-scrub ok 2024-03-04T15:54:27.076 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[28843]: audit 2024-03-04T15:54:25.751801+0000 mon.a (mon.0) 965 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi117-63960-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.076 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.305978+0000 mon.a (mon.0) 959 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:27.076 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.311288+0000 mon.a (mon.0) 960 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi117-63931-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.076 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.312672+0000 mon.a (mon.0) 961 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.077 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.321629+0000 mon.a (mon.0) 962 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:27.077 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.322481+0000 mon.c (mon.2) 129 : audit 1 from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:27.077 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.323369+0000 mon.a (mon.0) 963 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:27.077 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.326437+0000 mon.c (mon.2) 130 : audit 1 from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi117-63960-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.077 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.327684+0000 mon.a (mon.0) 964 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:27.077 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.330859+0000 mon.b (mon.1) 45 : audit 1 from='client.? 172.21.15.117:0/807910892' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:27.077 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: cluster 2024-03-04T15:54:25.443021+0000 osd.1 (osd.1) 7 : cluster 0 37.9 deep-scrub starts 2024-03-04T15:54:27.077 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: cluster 2024-03-04T15:54:25.499224+0000 osd.1 (osd.1) 8 : cluster 0 37.9 deep-scrub ok 2024-03-04T15:54:27.077 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:26 smithi117 bash[21480]: audit 2024-03-04T15:54:25.751801+0000 mon.a (mon.0) 965 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi117-63960-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.211 INFO:tasks.workunit.client.0.smithi117.stdout:915-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:27.211 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:20.562720+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:27.211 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.208614+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-03-04T15:54:27.211 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.221650+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:27.211 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.248401+0000 mon.a [INF] from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi117-64656-1"}]: dispatch 2024-03-04T15:54:27.211 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.248970+0000 mon.a [INF] from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:27.212 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.252817+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:27.212 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.263417+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi117-64169-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.212 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.264362+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:27.212 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.269683+0000 mon.c [INF] from='client.? 172.21.15.117:0/3134094376' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:27.212 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.271342+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.212 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.272060+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:27.212 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.277414+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi117-64587-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.212 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.281337+0000 mon.c [INF] from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi117-63931-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.212 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.281622+0000 mon.c [INF] from='client.? 172.21.15.117:0/1728754262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:27.212 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.290688+0000 mon.c [INF] from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:30.162 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.305978+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:30.162 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.311288+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi117-63931-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:30.162 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.312672+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:30.162 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.321629+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:30.162 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.322481+0000 mon.c [INF] from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:30.162 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.323369+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:30.162 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.326437+0000 mon.c [INF] from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi117-63960-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:30.162 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.327684+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:30.162 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.330859+0000 mon.b [INF] from='client.? 172.21.15.117:0/807910892' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:30.162 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:25.751801+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi117-63960-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:30.163 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:26.661880+0000 mon.a [INF] from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi117-64656-1"}]': finished 2024-03-04T15:54:30.163 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:26.662108+0000 mon.a [INF] from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:30.163 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:26.662314+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi117-64169-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.163 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:26.662566+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.163 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:26.662741+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:30.163 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:26.662913+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi117-64587-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.163 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:26.663081+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:30.163 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:26.663237+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi117-63931-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.163 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:26.663380+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.163 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:26.663510+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi117-63960-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.163 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:26.679336+0000 mon.c [INF] from='client.? 172.21.15.117:0/3837496039' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi117-64587-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi117-64587-2"}]: dispatch 2024-03-04T15:54:30.163 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:26.681199+0000 mon.c [INF] from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi117-63960-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:30.163 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:26.683510+0000 mon.c [INF] from='client.? 172.21.15.117:0/3022824111' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi117-64169-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:30.163 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:26.684073+0000 mon.c [INF] from='client.? 172.21.15.117:0/2687162500' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi117-64189-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:30.164 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:26.684701+0000 mon.c [INF] from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi117-63931-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:30.164 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:26.685283+0000 mon.c [INF] from='client.? 172.21.15.117:0/1728754262' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi117-63986-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:30.164 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:26.708223+0000 mon.c [INF] from='client.? 172.21.15.117:0/3134094376' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi117-63923-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-04T15:54:30.164 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:27.155426+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi117-64587-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi117-64587-2"}]: dispatch 2024-03-04T15:54:30.164 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:27.156682+0000 mon.a [INF] from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi117-64656-1"}]: dispatch 2024-03-04T15:54:30.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: cluster 2024-03-04T15:54:25.435561+0000 mgr.y (mgr.24428) 185 : cluster 0 pgmap v130: 940 pgs: 146 creating+peering, 485 active+clean, 309 unknown; 611 KiB data, 297 MiB used, 715 GiB / 715 GiB avail; 6.2 KiB/s rd, 10 KiB/s wr, 406 op/s 2024-03-04T15:54:30.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:26.649017+0000 mon.a (mon.0) 966 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:30.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:26.661880+0000 mon.a (mon.0) 967 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi117-64656-1"}]': finished 2024-03-04T15:54:30.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:26.662108+0000 mon.a (mon.0) 968 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:30.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:26.662314+0000 mon.a (mon.0) 969 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi117-64169-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:26.662566+0000 mon.a (mon.0) 970 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:26.662741+0000 mon.a (mon.0) 971 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:30.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:26.662913+0000 mon.a (mon.0) 972 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi117-64587-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:26.663081+0000 mon.a (mon.0) 973 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:30.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:26.663237+0000 mon.a (mon.0) 974 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi117-63931-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:26.663380+0000 mon.a (mon.0) 975 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:26.663510+0000 mon.a (mon.0) 976 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi117-63960-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:26.679336+0000 mon.c (mon.2) 131 : audit 1 from='client.? 172.21.15.117:0/3837496039' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi117-64587-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi117-64587-2"}]: dispatch 2024-03-04T15:54:30.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:26.681199+0000 mon.c (mon.2) 132 : audit 1 from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi117-63960-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:30.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:26.683510+0000 mon.c (mon.2) 133 : audit 1 from='client.? 172.21.15.117:0/3022824111' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi117-64169-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:30.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:26.684073+0000 mon.c (mon.2) 134 : audit 1 from='client.? 172.21.15.117:0/2687162500' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi117-64189-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:30.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:26.684701+0000 mon.c (mon.2) 135 : audit 1 from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi117-63931-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:30.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:26.685283+0000 mon.c (mon.2) 136 : audit 1 from='client.? 172.21.15.117:0/1728754262' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi117-63986-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:30.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:26.708223+0000 mon.c (mon.2) 137 : audit 1 from='client.? 172.21.15.117:0/3134094376' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi117-63923-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-04T15:54:30.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: cluster 2024-03-04T15:54:27.151730+0000 mon.a (mon.0) 977 : cluster 0 osdmap e88: 8 total, 8 up, 8 in 2024-03-04T15:54:30.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:27.155426+0000 mon.a (mon.0) 978 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi117-64587-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi117-64587-2"}]: dispatch 2024-03-04T15:54:30.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:27.156682+0000 mon.a (mon.0) 979 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi117-64656-1"}]: dispatch 2024-03-04T15:54:30.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:27.158048+0000 mon.a (mon.0) 980 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi117-63960-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:30.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:27.159416+0000 mon.a (mon.0) 981 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi117-64169-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:30.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:27.160867+0000 mon.a (mon.0) 982 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi117-64189-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:30.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:27.161372+0000 mon.a (mon.0) 983 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-4", "tierpool": "test-rados-api-smithi117-64061-4-cache"}]: dispatch 2024-03-04T15:54:30.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:27.161824+0000 mon.a (mon.0) 984 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi117-63931-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:30.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:27.162354+0000 mon.a (mon.0) 985 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi117-63986-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:30.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:27.163753+0000 mon.a (mon.0) 986 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi117-63923-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-04T15:54:30.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: audit 2024-03-04T15:54:27.214093+0000 mon.c (mon.2) 138 : audit 1 from='client.? 172.21.15.117:0/1028889172' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-04T15:54:30.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:30 smithi151 bash[23756]: cluster 2024-03-04T15:54:27.572168+0000 mon.a (mon.0) 987 : cluster 3 Health check update: 14 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: cluster 2024-03-04T15:54:25.435561+0000 mgr.y (mgr.24428) 185 : cluster 0 pgmap v130: 940 pgs: 146 creating+peering, 485 active+clean, 309 unknown; 611 KiB data, 297 MiB used, 715 GiB / 715 GiB avail; 6.2 KiB/s rd, 10 KiB/s wr, 406 op/s 2024-03-04T15:54:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:26.649017+0000 mon.a (mon.0) 966 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:26.661880+0000 mon.a (mon.0) 967 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi117-64656-1"}]': finished 2024-03-04T15:54:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:26.662108+0000 mon.a (mon.0) 968 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:26.662314+0000 mon.a (mon.0) 969 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi117-64169-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:26.662566+0000 mon.a (mon.0) 970 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:26.662741+0000 mon.a (mon.0) 971 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:26.662913+0000 mon.a (mon.0) 972 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi117-64587-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:26.663081+0000 mon.a (mon.0) 973 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:26.663237+0000 mon.a (mon.0) 974 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi117-63931-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:26.663380+0000 mon.a (mon.0) 975 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:26.663510+0000 mon.a (mon.0) 976 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi117-63960-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:26.679336+0000 mon.c (mon.2) 131 : audit 1 from='client.? 172.21.15.117:0/3837496039' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi117-64587-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi117-64587-2"}]: dispatch 2024-03-04T15:54:30.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: cluster 2024-03-04T15:54:25.435561+0000 mgr.y (mgr.24428) 185 : cluster 0 pgmap v130: 940 pgs: 146 creating+peering, 485 active+clean, 309 unknown; 611 KiB data, 297 MiB used, 715 GiB / 715 GiB avail; 6.2 KiB/s rd, 10 KiB/s wr, 406 op/s 2024-03-04T15:54:30.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:26.649017+0000 mon.a (mon.0) 966 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:30.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:26.661880+0000 mon.a (mon.0) 967 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectssmithi117-64656-1"}]': finished 2024-03-04T15:54:30.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:26.662108+0000 mon.a (mon.0) 968 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:30.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:26.662314+0000 mon.a (mon.0) 969 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi117-64169-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:26.662566+0000 mon.a (mon.0) 970 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:26.662741+0000 mon.a (mon.0) 971 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:30.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:26.662913+0000 mon.a (mon.0) 972 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReadOpsmithi117-64587-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:26.663081+0000 mon.a (mon.0) 973 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi117-63923-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:30.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:26.663237+0000 mon.a (mon.0) 974 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi117-63931-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:26.663380+0000 mon.a (mon.0) 975 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:26.663510+0000 mon.a (mon.0) 976 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi117-63960-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:30.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:26.679336+0000 mon.c (mon.2) 131 : audit 1 from='client.? 172.21.15.117:0/3837496039' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi117-64587-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi117-64587-2"}]: dispatch 2024-03-04T15:54:30.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:26.681199+0000 mon.c (mon.2) 132 : audit 1 from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi117-63960-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:30.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:26.683510+0000 mon.c (mon.2) 133 : audit 1 from='client.? 172.21.15.117:0/3022824111' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi117-64169-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:30.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:26.684073+0000 mon.c (mon.2) 134 : audit 1 from='client.? 172.21.15.117:0/2687162500' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi117-64189-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:30.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:26.684701+0000 mon.c (mon.2) 135 : audit 1 from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi117-63931-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:30.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:26.685283+0000 mon.c (mon.2) 136 : audit 1 from='client.? 172.21.15.117:0/1728754262' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi117-63986-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:30.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:26.708223+0000 mon.c (mon.2) 137 : audit 1 from='client.? 172.21.15.117:0/3134094376' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi117-63923-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-04T15:54:30.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: cluster 2024-03-04T15:54:27.151730+0000 mon.a (mon.0) 977 : cluster 0 osdmap e88: 8 total, 8 up, 8 in 2024-03-04T15:54:30.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:27.155426+0000 mon.a (mon.0) 978 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi117-64587-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi117-64587-2"}]: dispatch 2024-03-04T15:54:30.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:27.156682+0000 mon.a (mon.0) 979 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi117-64656-1"}]: dispatch 2024-03-04T15:54:30.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:27.158048+0000 mon.a (mon.0) 980 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi117-63960-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:30.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:27.159416+0000 mon.a (mon.0) 981 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi117-64169-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:30.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:27.160867+0000 mon.a (mon.0) 982 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi117-64189-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:30.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:26.681199+0000 mon.c (mon.2) 132 : audit 1 from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi117-63960-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:30.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:26.683510+0000 mon.c (mon.2) 133 : audit 1 from='client.? 172.21.15.117:0/3022824111' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi117-64169-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:30.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:26.684073+0000 mon.c (mon.2) 134 : audit 1 from='client.? 172.21.15.117:0/2687162500' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi117-64189-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:30.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:26.684701+0000 mon.c (mon.2) 135 : audit 1 from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi117-63931-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:30.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:26.685283+0000 mon.c (mon.2) 136 : audit 1 from='client.? 172.21.15.117:0/1728754262' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi117-63986-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:30.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:26.708223+0000 mon.c (mon.2) 137 : audit 1 from='client.? 172.21.15.117:0/3134094376' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi117-63923-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-04T15:54:30.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: cluster 2024-03-04T15:54:27.151730+0000 mon.a (mon.0) 977 : cluster 0 osdmap e88: 8 total, 8 up, 8 in 2024-03-04T15:54:30.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:27.155426+0000 mon.a (mon.0) 978 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReadOpsmithi117-64587-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi117-64587-2"}]: dispatch 2024-03-04T15:54:30.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:27.156682+0000 mon.a (mon.0) 979 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi117-64656-1"}]: dispatch 2024-03-04T15:54:30.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:27.158048+0000 mon.a (mon.0) 980 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi117-63960-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:30.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:27.159416+0000 mon.a (mon.0) 981 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi117-64169-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:30.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:27.160867+0000 mon.a (mon.0) 982 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi117-64189-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:30.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:27.161372+0000 mon.a (mon.0) 983 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-4", "tierpool": "test-rados-api-smithi117-64061-4-cache"}]: dispatch 2024-03-04T15:54:30.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:27.161824+0000 mon.a (mon.0) 984 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi117-63931-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:30.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:27.162354+0000 mon.a (mon.0) 985 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi117-63986-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:30.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:27.163753+0000 mon.a (mon.0) 986 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi117-63923-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-04T15:54:30.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: audit 2024-03-04T15:54:27.214093+0000 mon.c (mon.2) 138 : audit 1 from='client.? 172.21.15.117:0/1028889172' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-04T15:54:30.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[28843]: cluster 2024-03-04T15:54:27.572168+0000 mon.a (mon.0) 987 : cluster 3 Health check update: 14 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:30.574 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:27.161372+0000 mon.a (mon.0) 983 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-4", "tierpool": "test-rados-api-smithi117-64061-4-cache"}]: dispatch 2024-03-04T15:54:30.574 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:27.161824+0000 mon.a (mon.0) 984 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi117-63931-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:30.574 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:27.162354+0000 mon.a (mon.0) 985 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi117-63986-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:30.574 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:27.163753+0000 mon.a (mon.0) 986 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi117-63923-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-04T15:54:30.574 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: audit 2024-03-04T15:54:27.214093+0000 mon.c (mon.2) 138 : audit 1 from='client.? 172.21.15.117:0/1028889172' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-04T15:54:30.574 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:30 smithi117 bash[21480]: cluster 2024-03-04T15:54:27.572168+0000 mon.a (mon.0) 987 : cluster 3 Health check update: 14 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:31.067 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:27.158048+000 api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-03-04T15:54:31.068 INFO:tasks.workunit.client.0.smithi117.stdout: api_service_pp: [----------] Global test environment set-up. 2024-03-04T15:54:31.068 INFO:tasks.workunit.client.0.smithi117.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-03-04T15:54:31.068 INFO:tasks.workunit.client.0.smithi117.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-03-04T15:54:31.068 INFO:tasks.workunit.client.0.smithi117.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (5062 ms) 2024-03-04T15:54:31.068 INFO:tasks.workunit.client.0.smithi117.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-03-04T15:54:31.068 INFO:tasks.workunit.client.0.smithi117.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (84 ms) 2024-03-04T15:54:31.068 INFO:tasks.workunit.client.0.smithi117.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-03-04T15:54:31.068 INFO:tasks.workunit.client.0.smithi117.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20054 ms) 2024-03-04T15:54:31.068 INFO:tasks.workunit.client.0.smithi117.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-03-04T15:54:31.068 INFO:tasks.workunit.client.0.smithi117.stdout: api_service_pp: attempt 0 of 20 2024-03-04T15:54:31.068 INFO:tasks.workunit.client.0.smithi117.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (5765 ms) 2024-03-04T15:54:31.068 INFO:tasks.workunit.client.0.smithi117.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (30965 ms total) 2024-03-04T15:54:31.068 INFO:tasks.workunit.client.0.smithi117.stdout: api_service_pp: 2024-03-04T15:54:31.068 INFO:tasks.workunit.client.0.smithi117.stdout: api_service_pp: [----------] Global test environment tear-down 2024-03-04T15:54:31.069 INFO:tasks.workunit.client.0.smithi117.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (30965 ms total) 2024-03-04T15:54:31.069 INFO:tasks.workunit.client.0.smithi117.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-03-04T15:54:31.521 INFO:tasks.workunit.client.0.smithi117.stdout:0 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi117-63960-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:31.521 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:27.159416+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi117-64169-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:31.521 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:27.160867+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi117-64189-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:31.522 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:27.161372+0000 mon.a [INF] from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-4", "tierpool": "test-rados-api-smithi117-64061-4-cache"}]: dispatch 2024-03-04T15:54:31.522 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:27.161824+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi117-63931-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:31.522 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:27.162354+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi117-63986-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:31.522 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:27.163753+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi117-63923-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-03-04T15:54:31.522 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:27.214093+0000 mon.c [INF] from='client.? 172.21.15.117:0/1028889172' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-04T15:54:31.522 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:27.572168+0000 mon.a [WRN] Health check update: 14 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:31.522 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:27.214278+0000 client.admin [INF] threexx 2024-03-04T15:54:31.522 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:27.579680+0000 mon.a [INF] from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi117-64656-1"}]': finished 2024-03-04T15:54:31.522 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:27.579740+0000 mon.a [INF] from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-4", "tierpool": "test-rados-api-smithi117-64061-4-cache"}]': finished 2024-03-04T15:54:31.522 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:27.579788+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi117-63923-3", "field": "max_bytes", "val": "4096"}]': finished 2024-03-04T15:54:31.522 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:27.634872+0000 mon.c [INF] from='client.? 172.21.15.117:0/3879512807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi117-64656-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:31.522 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:30.172016+0000 mon.a [INF] from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-4", "overlaypool": "test-rados-api-smithi117-64061-4-cache"}]: dispatch 2024-03-04T15:54:31.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: cluster 2024-03-04T15:54:23.305036+0000 osd.5 (osd.5) 3 : cluster 0 37.1 deep-scrub starts 2024-03-04T15:54:31.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: cluster 2024-03-04T15:54:23.358230+0000 osd.5 (osd.5) 4 : cluster 0 37.1 deep-scrub ok 2024-03-04T15:54:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: cluster 2024-03-04T15:54:23.376642+0000 osd.4 (osd.4) 3 : cluster 0 37.4 deep-scrub starts 2024-03-04T15:54:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: cluster 2024-03-04T15:54:23.378858+0000 osd.4 (osd.4) 4 : cluster 0 37.4 deep-scrub ok 2024-03-04T15:54:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: cluster 2024-03-04T15:54:23.980822+0000 osd.6 (osd.6) 3 : cluster 0 37.7 deep-scrub starts 2024-03-04T15:54:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: cluster 2024-03-04T15:54:23.983377+0000 osd.6 (osd.6) 4 : cluster 0 37.7 deep-scrub ok 2024-03-04T15:54:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: cluster 2024-03-04T15:54:24.295334+0000 osd.5 (osd.5) 5 : cluster 0 37.5 deep-scrub starts 2024-03-04T15:54:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: cluster 2024-03-04T15:54:24.297801+0000 osd.5 (osd.5) 6 : cluster 0 37.5 deep-scrub ok 2024-03-04T15:54:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: audit 2024-03-04T15:54:26.476859+0000 mgr.y (mgr.24428) 186 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:54:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: cluster 2024-03-04T15:54:27.214278+0000 client.admin (client.?) 0 : cluster 1 threexx 2024-03-04T15:54:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: cluster 2024-03-04T15:54:27.436620+0000 mgr.y (mgr.24428) 187 : cluster 0 pgmap v132: 876 pgs: 305 creating+peering, 475 active+clean, 96 unknown; 145 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 9.6 MiB/s rd, 9 op/s 2024-03-04T15:54:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: audit 2024-03-04T15:54:27.579680+0000 mon.a (mon.0) 988 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi117-64656-1"}]': finished 2024-03-04T15:54:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: audit 2024-03-04T15:54:27.579740+0000 mon.a (mon.0) 989 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-4", "tierpool": "test-rados-api-smithi117-64061-4-cache"}]': finished 2024-03-04T15:54:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: audit 2024-03-04T15:54:27.579788+0000 mon.a (mon.0) 990 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi117-63923-3", "field": "max_bytes", "val": "4096"}]': finished 2024-03-04T15:54:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: audit 2024-03-04T15:54:27.634872+0000 mon.c (mon.2) 139 : audit 1 from='client.? 172.21.15.117:0/3879512807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi117-64656-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: cluster 2024-03-04T15:54:29.437987+0000 mgr.y (mgr.24428) 188 : cluster 0 pgmap v134: 908 pgs: 305 creating+peering, 475 active+clean, 128 unknown; 145 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 27 MiB/s wr, 19 op/s 2024-03-04T15:54:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: cluster 2024-03-04T15:54:30.149216+0000 mon.a (mon.0) 991 : cluster 0 osdmap e89: 8 total, 8 up, 8 in 2024-03-04T15:54:31.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: audit 2024-03-04T15:54:30.171676+0000 mon.a (mon.0) 992 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:31.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: audit 2024-03-04T15:54:30.172016+0000 mon.a (mon.0) 993 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-4", "overlaypool": "test-rados-api-smithi117-64061-4-cache"}]: dispatch 2024-03-04T15:54:31.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: audit 2024-03-04T15:54:30.172911+0000 mon.a (mon.0) 994 : audit 1 from='client.? 172.21.15.117:0/208083896' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:31.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: cluster 2024-03-04T15:54:23.305036+0000 osd.5 (osd.5) 3 : cluster 0 37.1 deep-scrub starts 2024-03-04T15:54:31.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: cluster 2024-03-04T15:54:23.358230+0000 osd.5 (osd.5) 4 : cluster 0 37.1 deep-scrub ok 2024-03-04T15:54:31.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: cluster 2024-03-04T15:54:23.376642+0000 osd.4 (osd.4) 3 : cluster 0 37.4 deep-scrub starts 2024-03-04T15:54:31.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: cluster 2024-03-04T15:54:23.378858+0000 osd.4 (osd.4) 4 : cluster 0 37.4 deep-scrub ok 2024-03-04T15:54:31.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: cluster 2024-03-04T15:54:23.980822+0000 osd.6 (osd.6) 3 : cluster 0 37.7 deep-scrub starts 2024-03-04T15:54:31.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: cluster 2024-03-04T15:54:23.983377+0000 osd.6 (osd.6) 4 : cluster 0 37.7 deep-scrub ok 2024-03-04T15:54:31.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: cluster 2024-03-04T15:54:24.295334+0000 osd.5 (osd.5) 5 : cluster 0 37.5 deep-scrub starts 2024-03-04T15:54:31.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: cluster 2024-03-04T15:54:24.297801+0000 osd.5 (osd.5) 6 : cluster 0 37.5 deep-scrub ok 2024-03-04T15:54:31.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: audit 2024-03-04T15:54:26.476859+0000 mgr.y (mgr.24428) 186 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:54:31.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: cluster 2024-03-04T15:54:27.214278+0000 client.admin (client.?) 0 : cluster 1 threexx 2024-03-04T15:54:31.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: cluster 2024-03-04T15:54:27.436620+0000 mgr.y (mgr.24428) 187 : cluster 0 pgmap v132: 876 pgs: 305 creating+peering, 475 active+clean, 96 unknown; 145 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 9.6 MiB/s rd, 9 op/s 2024-03-04T15:54:31.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: audit 2024-03-04T15:54:27.579680+0000 mon.a (mon.0) 988 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi117-64656-1"}]': finished 2024-03-04T15:54:31.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: audit 2024-03-04T15:54:27.579740+0000 mon.a (mon.0) 989 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-4", "tierpool": "test-rados-api-smithi117-64061-4-cache"}]': finished 2024-03-04T15:54:31.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: audit 2024-03-04T15:54:27.579788+0000 mon.a (mon.0) 990 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi117-63923-3", "field": "max_bytes", "val": "4096"}]': finished 2024-03-04T15:54:31.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: audit 2024-03-04T15:54:27.634872+0000 mon.c (mon.2) 139 : audit 1 from='client.? 172.21.15.117:0/3879512807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi117-64656-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:31.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: cluster 2024-03-04T15:54:29.437987+0000 mgr.y (mgr.24428) 188 : cluster 0 pgmap v134: 908 pgs: 305 creating+peering, 475 active+clean, 128 unknown; 145 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 27 MiB/s wr, 19 op/s 2024-03-04T15:54:31.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: cluster 2024-03-04T15:54:30.149216+0000 mon.a (mon.0) 991 : cluster 0 osdmap e89: 8 total, 8 up, 8 in 2024-03-04T15:54:31.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: audit 2024-03-04T15:54:30.171676+0000 mon.a (mon.0) 992 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:31.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: audit 2024-03-04T15:54:30.172016+0000 mon.a (mon.0) 993 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-4", "overlaypool": "test-rados-api-smithi117-64061-4-cache"}]: dispatch 2024-03-04T15:54:31.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: audit 2024-03-04T15:54:30.172911+0000 mon.a (mon.0) 994 : audit 1 from='client.? 172.21.15.117:0/208083896' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:31.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: audit 2024-03-04T15:54:30.173664+0000 mon.a (mon.0) 995 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi117-64587-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi117-64587-2"}]': finished 2024-03-04T15:54:31.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: audit 2024-03-04T15:54:30.173746+0000 mon.a (mon.0) 996 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi117-63960-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi117-63960-10"}]': finished 2024-03-04T15:54:31.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: audit 2024-03-04T15:54:30.173803+0000 mon.a (mon.0) 997 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi117-64169-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi117-64169-7"}]': finished 2024-03-04T15:54:31.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: audit 2024-03-04T15:54:30.173856+0000 mon.a (mon.0) 998 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi117-64189-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi117-64189-7"}]': finished 2024-03-04T15:54:31.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: audit 2024-03-04T15:54:30.173904+0000 mon.a (mon.0) 999 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi117-63931-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi117-63931-16"}]': finished 2024-03-04T15:54:31.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: audit 2024-03-04T15:54:30.173953+0000 mon.a (mon.0) 1000 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi117-63986-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi117-63986-10"}]': finished 2024-03-04T15:54:31.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: cluster 2024-03-04T15:54:30.179389+0000 mon.a (mon.0) 1001 : cluster 0 osdmap e90: 8 total, 8 up, 8 in 2024-03-04T15:54:31.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: audit 2024-03-04T15:54:31.053875+0000 mon.a (mon.0) 1002 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-04T15:54:31.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: audit 2024-03-04T15:54:31.058475+0000 mon.a (mon.0) 1003 : audit 0 from='client.? 172.21.15.117:0/1871382937' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-04T15:54:31.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[28843]: audit 2024-03-04T15:54:31.066923+0000 mon.a (mon.0) 1004 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi117-64656-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:31.823 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: audit 2024-03-04T15:54:30.173664+0000 mon.a (mon.0) 995 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi117-64587-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi117-64587-2"}]': finished 2024-03-04T15:54:31.823 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: audit 2024-03-04T15:54:30.173746+0000 mon.a (mon.0) 996 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi117-63960-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi117-63960-10"}]': finished 2024-03-04T15:54:31.823 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: audit 2024-03-04T15:54:30.173803+0000 mon.a (mon.0) 997 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi117-64169-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi117-64169-7"}]': finished 2024-03-04T15:54:31.823 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: audit 2024-03-04T15:54:30.173856+0000 mon.a (mon.0) 998 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi117-64189-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi117-64189-7"}]': finished 2024-03-04T15:54:31.823 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: audit 2024-03-04T15:54:30.173904+0000 mon.a (mon.0) 999 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi117-63931-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi117-63931-16"}]': finished 2024-03-04T15:54:31.823 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: audit 2024-03-04T15:54:30.173953+0000 mon.a (mon.0) 1000 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi117-63986-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi117-63986-10"}]': finished 2024-03-04T15:54:31.823 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: cluster 2024-03-04T15:54:30.179389+0000 mon.a (mon.0) 1001 : cluster 0 osdmap e90: 8 total, 8 up, 8 in 2024-03-04T15:54:31.823 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: audit 2024-03-04T15:54:31.053875+0000 mon.a (mon.0) 1002 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-04T15:54:31.823 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: audit 2024-03-04T15:54:31.058475+0000 mon.a (mon.0) 1003 : audit 0 from='client.? 172.21.15.117:0/1871382937' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-04T15:54:31.823 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:31 smithi117 bash[21480]: audit 2024-03-04T15:54:31.066923+0000 mon.a (mon.0) 1004 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi117-64656-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:31.934 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: cluster 2024-03-04T15:54:23.305036+0000 osd.5 (osd.5) 3 : cluster 0 37.1 deep-scrub starts 2024-03-04T15:54:31.934 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: cluster 2024-03-04T15:54:23.358230+0000 osd.5 (osd.5) 4 : cluster 0 37.1 deep-scrub ok 2024-03-04T15:54:31.934 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: cluster 2024-03-04T15:54:23.376642+0000 osd.4 (osd.4) 3 : cluster 0 37.4 deep-scrub starts 2024-03-04T15:54:31.934 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: cluster 2024-03-04T15:54:23.378858+0000 osd.4 (osd.4) 4 : cluster 0 37.4 deep-scrub ok 2024-03-04T15:54:31.934 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: cluster 2024-03-04T15:54:23.980822+0000 osd.6 (osd.6) 3 : cluster 0 37.7 deep-scrub starts 2024-03-04T15:54:31.934 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: cluster 2024-03-04T15:54:23.983377+0000 osd.6 (osd.6) 4 : cluster 0 37.7 deep-scrub ok 2024-03-04T15:54:31.934 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: cluster 2024-03-04T15:54:24.295334+0000 osd.5 (osd.5) 5 : cluster 0 37.5 deep-scrub starts 2024-03-04T15:54:31.934 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: cluster 2024-03-04T15:54:24.297801+0000 osd.5 (osd.5) 6 : cluster 0 37.5 deep-scrub ok 2024-03-04T15:54:31.934 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: audit 2024-03-04T15:54:26.476859+0000 mgr.y (mgr.24428) 186 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:54:31.934 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: cluster 2024-03-04T15:54:27.214278+0000 client.admin (client.?) 0 : cluster 1 threexx 2024-03-04T15:54:31.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: cluster 2024-03-04T15:54:27.436620+0000 mgr.y (mgr.24428) 187 : cluster 0 pgmap v132: 876 pgs: 305 creating+peering, 475 active+clean, 96 unknown; 145 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 9.6 MiB/s rd, 9 op/s 2024-03-04T15:54:31.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: audit 2024-03-04T15:54:27.579680+0000 mon.a (mon.0) 988 : audit 1 from='client.? 172.21.15.117:0/2947554' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectssmithi117-64656-1"}]': finished 2024-03-04T15:54:31.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: audit 2024-03-04T15:54:27.579740+0000 mon.a (mon.0) 989 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-4", "tierpool": "test-rados-api-smithi117-64061-4-cache"}]': finished 2024-03-04T15:54:31.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: audit 2024-03-04T15:54:27.579788+0000 mon.a (mon.0) 990 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi117-63923-3", "field": "max_bytes", "val": "4096"}]': finished 2024-03-04T15:54:31.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: audit 2024-03-04T15:54:27.634872+0000 mon.c (mon.2) 139 : audit 1 from='client.? 172.21.15.117:0/3879512807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi117-64656-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:31.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: cluster 2024-03-04T15:54:29.437987+0000 mgr.y (mgr.24428) 188 : cluster 0 pgmap v134: 908 pgs: 305 creating+peering, 475 active+clean, 128 unknown; 145 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 8.6 MiB/s rd, 27 MiB/s wr, 19 op/s 2024-03-04T15:54:31.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: cluster 2024-03-04T15:54:30.149216+0000 mon.a (mon.0) 991 : cluster 0 osdmap e89: 8 total, 8 up, 8 in 2024-03-04T15:54:31.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: audit 2024-03-04T15:54:30.171676+0000 mon.a (mon.0) 992 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:31.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: audit 2024-03-04T15:54:30.172016+0000 mon.a (mon.0) 993 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-4", "overlaypool": "test-rados-api-smithi117-64061-4-cache"}]: dispatch 2024-03-04T15:54:31.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: audit 2024-03-04T15:54:30.172911+0000 mon.a (mon.0) 994 : audit 1 from='client.? 172.21.15.117:0/208083896' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:31.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: audit 2024-03-04T15:54:30.173664+0000 mon.a (mon.0) 995 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi117-64587-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi117-64587-2"}]': finished 2024-03-04T15:54:31.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: audit 2024-03-04T15:54:30.173746+0000 mon.a (mon.0) 996 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi117-63960-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi117-63960-10"}]': finished 2024-03-04T15:54:31.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: audit 2024-03-04T15:54:30.173803+0000 mon.a (mon.0) 997 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi117-64169-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi117-64169-7"}]': finished 2024-03-04T15:54:31.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: audit 2024-03-04T15:54:30.173856+0000 mon.a (mon.0) 998 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi117-64189-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi117-64189-7"}]': finished 2024-03-04T15:54:31.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: audit 2024-03-04T15:54:30.173904+0000 mon.a (mon.0) 999 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi117-63931-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi117-63931-16"}]': finished 2024-03-04T15:54:31.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: audit 2024-03-04T15:54:30.173953+0000 mon.a (mon.0) 1000 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi117-63986-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi117-63986-10"}]': finished 2024-03-04T15:54:31.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: cluster 2024-03-04T15:54:30.179389+0000 mon.a (mon.0) 1001 : cluster 0 osdmap e90: 8 total, 8 up, 8 in 2024-03-04T15:54:31.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: audit 2024-03-04T15:54:31.053875+0000 mon.a (mon.0) 1002 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-04T15:54:31.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: audit 2024-03-04T15:54:31.058475+0000 mon.a (mon.0) 1003 : audit 0 from='client.? 172.21.15.117:0/1871382937' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-04T15:54:31.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:31 smithi151 bash[23756]: audit 2024-03-04T15:54:31.066923+0000 mon.a (mon.0) 1004 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi117-64656-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:32.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.058682+0000 mgr.y (mgr.24428) 189 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-04T15:54:32.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: cluster 2024-03-04T15:54:31.445644+0000 mgr.y (mgr.24428) 190 : cluster 0 pgmap v136: 956 pgs: 299 creating+peering, 657 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 8.0 MiB/s rd, 25 MiB/s wr, 26 op/s 2024-03-04T15:54:32.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.520222+0000 mon.a (mon.0) 1005 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:32.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.520862+0000 mon.a (mon.0) 1006 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-03-04T15:54:32.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.528456+0000 mon.a (mon.0) 1007 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:32.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.532947+0000 mon.c (mon.2) 140 : audit 1 from='client.? 172.21.15.117:0/1028889172' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-04T15:54:32.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.535042+0000 mon.a (mon.0) 1008 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-4", "overlaypool": "test-rados-api-smithi117-64061-4-cache"}]': finished 2024-03-04T15:54:32.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.535125+0000 mon.a (mon.0) 1009 : audit 1 from='client.? 172.21.15.117:0/208083896' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:32.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.535200+0000 mon.a (mon.0) 1010 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi117-64656-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:32.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: cluster 2024-03-04T15:54:31.541682+0000 mon.a (mon.0) 1011 : cluster 0 osdmap e91: 8 total, 8 up, 8 in 2024-03-04T15:54:32.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.549290+0000 mon.b (mon.1) 46 : audit 1 from='client.? 172.21.15.117:0/807910892' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:32.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.553249+0000 mon.b (mon.1) 47 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:32.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.556655+0000 mon.a (mon.0) 1012 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-4-cache", "mode": "writeback"}]: dispatch 2024-03-04T15:54:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: cluster 2024-03-04T15:54:31.556713+0000 client.admin (client.?) 0 : cluster 1 fourxx 2024-03-04T15:54:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.557444+0000 mon.a (mon.0) 1013 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-04T15:54:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.561318+0000 mon.c (mon.2) 141 : audit 1 from='client.? 172.21.15.117:0/3879512807' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi117-64656-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi117-64656-2"}]: dispatch 2024-03-04T15:54:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.567127+0000 mon.a (mon.0) 1014 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi117-64656-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi117-64656-2"}]: dispatch 2024-03-04T15:54:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.567714+0000 mon.c (mon.2) 142 : audit 1 from='client.? 172.21.15.117:0/3140935506' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.568454+0000 mon.a (mon.0) 1015 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.588777+0000 mon.a (mon.0) 1016 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.589994+0000 mon.a (mon.0) 1017 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.592709+0000 mon.c (mon.2) 143 : audit 1 from='client.? 172.21.15.117:0/3288255947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.593789+0000 mon.a (mon.0) 1018 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.595081+0000 mon.c (mon.2) 144 : audit 1 from='client.? 172.21.15.117:0/3288255947' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.595581+0000 mon.a (mon.0) 1019 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:31.597269+0000 mon.c (mon.2) 145 : audit 1 from='client.? 172.21.15.117:0/3288255947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:32.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[21480]: audit 2024-03-04T15:54:32.080303+0000 mon.a (mon.0) 1020 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.058682+0000 mgr.y (mgr.24428) 189 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-04T15:54:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: cluster 2024-03-04T15:54:31.445644+0000 mgr.y (mgr.24428) 190 : cluster 0 pgmap v136: 956 pgs: 299 creating+peering, 657 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 8.0 MiB/s rd, 25 MiB/s wr, 26 op/s 2024-03-04T15:54:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.520222+0000 mon.a (mon.0) 1005 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.520862+0000 mon.a (mon.0) 1006 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-03-04T15:54:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.528456+0000 mon.a (mon.0) 1007 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.532947+0000 mon.c (mon.2) 140 : audit 1 from='client.? 172.21.15.117:0/1028889172' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-04T15:54:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.535042+0000 mon.a (mon.0) 1008 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-4", "overlaypool": "test-rados-api-smithi117-64061-4-cache"}]': finished 2024-03-04T15:54:32.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.535125+0000 mon.a (mon.0) 1009 : audit 1 from='client.? 172.21.15.117:0/208083896' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:32.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.535200+0000 mon.a (mon.0) 1010 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi117-64656-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:32.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: cluster 2024-03-04T15:54:31.541682+0000 mon.a (mon.0) 1011 : cluster 0 osdmap e91: 8 total, 8 up, 8 in 2024-03-04T15:54:32.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.549290+0000 mon.b (mon.1) 46 : audit 1 from='client.? 172.21.15.117:0/807910892' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:32.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.553249+0000 mon.b (mon.1) 47 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:32.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.556655+0000 mon.a (mon.0) 1012 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-4-cache", "mode": "writeback"}]: dispatch 2024-03-04T15:54:32.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: cluster 2024-03-04T15:54:31.556713+0000 client.admin (client.?) 0 : cluster 1 fourxx 2024-03-04T15:54:32.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.557444+0000 mon.a (mon.0) 1013 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-04T15:54:32.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.561318+0000 mon.c (mon.2) 141 : audit 1 from='client.? 172.21.15.117:0/3879512807' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi117-64656-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi117-64656-2"}]: dispatch 2024-03-04T15:54:32.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.567127+0000 mon.a (mon.0) 1014 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi117-64656-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi117-64656-2"}]: dispatch 2024-03-04T15:54:32.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.567714+0000 mon.c (mon.2) 142 : audit 1 from='client.? 172.21.15.117:0/3140935506' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:32.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.568454+0000 mon.a (mon.0) 1015 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:32.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.588777+0000 mon.a (mon.0) 1016 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:32.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.589994+0000 mon.a (mon.0) 1017 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:32.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.592709+0000 mon.c (mon.2) 143 : audit 1 from='client.? 172.21.15.117:0/3288255947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:32.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.593789+0000 mon.a (mon.0) 1018 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:32.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.595081+0000 mon.c (mon.2) 144 : audit 1 from='client.? 172.21.15.117:0/3288255947' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:32.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.595581+0000 mon.a (mon.0) 1019 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:32.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:31.597269+0000 mon.c (mon.2) 145 : audit 1 from='client.? 172.21.15.117:0/3288255947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:32.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:32 smithi117 bash[28843]: audit 2024-03-04T15:54:32.080303+0000 mon.a (mon.0) 1020 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:32.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.058682+0000 mgr.y (mgr.24428) 189 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-03-04T15:54:32.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: cluster 2024-03-04T15:54:31.445644+0000 mgr.y (mgr.24428) 190 : cluster 0 pgmap v136: 956 pgs: 299 creating+peering, 657 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 8.0 MiB/s rd, 25 MiB/s wr, 26 op/s 2024-03-04T15:54:32.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.520222+0000 mon.a (mon.0) 1005 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:32.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.520862+0000 mon.a (mon.0) 1006 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-03-04T15:54:32.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.528456+0000 mon.a (mon.0) 1007 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:32.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.532947+0000 mon.c (mon.2) 140 : audit 1 from='client.? 172.21.15.117:0/1028889172' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-04T15:54:32.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.535042+0000 mon.a (mon.0) 1008 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-4", "overlaypool": "test-rados-api-smithi117-64061-4-cache"}]': finished 2024-03-04T15:54:32.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.535125+0000 mon.a (mon.0) 1009 : audit 1 from='client.? 172.21.15.117:0/208083896' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:32.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.535200+0000 mon.a (mon.0) 1010 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi117-64656-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:32.937 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: cluster 2024-03-04T15:54:31.541682+0000 mon.a (mon.0) 1011 : cluster 0 osdmap e91: 8 total, 8 up, 8 in 2024-03-04T15:54:32.937 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.549290+0000 mon.b (mon.1) 46 : audit 1 from='client.? 172.21.15.117:0/807910892' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:32.937 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.553249+0000 mon.b (mon.1) 47 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:32.937 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.556655+0000 mon.a (mon.0) 1012 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-4-cache", "mode": "writeback"}]: dispatch 2024-03-04T15:54:32.937 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: cluster 2024-03-04T15:54:31.556713+0000 client.admin (client.?) 0 : cluster 1 fourxx 2024-03-04T15:54:32.937 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.557444+0000 mon.a (mon.0) 1013 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-03-04T15:54:32.937 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.561318+0000 mon.c (mon.2) 141 : audit 1 from='client.? 172.21.15.117:0/3879512807' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi117-64656-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi117-64656-2"}]: dispatch 2024-03-04T15:54:32.937 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.567127+0000 mon.a (mon.0) 1014 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi117-64656-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi117-64656-2"}]: dispatch 2024-03-04T15:54:32.937 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.567714+0000 mon.c (mon.2) 142 : audit 1 from='client.? 172.21.15.117:0/3140935506' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:32.937 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.568454+0000 mon.a (mon.0) 1015 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:32.937 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.588777+0000 mon.a (mon.0) 1016 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:32.937 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.589994+0000 mon.a (mon.0) 1017 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:32.937 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.592709+0000 mon.c (mon.2) 143 : audit 1 from='client.? 172.21.15.117:0/3288255947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:32.937 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.593789+0000 mon.a (mon.0) 1018 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:32.937 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.595081+0000 mon.c (mon.2) 144 : audit 1 from='client.? 172.21.15.117:0/3288255947' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:32.938 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.595581+0000 mon.a (mon.0) 1019 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:32.938 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:31.597269+0000 mon.c (mon.2) 145 : audit 1 from='client.? 172.21.15.117:0/3288255947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:32.938 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:32 smithi151 bash[23756]: audit 2024-03-04T15:54:32.080303+0000 mon.a (mon.0) 1020 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:33.029 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:30.172911+0000 mon.a [INF] from='client.? 172.21.15.117:0/208083896' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": open_pools_parallel: process_1_[64501]: starting. 2024-03-04T15:54:33.030 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_1_[64501]: creating pool ceph_test_rados_open_pools_parallel.smithi117-64427 2024-03-04T15:54:33.030 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_1_[64501]: created object 0... 2024-03-04T15:54:33.030 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_1_[64501]: created object 25... 2024-03-04T15:54:33.030 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_1_[64501]: created object 49... 2024-03-04T15:54:33.030 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_1_[64501]: finishing. 2024-03-04T15:54:33.030 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_1_[64501]: shutting down. 2024-03-04T15:54:33.030 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_2_[64502]: starting. 2024-03-04T15:54:33.030 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_2_[64502]: rados_pool_create. 2024-03-04T15:54:33.030 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_2_[64502]: rados_ioctx_create. 2024-03-04T15:54:33.030 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_2_[64502]: shutting down. 2024-03-04T15:54:33.030 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: ******************************* 2024-03-04T15:54:33.030 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_3_[65984]: starting. 2024-03-04T15:54:33.030 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_3_[65984]: creating pool ceph_test_rados_open_pools_parallel.smithi117-64427 2024-03-04T15:54:33.030 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_3_[65984]: created object 0... 2024-03-04T15:54:33.030 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_3_[65984]: created object 25... 2024-03-04T15:54:33.031 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_3_[65984]: created object 49... 2024-03-04T15:54:33.031 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_3_[65984]: finishing. 2024-03-04T15:54:33.031 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_3_[65984]: shutting down. 2024-03-04T15:54:33.031 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: ******************************* 2024-03-04T15:54:33.031 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_4_[65985]: starting. 2024-03-04T15:54:33.031 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_4_[65985]: rados_pool_create. 2024-03-04T15:54:33.031 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_4_[65985]: rados_ioctx_create. 2024-03-04T15:54:33.031 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: process_4_[65985]: shutting down. 2024-03-04T15:54:33.031 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: ******************************* 2024-03-04T15:54:33.031 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: ******************************* 2024-03-04T15:54:33.031 INFO:tasks.workunit.client.0.smithi117.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-03-04T15:54:33.032 INFO:tasks.workunit.client.0.smithi117.stdout: cls: Running main() from gmock_main.cc 2024-03-04T15:54:33.032 INFO:tasks.workunit.client.0.smithi117.stdout: cls: [==========] Running 2 tests from 1 test suite. 2024-03-04T15:54:33.032 INFO:tasks.workunit.client.0.smithi117.stdout: cls: [----------] Global test environment set-up. 2024-03-04T15:54:33.032 INFO:tasks.workunit.client.0.smithi117.stdout: cls: [----------] 2 tests from NeoRadosCls 2024-03-04T15:54:33.032 INFO:tasks.workunit.client.0.smithi117.stdout: cls: [ RUN ] NeoRadosCls.DNE 2024-03-04T15:54:33.032 INFO:tasks.workunit.client.0.smithi117.stdout: cls: [ OK ] NeoRadosCls.DNE (18526 ms) 2024-03-04T15:54:33.032 INFO:tasks.workunit.client.0.smithi117.stdout: cls: [ RUN ] NeoRadosCls.RemoteReads 2024-03-04T15:54:33.032 INFO:tasks.workunit.client.0.smithi117.stdout: cls: [ OK ] NeoRadosCls.RemoteReads (14332 ms) 2024-03-04T15:54:33.032 INFO:tasks.workunit.client.0.smithi117.stdout: cls: [----------] 2 tests from NeoRadosCls (32858 ms total) 2024-03-04T15:54:33.032 INFO:tasks.workunit.client.0.smithi117.stdout: cls: 2024-03-04T15:54:33.032 INFO:tasks.workunit.client.0.smithi117.stdout: cls: [----------] Global test environment tear-down 2024-03-04T15:54:33.032 INFO:tasks.workunit.client.0.smithi117.stdout: cls: [==========] 2 tests from 1 test suite ran. (32859 ms total) 2024-03-04T15:54:33.032 INFO:tasks.workunit.client.0.smithi117.stdout: cls: [ PASSED ] 2 tests. 2024-03-04T15:54:33.051 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_1_[64485]: starting. 2024-03-04T15:54:33.051 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_1_[64485]: creating pool ceph_test_rados_delete_pools_parallel.smithi117-64440 2024-03-04T15:54:33.051 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_1_[64485]: created object 0... 2024-03-04T15:54:33.051 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_1_[64485]: created object 25... 2024-03-04T15:54:33.051 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_1_[64485]: created object 49... 2024-03-04T15:54:33.051 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_1_[64485]: finishing. 2024-03-04T15:54:33.051 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_1_[64485]: shutting down. 2024-03-04T15:54:33.051 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_2_[64488]: starting. 2024-03-04T15:54:33.051 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_2_[64488]: deleting pool ceph_test_rados_delete_pools_parallel.smithi117-64440 2024-03-04T15:54:33.051 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_2_[64488]: shutting down. 2024-03-04T15:54:33.051 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: ******************************* 2024-03-04T15:54:33.052 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_3_[65732]: starting. 2024-03-04T15:54:33.052 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_3_[65732]: creating pool ceph_test_rados_delete_pools_parallel.smithi117-64440 2024-03-04T15:54:33.052 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_3_[65732]: created object 0... 2024-03-04T15:54:33.052 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_3_[65732]: created object 25... 2024-03-04T15:54:33.052 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_3_[65732]: created object 49... 2024-03-04T15:54:33.052 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_3_[65732]: finishing. 2024-03-04T15:54:33.052 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_3_[65732]: shutting down. 2024-03-04T15:54:33.052 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: ******************************* 2024-03-04T15:54:33.052 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_5_[65734]: starting. 2024-03-04T15:54:33.052 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_5_[65734]: listing objects. 2024-03-04T15:54:33.052 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_5_[65734]: listed object 0... 2024-03-04T15:54:33.052 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_5_[65734]: listed object 25... 2024-03-04T15:54:33.052 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_5_[65734]: saw 50 objects 2024-03-04T15:54:33.052 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_5_[65734]: shutting down. 2024-03-04T15:54:33.052 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: ******************************* 2024-03-04T15:54:33.052 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_4_[65733]: starting. 2024-03-04T15:54:33.053 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_4_[65733]: deleting pool ceph_test_rados_delete_pools_parallel.smithi117-64440 2024-03-04T15:54:33.053 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: process_4_[65733]: shutting down. 2024-03-04T15:54:33.053 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: ******************************* 2024-03-04T15:54:33.053 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: ******************************* 2024-03-04T15:54:33.053 INFO:tasks.workunit.client.0.smithi117.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-03-04T15:54:34.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: audit 2024-03-04T15:54:32.522980+0000 mon.a (mon.0) 1021 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-03-04T15:54:34.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: audit 2024-03-04T15:54:32.533160+0000 mon.a (mon.0) 1022 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:34.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: cluster 2024-03-04T15:54:32.536916+0000 mon.a (mon.0) 1023 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:54:34.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: audit 2024-03-04T15:54:32.539907+0000 mon.a (mon.0) 1024 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-4-cache", "mode": "writeback"}]': finished 2024-03-04T15:54:34.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: audit 2024-03-04T15:54:32.540097+0000 mon.a (mon.0) 1025 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]': finished 2024-03-04T15:54:34.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: audit 2024-03-04T15:54:32.540286+0000 mon.a (mon.0) 1026 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:34.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: audit 2024-03-04T15:54:32.540478+0000 mon.a (mon.0) 1027 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:34.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: audit 2024-03-04T15:54:32.540590+0000 mon.a (mon.0) 1028 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:34.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: audit 2024-03-04T15:54:32.549086+0000 mon.b (mon.1) 48 : audit 1 from='client.? 172.21.15.117:0/807910892' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi117-63937-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:34.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: audit 2024-03-04T15:54:32.565243+0000 mon.c (mon.2) 146 : audit 1 from='client.? 172.21.15.117:0/3140935506' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:34.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: audit 2024-03-04T15:54:32.568967+0000 mon.c (mon.2) 147 : audit 1 from='client.? 172.21.15.117:0/3288255947' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi117-64235-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:34.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: audit 2024-03-04T15:54:32.753031+0000 mon.b (mon.1) 49 : audit 0 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-03-04T15:54:34.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: audit 2024-03-04T15:54:32.757811+0000 mon.b (mon.1) 50 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:34.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: audit 2024-03-04T15:54:32.975787+0000 mon.c (mon.2) 148 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:54:34.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: cluster 2024-03-04T15:54:33.021601+0000 mon.a (mon.0) 1029 : cluster 0 osdmap e92: 8 total, 8 up, 8 in 2024-03-04T15:54:34.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: cluster 2024-03-04T15:54:33.022905+0000 mon.a (mon.0) 1030 : cluster 3 Health check update: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:34.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: audit 2024-03-04T15:54:33.024526+0000 mon.c (mon.2) 149 : audit 1 from='client.? 172.21.15.117:0/819447599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi117-63915-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:34.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: audit 2024-03-04T15:54:33.025333+0000 mon.c (mon.2) 150 : audit 1 from='client.? 172.21.15.117:0/3837496039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi117-64587-2"}]: dispatch 2024-03-04T15:54:34.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: audit 2024-03-04T15:54:33.037603+0000 mon.a (mon.0) 1031 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:34.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: audit 2024-03-04T15:54:33.044430+0000 mon.a (mon.0) 1032 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi117-64235-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:34.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: audit 2024-03-04T15:54:33.477775+0000 mon.a (mon.0) 1033 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-4"}]: dispatch 2024-03-04T15:54:34.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: audit 2024-03-04T15:54:33.481310+0000 mon.a (mon.0) 1034 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi117-63915-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:34.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:33 smithi151 bash[23756]: audit 2024-03-04T15:54:33.481553+0000 mon.a (mon.0) 1035 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi117-64587-2"}]: dispatch 2024-03-04T15:54:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: audit 2024-03-04T15:54:32.522980+0000 mon.a (mon.0) 1021 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-03-04T15:54:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: audit 2024-03-04T15:54:32.533160+0000 mon.a (mon.0) 1022 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: cluster 2024-03-04T15:54:32.536916+0000 mon.a (mon.0) 1023 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:54:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: audit 2024-03-04T15:54:32.539907+0000 mon.a (mon.0) 1024 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-4-cache", "mode": "writeback"}]': finished 2024-03-04T15:54:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: audit 2024-03-04T15:54:32.540097+0000 mon.a (mon.0) 1025 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]': finished 2024-03-04T15:54:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: audit 2024-03-04T15:54:32.540286+0000 mon.a (mon.0) 1026 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: audit 2024-03-04T15:54:32.540478+0000 mon.a (mon.0) 1027 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: audit 2024-03-04T15:54:32.540590+0000 mon.a (mon.0) 1028 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: audit 2024-03-04T15:54:32.549086+0000 mon.b (mon.1) 48 : audit 1 from='client.? 172.21.15.117:0/807910892' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi117-63937-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: audit 2024-03-04T15:54:32.565243+0000 mon.c (mon.2) 146 : audit 1 from='client.? 172.21.15.117:0/3140935506' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:34.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: audit 2024-03-04T15:54:32.568967+0000 mon.c (mon.2) 147 : audit 1 from='client.? 172.21.15.117:0/3288255947' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi117-64235-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:34.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: audit 2024-03-04T15:54:32.753031+0000 mon.b (mon.1) 49 : audit 0 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-03-04T15:54:34.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: audit 2024-03-04T15:54:32.757811+0000 mon.b (mon.1) 50 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:34.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: audit 2024-03-04T15:54:32.975787+0000 mon.c (mon.2) 148 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:54:34.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: cluster 2024-03-04T15:54:33.021601+0000 mon.a (mon.0) 1029 : cluster 0 osdmap e92: 8 total, 8 up, 8 in 2024-03-04T15:54:34.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: audit 2024-03-04T15:54:32.522980+0000 mon.a (mon.0) 1021 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-03-04T15:54:34.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: audit 2024-03-04T15:54:32.533160+0000 mon.a (mon.0) 1022 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:34.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: cluster 2024-03-04T15:54:32.536916+0000 mon.a (mon.0) 1023 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:54:34.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: audit 2024-03-04T15:54:32.539907+0000 mon.a (mon.0) 1024 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-4-cache", "mode": "writeback"}]': finished 2024-03-04T15:54:34.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: audit 2024-03-04T15:54:32.540097+0000 mon.a (mon.0) 1025 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi117-64254-1"}]': finished 2024-03-04T15:54:34.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: audit 2024-03-04T15:54:32.540286+0000 mon.a (mon.0) 1026 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:34.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: audit 2024-03-04T15:54:32.540478+0000 mon.a (mon.0) 1027 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:34.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: audit 2024-03-04T15:54:32.540590+0000 mon.a (mon.0) 1028 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:34.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: audit 2024-03-04T15:54:32.549086+0000 mon.b (mon.1) 48 : audit 1 from='client.? 172.21.15.117:0/807910892' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi117-63937-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:34.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: audit 2024-03-04T15:54:32.565243+0000 mon.c (mon.2) 146 : audit 1 from='client.? 172.21.15.117:0/3140935506' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:34.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: audit 2024-03-04T15:54:32.568967+0000 mon.c (mon.2) 147 : audit 1 from='client.? 172.21.15.117:0/3288255947' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi117-64235-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:34.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: audit 2024-03-04T15:54:32.753031+0000 mon.b (mon.1) 49 : audit 0 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-03-04T15:54:34.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: audit 2024-03-04T15:54:32.757811+0000 mon.b (mon.1) 50 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:34.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: audit 2024-03-04T15:54:32.975787+0000 mon.c (mon.2) 148 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:54:34.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: cluster 2024-03-04T15:54:33.021601+0000 mon.a (mon.0) 1029 : cluster 0 osdmap e92: 8 total, 8 up, 8 in 2024-03-04T15:54:34.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: cluster 2024-03-04T15:54:33.022905+0000 mon.a (mon.0) 1030 : cluster 3 Health check update: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:34.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: audit 2024-03-04T15:54:33.024526+0000 mon.c (mon.2) 149 : audit 1 from='client.? 172.21.15.117:0/819447599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi117-63915-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:34.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: audit 2024-03-04T15:54:33.025333+0000 mon.c (mon.2) 150 : audit 1 from='client.? 172.21.15.117:0/3837496039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi117-64587-2"}]: dispatch 2024-03-04T15:54:34.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: audit 2024-03-04T15:54:33.037603+0000 mon.a (mon.0) 1031 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:34.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: audit 2024-03-04T15:54:33.044430+0000 mon.a (mon.0) 1032 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi117-64235-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:34.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: audit 2024-03-04T15:54:33.477775+0000 mon.a (mon.0) 1033 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-4"}]: dispatch 2024-03-04T15:54:34.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: audit 2024-03-04T15:54:33.481310+0000 mon.a (mon.0) 1034 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi117-63915-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:34.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[28843]: audit 2024-03-04T15:54:33.481553+0000 mon.a (mon.0) 1035 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi117-64587-2"}]: dispatch 2024-03-04T15:54:34.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: cluster 2024-03-04T15:54:33.022905+0000 mon.a (mon.0) 1030 : cluster 3 Health check update: 13 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:34.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: audit 2024-03-04T15:54:33.024526+0000 mon.c (mon.2) 149 : audit 1 from='client.? 172.21.15.117:0/819447599' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi117-63915-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:34.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: audit 2024-03-04T15:54:33.025333+0000 mon.c (mon.2) 150 : audit 1 from='client.? 172.21.15.117:0/3837496039' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi117-64587-2"}]: dispatch 2024-03-04T15:54:34.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: audit 2024-03-04T15:54:33.037603+0000 mon.a (mon.0) 1031 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi117-64254-1"}]: dispatch 2024-03-04T15:54:34.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: audit 2024-03-04T15:54:33.044430+0000 mon.a (mon.0) 1032 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi117-64235-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:34.322 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: audit 2024-03-04T15:54:33.477775+0000 mon.a (mon.0) 1033 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-4"}]: dispatch 2024-03-04T15:54:34.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: audit 2024-03-04T15:54:33.481310+0000 mon.a (mon.0) 1034 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi117-63915-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:34.323 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:33 smithi117 bash[21480]: audit 2024-03-04T15:54:33.481553+0000 mon.a (mon.0) 1035 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi117-64587-2"}]: dispatch 2024-03-04T15:54:34.534 INFO:tasks.workunit.client.0.smithi117.stdout:"test-rados-api-smithi117-64101-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:34.534 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:30.173664+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReadOpsmithi117-64587-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReadOpsmithi117-64587-2"}]': finished 2024-03-04T15:54:34.534 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:30.173746+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi117-63960-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi117-63960-10"}]': finished 2024-03-04T15:54:34.535 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:30.173803+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi117-64169-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi117-64169-7"}]': finished 2024-03-04T15:54:34.535 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:30.173856+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi117-64189-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi117-64189-7"}]': finished 2024-03-04T15:54:34.535 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:30.173904+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi117-63931-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi117-63931-16"}]': finished 2024-03-04T15:54:34.535 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:30.173953+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi117-63986-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi117-63986-10"}]': finished 2024-03-04T15:54:34.535 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:31.053875+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-03-04T15:54:34.535 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: got: 2024-03-04T15:54:31.066923+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsNSsmithi117-64656-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:34.536 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (15834 ms) 2024-03-04T15:54:34.536 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (34479 ms total) 2024-03-04T15:54:34.536 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: 2024-03-04T15:54:34.536 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: [----------] Global test environment tear-down 2024-03-04T15:54:34.536 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (34479 ms total) 2024-03-04T15:54:34.536 INFO:tasks.workunit.client.0.smithi117.stdout: api_cmd: [ PASSED ] 4 tests. 2024-03-04T15:54:36.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:35 smithi117 bash[28843]: audit 2024-03-04T15:54:33.657070+0000 mon.c (mon.2) 151 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:54:36.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:35 smithi117 bash[28843]: audit 2024-03-04T15:54:33.658542+0000 mon.c (mon.2) 152 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:54:36.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:35 smithi117 bash[28843]: audit 2024-03-04T15:54:33.895033+0000 mon.a (mon.0) 1036 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:36.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:35 smithi117 bash[28843]: audit 2024-03-04T15:54:33.912023+0000 mon.a (mon.0) 1037 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi117-63937-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:36.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:35 smithi117 bash[28843]: audit 2024-03-04T15:54:33.912527+0000 mon.a (mon.0) 1038 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi117-64656-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi117-64656-2"}]': finished 2024-03-04T15:54:36.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:35 smithi117 bash[21480]: audit 2024-03-04T15:54:33.657070+0000 mon.c (mon.2) 151 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:54:36.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:35 smithi117 bash[21480]: audit 2024-03-04T15:54:33.658542+0000 mon.c (mon.2) 152 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:54:36.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:35 smithi117 bash[21480]: audit 2024-03-04T15:54:33.895033+0000 mon.a (mon.0) 1036 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:36.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:35 smithi117 bash[21480]: audit 2024-03-04T15:54:33.912023+0000 mon.a (mon.0) 1037 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi117-63937-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:36.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:35 smithi117 bash[21480]: audit 2024-03-04T15:54:33.912527+0000 mon.a (mon.0) 1038 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi117-64656-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi117-64656-2"}]': finished 2024-03-04T15:54:36.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:35 smithi151 bash[23756]: audit 2024-03-04T15:54:33.657070+0000 mon.c (mon.2) 151 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:54:36.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:35 smithi151 bash[23756]: audit 2024-03-04T15:54:33.658542+0000 mon.c (mon.2) 152 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:54:36.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:35 smithi151 bash[23756]: audit 2024-03-04T15:54:33.895033+0000 mon.a (mon.0) 1036 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:36.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:35 smithi151 bash[23756]: audit 2024-03-04T15:54:33.912023+0000 mon.a (mon.0) 1037 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi117-63937-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:36.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:35 smithi151 bash[23756]: audit 2024-03-04T15:54:33.912527+0000 mon.a (mon.0) 1038 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsNSsmithi117-64656-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsNSsmithi117-64656-2"}]': finished 2024-03-04T15:54:36.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:54:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:54:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:54:36.932 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:54:36 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:54:37.024 INFO:tasks.workunit.client.0.smithi117.stdout: ec_io: Running main() from gmock_main.cc 2024-03-04T15:54:37.024 INFO:tasks.workunit.client.0.smithi117.stdout: ec_io: [==========] Running 2 tests from 1 test suite. 2024-03-04T15:54:37.024 INFO:tasks.workunit.client.0.smithi117.stdout: ec_io: [----------] Global test environment set-up. 2024-03-04T15:54:37.024 INFO:tasks.workunit.client.0.smithi117.stdout: ec_io: [----------] 2 tests from NeoRadosECIo 2024-03-04T15:54:37.024 INFO:tasks.workunit.client.0.smithi117.stdout: ec_io: [ RUN ] NeoRadosECIo.SimpleWrite 2024-03-04T15:54:37.024 INFO:tasks.workunit.client.0.smithi117.stdout: ec_io: [ OK ] NeoRadosECIo.SimpleWrite (23601 ms) 2024-03-04T15:54:37.024 INFO:tasks.workunit.client.0.smithi117.stdout: ec_io: [ RUN ] NeoRadosECIo.ReadOp 2024-03-04T15:54:37.024 INFO:tasks.workunit.client.0.smithi117.stdout: ec_io: [ OK ] NeoRadosECIo.ReadOp (13191 ms) 2024-03-04T15:54:37.024 INFO:tasks.workunit.client.0.smithi117.stdout: ec_io: [----------] 2 tests from NeoRadosECIo (36792 ms total) 2024-03-04T15:54:37.025 INFO:tasks.workunit.client.0.smithi117.stdout: ec_io: 2024-03-04T15:54:37.025 INFO:tasks.workunit.client.0.smithi117.stdout: ec_io: [----------] Global test environment tear-down 2024-03-04T15:54:37.025 INFO:tasks.workunit.client.0.smithi117.stdout: ec_io: [==========] 2 tests from 1 test suite ran. (36792 ms total) 2024-03-04T15:54:37.025 INFO:tasks.workunit.client.0.smithi117.stdout: ec_io: [ PASSED ] 2 tests. 2024-03-04T15:54:37.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[21480]: audit 2024-03-04T15:54:33.912689+0000 mon.a (mon.0) 1039 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi117-64254-1"}]': finished 2024-03-04T15:54:37.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[21480]: audit 2024-03-04T15:54:33.912794+0000 mon.a (mon.0) 1040 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-4"}]': finished 2024-03-04T15:54:37.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[21480]: audit 2024-03-04T15:54:33.912968+0000 mon.a (mon.0) 1041 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi117-63915-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:37.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[21480]: audit 2024-03-04T15:54:33.913198+0000 mon.a (mon.0) 1042 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi117-64587-2"}]': finished 2024-03-04T15:54:37.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[21480]: audit 2024-03-04T15:54:33.930317+0000 mon.c (mon.2) 153 : audit 1 from='client.? 172.21.15.117:0/3837496039' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi117-64587-2"}]: dispatch 2024-03-04T15:54:37.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[21480]: audit 2024-03-04T15:54:34.381491+0000 mon.c (mon.2) 154 : audit 1 from='client.? 172.21.15.117:0/2687162500' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:37.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[21480]: audit 2024-03-04T15:54:34.381732+0000 mon.c (mon.2) 155 : audit 1 from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:37.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[21480]: audit 2024-03-04T15:54:34.381956+0000 mon.c (mon.2) 156 : audit 1 from='client.? 172.21.15.117:0/3022824111' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:37.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[21480]: cluster 2024-03-04T15:54:35.672700+0000 mon.a (mon.0) 1043 : cluster 0 osdmap e93: 8 total, 8 up, 8 in 2024-03-04T15:54:37.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[21480]: audit 2024-03-04T15:54:35.686747+0000 mon.a (mon.0) 1044 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:37.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[21480]: audit 2024-03-04T15:54:35.686984+0000 mon.a (mon.0) 1045 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-4", "tierpool": "test-rados-api-smithi117-64061-4-cache"}]: dispatch 2024-03-04T15:54:37.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[21480]: audit 2024-03-04T15:54:35.696128+0000 mon.a (mon.0) 1046 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi117-64587-2"}]: dispatch 2024-03-04T15:54:37.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[21480]: audit 2024-03-04T15:54:35.696877+0000 mon.a (mon.0) 1047 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi117-64235-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]': finished 2024-03-04T15:54:37.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[21480]: audit 2024-03-04T15:54:35.702235+0000 mon.a (mon.0) 1048 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:54:37.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[21480]: audit 2024-03-04T15:54:35.709557+0000 mon.c (mon.2) 157 : audit 1 from='client.? 172.21.15.117:0/2077663049' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:37.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[21480]: audit 2024-03-04T15:54:35.712753+0000 mon.c (mon.2) 158 : audit 1 from='client.? 172.21.15.117:0/4079046038' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:37.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[21480]: cluster 2024-03-04T15:54:36.151442+0000 mon.a (mon.0) 1049 : cluster 0 osdmap e94: 8 total, 8 up, 8 in 2024-03-04T15:54:37.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[28843]: audit 2024-03-04T15:54:33.912689+0000 mon.a (mon.0) 1039 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi117-64254-1"}]': finished 2024-03-04T15:54:37.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[28843]: audit 2024-03-04T15:54:33.912794+0000 mon.a (mon.0) 1040 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-4"}]': finished 2024-03-04T15:54:37.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[28843]: audit 2024-03-04T15:54:33.912968+0000 mon.a (mon.0) 1041 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi117-63915-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:37.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[28843]: audit 2024-03-04T15:54:33.913198+0000 mon.a (mon.0) 1042 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi117-64587-2"}]': finished 2024-03-04T15:54:37.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[28843]: audit 2024-03-04T15:54:33.930317+0000 mon.c (mon.2) 153 : audit 1 from='client.? 172.21.15.117:0/3837496039' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi117-64587-2"}]: dispatch 2024-03-04T15:54:37.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[28843]: audit 2024-03-04T15:54:34.381491+0000 mon.c (mon.2) 154 : audit 1 from='client.? 172.21.15.117:0/2687162500' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:37.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[28843]: audit 2024-03-04T15:54:34.381732+0000 mon.c (mon.2) 155 : audit 1 from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:37.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[28843]: audit 2024-03-04T15:54:34.381956+0000 mon.c (mon.2) 156 : audit 1 from='client.? 172.21.15.117:0/3022824111' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:37.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[28843]: cluster 2024-03-04T15:54:35.672700+0000 mon.a (mon.0) 1043 : cluster 0 osdmap e93: 8 total, 8 up, 8 in 2024-03-04T15:54:37.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[28843]: audit 2024-03-04T15:54:35.686747+0000 mon.a (mon.0) 1044 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:37.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[28843]: audit 2024-03-04T15:54:35.686984+0000 mon.a (mon.0) 1045 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-4", "tierpool": "test-rados-api-smithi117-64061-4-cache"}]: dispatch 2024-03-04T15:54:37.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[28843]: audit 2024-03-04T15:54:35.696128+0000 mon.a (mon.0) 1046 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi117-64587-2"}]: dispatch 2024-03-04T15:54:37.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[28843]: audit 2024-03-04T15:54:35.696877+0000 mon.a (mon.0) 1047 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi117-64235-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]': finished 2024-03-04T15:54:37.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[28843]: audit 2024-03-04T15:54:35.702235+0000 mon.a (mon.0) 1048 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:54:37.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[28843]: audit 2024-03-04T15:54:35.709557+0000 mon.c (mon.2) 157 : audit 1 from='client.? 172.21.15.117:0/2077663049' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:37.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[28843]: audit 2024-03-04T15:54:35.712753+0000 mon.c (mon.2) 158 : audit 1 from='client.? 172.21.15.117:0/4079046038' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:37.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:37 smithi117 bash[28843]: cluster 2024-03-04T15:54:36.151442+0000 mon.a (mon.0) 1049 : cluster 0 osdmap e94: 8 total, 8 up, 8 in 2024-03-04T15:54:37.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:37 smithi151 bash[23756]: audit 2024-03-04T15:54:33.912689+0000 mon.a (mon.0) 1039 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi117-64254-1"}]': finished 2024-03-04T15:54:37.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:37 smithi151 bash[23756]: audit 2024-03-04T15:54:33.912794+0000 mon.a (mon.0) 1040 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-4"}]': finished 2024-03-04T15:54:37.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:37 smithi151 bash[23756]: audit 2024-03-04T15:54:33.912968+0000 mon.a (mon.0) 1041 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi117-63915-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:37.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:37 smithi151 bash[23756]: audit 2024-03-04T15:54:33.913198+0000 mon.a (mon.0) 1042 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReadOpsmithi117-64587-2"}]': finished 2024-03-04T15:54:37.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:37 smithi151 bash[23756]: audit 2024-03-04T15:54:33.930317+0000 mon.c (mon.2) 153 : audit 1 from='client.? 172.21.15.117:0/3837496039' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi117-64587-2"}]: dispatch 2024-03-04T15:54:37.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:37 smithi151 bash[23756]: audit 2024-03-04T15:54:34.381491+0000 mon.c (mon.2) 154 : audit 1 from='client.? 172.21.15.117:0/2687162500' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:37.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:37 smithi151 bash[23756]: audit 2024-03-04T15:54:34.381732+0000 mon.c (mon.2) 155 : audit 1 from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:37.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:37 smithi151 bash[23756]: audit 2024-03-04T15:54:34.381956+0000 mon.c (mon.2) 156 : audit 1 from='client.? 172.21.15.117:0/3022824111' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:37.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:37 smithi151 bash[23756]: cluster 2024-03-04T15:54:35.672700+0000 mon.a (mon.0) 1043 : cluster 0 osdmap e93: 8 total, 8 up, 8 in 2024-03-04T15:54:37.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:37 smithi151 bash[23756]: audit 2024-03-04T15:54:35.686747+0000 mon.a (mon.0) 1044 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:37.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:37 smithi151 bash[23756]: audit 2024-03-04T15:54:35.686984+0000 mon.a (mon.0) 1045 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-4", "tierpool": "test-rados-api-smithi117-64061-4-cache"}]: dispatch 2024-03-04T15:54:37.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:37 smithi151 bash[23756]: audit 2024-03-04T15:54:35.696128+0000 mon.a (mon.0) 1046 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi117-64587-2"}]: dispatch 2024-03-04T15:54:37.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:37 smithi151 bash[23756]: audit 2024-03-04T15:54:35.696877+0000 mon.a (mon.0) 1047 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi117-64235-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]': finished 2024-03-04T15:54:37.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:37 smithi151 bash[23756]: audit 2024-03-04T15:54:35.702235+0000 mon.a (mon.0) 1048 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:54:37.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:37 smithi151 bash[23756]: audit 2024-03-04T15:54:35.709557+0000 mon.c (mon.2) 157 : audit 1 from='client.? 172.21.15.117:0/2077663049' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:37.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:37 smithi151 bash[23756]: audit 2024-03-04T15:54:35.712753+0000 mon.c (mon.2) 158 : audit 1 from='client.? 172.21.15.117:0/4079046038' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:37.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:37 smithi151 bash[23756]: cluster 2024-03-04T15:54:36.151442+0000 mon.a (mon.0) 1049 : cluster 0 osdmap e94: 8 total, 8 up, 8 in 2024-03-04T15:54:37.818 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: [==========] Running 4 tests from 1 test suite. 2024-03-04T15:54:37.819 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: [----------] Global test environment set-up. 2024-03-04T15:54:37.819 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: [----------] 4 tests from LibRadosService 2024-03-04T15:54:37.819 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-03-04T15:54:37.819 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5787 ms) 2024-03-04T15:54:37.819 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-03-04T15:54:37.819 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: [ OK ] LibRadosService.RegisterLate (52 ms) 2024-03-04T15:54:37.819 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-03-04T15:54:37.819 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: cluster: 2024-03-04T15:54:37.819 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: id: e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T15:54:37.819 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: health: HEALTH_WARN 2024-03-04T15:54:37.819 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 2 pool(s) do not have an application enabled 2024-03-04T15:54:37.819 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 2024-03-04T15:54:37.819 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: services: 2024-03-04T15:54:37.819 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 12m) 2024-03-04T15:54:37.820 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: mgr: y(active, since 3m), standbys: x 2024-03-04T15:54:37.820 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: osd: 8 osds: 8 up (since 5m), 8 in (since 5m) 2024-03-04T15:54:37.820 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-03-04T15:54:37.820 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-03-04T15:54:37.820 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 2024-03-04T15:54:37.820 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: data: 2024-03-04T15:54:37.820 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: pools: 8 pools, 196 pgs 2024-03-04T15:54:37.820 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: objects: 198 objects, 582 KiB 2024-03-04T15:54:37.820 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: usage: 248 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:54:37.820 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: pgs: 7.653% pgs unknown 2024-03-04T15:54:37.820 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 25.000% pgs not active 2024-03-04T15:54:37.820 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 132 active+clean 2024-03-04T15:54:37.820 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 49 creating+peering 2024-03-04T15:54:37.820 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 15 unknown 2024-03-04T15:54:37.820 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 2024-03-04T15:54:37.821 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: io: 2024-03-04T15:54:37.821 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: client: 1023 B/s rd, 0 op/s rd, 0 op/s wr 2024-03-04T15:54:37.821 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 2024-03-04T15:54:37.821 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 2024-03-04T15:54:37.821 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: cluster: 2024-03-04T15:54:37.821 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: id: e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T15:54:37.821 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: health: HEALTH_WARN 2024-03-04T15:54:37.821 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 22 pool(s) do not have an application enabled 2024-03-04T15:54:37.821 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 2024-03-04T15:54:37.821 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: services: 2024-03-04T15:54:37.821 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 12m) 2024-03-04T15:54:37.821 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: mgr: y(active, since 3m), standbys: x 2024-03-04T15:54:37.821 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: osd: 8 osds: 8 up (since 5m), 8 in (since 5m) 2024-03-04T15:54:37.821 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-03-04T15:54:37.821 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-03-04T15:54:37.822 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-03-04T15:54:37.822 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 2024-03-04T15:54:37.822 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: data: 2024-03-04T15:54:37.822 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: pools: 41 pools, 1252 pgs 2024-03-04T15:54:37.822 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: objects: 198 objects, 582 KiB 2024-03-04T15:54:37.822 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: usage: 248 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:54:37.822 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: pgs: 40.495% pgs unknown 2024-03-04T15:54:37.822 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 46.166% pgs not active 2024-03-04T15:54:37.822 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 578 creating+peering 2024-03-04T15:54:37.822 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 507 unknown 2024-03-04T15:54:37.822 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 167 active+clean 2024-03-04T15:54:37.822 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 2024-03-04T15:54:37.822 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: io: 2024-03-04T15:54:37.822 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: client: 837 B/s rd, 0 op/s rd, 0 op/s wr 2024-03-04T15:54:37.822 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 2024-03-04T15:54:37.822 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 2024-03-04T15:54:37.823 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: [ OK ] LibRadosService.StatusFormat (11444 ms) 2024-03-04T15:54:37.823 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: [ RUN ] LibRadosService.Status 2024-03-04T15:54:37.823 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: [ OK ] LibRadosService.Status (20452 ms) 2024-03-04T15:54:37.823 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: [----------] 4 tests from LibRadosService (37735 ms total) 2024-03-04T15:54:37.823 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: 2024-03-04T15:54:37.823 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: [----------] Global test environment tear-down 2024-03-04T15:54:37.823 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: [==========] 4 tests from 1 test suite ran. (37735 ms total) 2024-03-04T15:54:37.823 INFO:tasks.workunit.client.0.smithi117.stdout: api_service: [ PASSED ] 4 tests. 2024-03-04T15:54:38.055 INFO:tasks.workunit.client.0.smithi117.stdout: cmd: Running main() from gmock_main.cc 2024-03-04T15:54:38.056 INFO:tasks.workunit.client.0.smithi117.stdout: cmd: [==========] Running 3 tests from 1 test suite. 2024-03-04T15:54:38.056 INFO:tasks.workunit.client.0.smithi117.stdout: cmd: [----------] Global test environment set-up. 2024-03-04T15:54:38.056 INFO:tasks.workunit.client.0.smithi117.stdout: cmd: [----------] 3 tests from NeoRadosCmd 2024-03-04T15:54:38.056 INFO:tasks.workunit.client.0.smithi117.stdout: cmd: [ RUN ] NeoRadosCmd.MonDescribe 2024-03-04T15:54:38.056 INFO:tasks.workunit.client.0.smithi117.stdout: cmd: [ OK ] NeoRadosCmd.MonDescribe (15596 ms) 2024-03-04T15:54:38.056 INFO:tasks.workunit.client.0.smithi117.stdout: cmd: [ RUN ] NeoRadosCmd.OSDCmd 2024-03-04T15:54:38.056 INFO:tasks.workunit.client.0.smithi117.stdout: cmd: [ OK ] NeoRadosCmd.OSDCmd (4762 ms) 2024-03-04T15:54:38.056 INFO:tasks.workunit.client.0.smithi117.stdout: cmd: [ RUN ] NeoRadosCmd.PGCmd 2024-03-04T15:54:38.056 INFO:tasks.workunit.client.0.smithi117.stdout: cmd: [ OK ] NeoRadosCmd.PGCmd (17469 ms) 2024-03-04T15:54:38.056 INFO:tasks.workunit.client.0.smithi117.stdout: cmd: [----------] 3 tests from NeoRadosCmd (37828 ms total) 2024-03-04T15:54:38.056 INFO:tasks.workunit.client.0.smithi117.stdout: cmd: 2024-03-04T15:54:38.056 INFO:tasks.workunit.client.0.smithi117.stdout: cmd: [----------] Global test environment tear-down 2024-03-04T15:54:38.056 INFO:tasks.workunit.client.0.smithi117.stdout: cmd: [==========] 3 tests from 1 test suite ran. (37841 ms total) 2024-03-04T15:54:38.056 INFO:tasks.workunit.client.0.smithi117.stdout: cmd: [ PASSED ] 3 tests. 2024-03-04T15:54:38.057 INFO:tasks.workunit.client.0.smithi117.stdout: watch_notify: Running main() from gmock_main.cc 2024-03-04T15:54:38.057 INFO:tasks.workunit.client.0.smithi117.stdout: watch_notify: [==========] Running 2 tests from 1 test suite. 2024-03-04T15:54:38.057 INFO:tasks.workunit.client.0.smithi117.stdout: watch_notify: [----------] Global test environment set-up. 2024-03-04T15:54:38.060 INFO:tasks.workunit.client.0.smithi117.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify 2024-03-04T15:54:38.060 INFO:tasks.workunit.client.0.smithi117.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotify 2024-03-04T15:54:38.060 INFO:tasks.workunit.client.0.smithi117.stdout: watch_notify: handle_notify cookie 94860408490320 notify_id 347892350977 notifier_gid 24803 2024-03-04T15:54:38.060 INFO:tasks.workunit.client.0.smithi117.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotify (20289 ms) 2024-03-04T15:54:38.060 INFO:tasks.workunit.client.0.smithi117.stdout: watch_notify: [ RUN ] NeoRadosWatchNotify.WatchNotifyTimeout 2024-03-04T15:54:38.060 INFO:tasks.workunit.client.0.smithi117.stdout: watch_notify: Trying... 2024-03-04T15:54:38.060 INFO:tasks.workunit.client.0.smithi117.stdout: watch_notify: handle_notify cookie 94860421271984 notify_id 390842023938 notifier_gid 25160 2024-03-04T15:54:38.060 INFO:tasks.workunit.client.0.smithi117.stdout: watch_notify: Waiting for 3.000000000s 2024-03-04T15:54:38.060 INFO:tasks.workunit.client.0.smithi117.stdout: watch_notify: Timed out. 2024-03-04T15:54:38.061 INFO:tasks.workunit.client.0.smithi117.stdout: watch_notify: Flushing... 2024-03-04T15:54:38.061 INFO:tasks.workunit.client.0.smithi117.stdout: watch_notify: Flushed... 2024-03-04T15:54:38.061 INFO:tasks.workunit.client.0.smithi117.stdout: watch_notify: [ OK ] NeoRadosWatchNotify.WatchNotifyTimeout (17471 ms) 2024-03-04T15:54:38.061 INFO:tasks.workunit.client.0.smithi117.stdout: watch_notify: [----------] 2 tests from NeoRadosWatchNotify (37760 ms total) 2024-03-04T15:54:38.061 INFO:tasks.workunit.client.0.smithi117.stdout: watch_notify: 2024-03-04T15:54:38.061 INFO:tasks.workunit.client.0.smithi117.stdout: watch_notify: [----------] Global test environment tear-down 2024-03-04T15:54:38.061 INFO:tasks.workunit.client.0.smithi117.stdout: watch_notify: [==========] 2 tests from 1 test suite ran. (37760 ms total) 2024-03-04T15:54:38.061 INFO:tasks.workunit.client.0.smithi117.stdout: watch_notify: [ PASSED ] 2 tests. 2024-03-04T15:54:38.315 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: Running main() from gmock_main.cc 2024-03-04T15:54:38.315 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-03-04T15:54:38.315 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [----------] Global test environment set-up. 2024-03-04T15:54:38.315 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-03-04T15:54:38.315 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: seed 63937 2024-03-04T15:54:38.315 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-03-04T15:54:38.315 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-03-04T15:54:38.315 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-03-04T15:54:38.315 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (194 ms) 2024-03-04T15:54:38.315 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-03-04T15:54:38.315 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (81 ms) 2024-03-04T15:54:38.315 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-03-04T15:54:38.315 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (4 ms) 2024-03-04T15:54:38.316 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-03-04T15:54:38.316 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (6 ms) 2024-03-04T15:54:38.316 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-03-04T15:54:38.316 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (5 ms) 2024-03-04T15:54:38.316 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-03-04T15:54:38.316 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (9 ms) 2024-03-04T15:54:38.316 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-03-04T15:54:38.316 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (6 ms) 2024-03-04T15:54:38.316 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-03-04T15:54:38.316 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (7 ms) 2024-03-04T15:54:38.316 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-03-04T15:54:38.316 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (4 ms) 2024-03-04T15:54:38.316 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-03-04T15:54:38.316 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (4 ms) 2024-03-04T15:54:38.316 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-03-04T15:54:38.317 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (7 ms) 2024-03-04T15:54:38.317 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-03-04T15:54:38.317 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (4 ms) 2024-03-04T15:54:38.317 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-03-04T15:54:38.317 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (4 ms) 2024-03-04T15:54:38.317 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-03-04T15:54:38.317 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (6 ms) 2024-03-04T15:54:38.317 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-03-04T15:54:38.317 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (15 ms) 2024-03-04T15:54:38.317 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-03-04T15:54:38.317 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (6 ms) 2024-03-04T15:54:38.317 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-03-04T15:54:38.317 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (6 ms) 2024-03-04T15:54:38.317 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-03-04T15:54:38.317 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (4 ms) 2024-03-04T15:54:38.318 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-03-04T15:54:38.318 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (8 ms) 2024-03-04T15:54:38.318 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (381 ms total) 2024-03-04T15:54:38.318 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: 2024-03-04T15:54:38.318 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-03-04T15:54:38.318 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-03-04T15:54:38.318 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1256 ms) 2024-03-04T15:54:38.318 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-03-04T15:54:38.318 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (18 ms) 2024-03-04T15:54:38.318 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-03-04T15:54:38.318 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (4 ms) 2024-03-04T15:54:38.318 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-03-04T15:54:38.318 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (5 ms) 2024-03-04T15:54:38.318 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-03-04T15:54:38.318 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (4 ms) 2024-03-04T15:54:38.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[21480]: cluster 2024-03-04T15:54:33.446386+0000 mgr.y (mgr.24428) 191 : cluster 0 pgmap v139: 748 pgs: 128 unknown, 174 creating+peering, 446 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 873 B/s rd, 0 op/s 2024-03-04T15:54:38.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[21480]: cluster 2024-03-04T15:54:33.662554+0000 mgr.y (mgr.24428) 192 : cluster 0 pgmap v140: 748 pgs: 128 unknown, 174 creating+peering, 446 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:54:38.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[21480]: cluster 2024-03-04T15:54:35.663842+0000 mgr.y (mgr.24428) 193 : cluster 0 pgmap v142: 796 pgs: 200 unknown, 150 creating+peering, 446 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail 2024-03-04T15:54:38.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[21480]: audit 2024-03-04T15:54:36.486794+0000 mgr.y (mgr.24428) 194 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:54:38.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[21480]: cluster 2024-03-04T15:54:36.988354+0000 mon.a (mon.0) 1050 : cluster 3 Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-04T15:54:38.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[21480]: cluster 2024-03-04T15:54:36.994382+0000 mon.a (mon.0) 1051 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:54:38.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[21480]: audit 2024-03-04T15:54:37.003832+0000 mon.a (mon.0) 1052 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:38.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[21480]: audit 2024-03-04T15:54:37.004727+0000 mon.a (mon.0) 1053 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:38.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[21480]: audit 2024-03-04T15:54:37.005392+0000 mon.a (mon.0) 1054 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:38.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[21480]: audit 2024-03-04T15:54:37.006294+0000 mon.a (mon.0) 1055 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:38.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[21480]: audit 2024-03-04T15:54:37.012413+0000 mon.a (mon.0) 1056 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi117-63937-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi117-63937-22"}]': finished 2024-03-04T15:54:38.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[21480]: audit 2024-03-04T15:54:37.012626+0000 mon.a (mon.0) 1057 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-4", "tierpool": "test-rados-api-smithi117-64061-4-cache"}]': finished 2024-03-04T15:54:38.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[21480]: audit 2024-03-04T15:54:37.012755+0000 mon.a (mon.0) 1058 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi117-64587-2"}]': finished 2024-03-04T15:54:38.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[21480]: cluster 2024-03-04T15:54:37.024057+0000 mon.a (mon.0) 1059 : cluster 0 osdmap e95: 8 total, 8 up, 8 in 2024-03-04T15:54:38.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[21480]: audit 2024-03-04T15:54:37.032954+0000 mon.a (mon.0) 1060 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:38.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[21480]: audit 2024-03-04T15:54:37.035336+0000 mon.a (mon.0) 1061 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:38.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[21480]: audit 2024-03-04T15:54:37.042875+0000 mon.a (mon.0) 1062 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:38.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[21480]: audit 2024-03-04T15:54:37.061826+0000 mon.a (mon.0) 1063 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:38.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[21480]: audit 2024-03-04T15:54:38.005328+0000 mon.a (mon.0) 1064 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:38.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[28843]: cluster 2024-03-04T15:54:33.446386+0000 mgr.y (mgr.24428) 191 : cluster 0 pgmap v139: 748 pgs: 128 unknown, 174 creating+peering, 446 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 873 B/s rd, 0 op/s 2024-03-04T15:54:38.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[28843]: cluster 2024-03-04T15:54:33.662554+0000 mgr.y (mgr.24428) 192 : cluster 0 pgmap v140: 748 pgs: 128 unknown, 174 creating+peering, 446 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:54:38.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[28843]: cluster 2024-03-04T15:54:35.663842+0000 mgr.y (mgr.24428) 193 : cluster 0 pgmap v142: 796 pgs: 200 unknown, 150 creating+peering, 446 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail 2024-03-04T15:54:38.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[28843]: audit 2024-03-04T15:54:36.486794+0000 mgr.y (mgr.24428) 194 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:54:38.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[28843]: cluster 2024-03-04T15:54:36.988354+0000 mon.a (mon.0) 1050 : cluster 3 Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-04T15:54:38.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[28843]: cluster 2024-03-04T15:54:36.994382+0000 mon.a (mon.0) 1051 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:54:38.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[28843]: audit 2024-03-04T15:54:37.003832+0000 mon.a (mon.0) 1052 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:38.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[28843]: audit 2024-03-04T15:54:37.004727+0000 mon.a (mon.0) 1053 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:38.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[28843]: audit 2024-03-04T15:54:37.005392+0000 mon.a (mon.0) 1054 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:38.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[28843]: audit 2024-03-04T15:54:37.006294+0000 mon.a (mon.0) 1055 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:38.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[28843]: audit 2024-03-04T15:54:37.012413+0000 mon.a (mon.0) 1056 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi117-63937-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi117-63937-22"}]': finished 2024-03-04T15:54:38.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[28843]: audit 2024-03-04T15:54:37.012626+0000 mon.a (mon.0) 1057 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-4", "tierpool": "test-rados-api-smithi117-64061-4-cache"}]': finished 2024-03-04T15:54:38.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[28843]: audit 2024-03-04T15:54:37.012755+0000 mon.a (mon.0) 1058 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi117-64587-2"}]': finished 2024-03-04T15:54:38.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[28843]: cluster 2024-03-04T15:54:37.024057+0000 mon.a (mon.0) 1059 : cluster 0 osdmap e95: 8 total, 8 up, 8 in 2024-03-04T15:54:38.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[28843]: audit 2024-03-04T15:54:37.032954+0000 mon.a (mon.0) 1060 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:38.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[28843]: audit 2024-03-04T15:54:37.035336+0000 mon.a (mon.0) 1061 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:38.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[28843]: audit 2024-03-04T15:54:37.042875+0000 mon.a (mon.0) 1062 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:38.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[28843]: audit 2024-03-04T15:54:37.061826+0000 mon.a (mon.0) 1063 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:38.322 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:38 smithi117 bash[28843]: audit 2024-03-04T15:54:38.005328+0000 mon.a (mon.0) 1064 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:38.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:38 smithi151 bash[23756]: cluster 2024-03-04T15:54:33.446386+0000 mgr.y (mgr.24428) 191 : cluster 0 pgmap v139: 748 pgs: 128 unknown, 174 creating+peering, 446 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 873 B/s rd, 0 op/s 2024-03-04T15:54:38.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:38 smithi151 bash[23756]: cluster 2024-03-04T15:54:33.662554+0000 mgr.y (mgr.24428) 192 : cluster 0 pgmap v140: 748 pgs: 128 unknown, 174 creating+peering, 446 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:54:38.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:38 smithi151 bash[23756]: cluster 2024-03-04T15:54:35.663842+0000 mgr.y (mgr.24428) 193 : cluster 0 pgmap v142: 796 pgs: 200 unknown, 150 creating+peering, 446 active+clean; 145 MiB data, 759 MiB used, 714 GiB / 715 GiB avail 2024-03-04T15:54:38.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:38 smithi151 bash[23756]: audit 2024-03-04T15:54:36.486794+0000 mgr.y (mgr.24428) 194 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:54:38.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:38 smithi151 bash[23756]: cluster 2024-03-04T15:54:36.988354+0000 mon.a (mon.0) 1050 : cluster 3 Health check failed: 1 stray daemon(s) not managed by cephadm (CEPHADM_STRAY_DAEMON) 2024-03-04T15:54:38.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:38 smithi151 bash[23756]: cluster 2024-03-04T15:54:36.994382+0000 mon.a (mon.0) 1051 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:54:38.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:38 smithi151 bash[23756]: audit 2024-03-04T15:54:37.003832+0000 mon.a (mon.0) 1052 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:38.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:38 smithi151 bash[23756]: audit 2024-03-04T15:54:37.004727+0000 mon.a (mon.0) 1053 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:38.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:38 smithi151 bash[23756]: audit 2024-03-04T15:54:37.005392+0000 mon.a (mon.0) 1054 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:38.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:38 smithi151 bash[23756]: audit 2024-03-04T15:54:37.006294+0000 mon.a (mon.0) 1055 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:38.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:38 smithi151 bash[23756]: audit 2024-03-04T15:54:37.012413+0000 mon.a (mon.0) 1056 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi117-63937-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi117-63937-22"}]': finished 2024-03-04T15:54:38.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:38 smithi151 bash[23756]: audit 2024-03-04T15:54:37.012626+0000 mon.a (mon.0) 1057 : audit 1 from='client.? 172.21.15.117:0/767414741' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-4", "tierpool": "test-rados-api-smithi117-64061-4-cache"}]': finished 2024-03-04T15:54:38.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:38 smithi151 bash[23756]: audit 2024-03-04T15:54:37.012755+0000 mon.a (mon.0) 1058 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReadOpsmithi117-64587-2"}]': finished 2024-03-04T15:54:38.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:38 smithi151 bash[23756]: cluster 2024-03-04T15:54:37.024057+0000 mon.a (mon.0) 1059 : cluster 0 osdmap e95: 8 total, 8 up, 8 in 2024-03-04T15:54:38.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:38 smithi151 bash[23756]: audit 2024-03-04T15:54:37.032954+0000 mon.a (mon.0) 1060 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:38.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:38 smithi151 bash[23756]: audit 2024-03-04T15:54:37.035336+0000 mon.a (mon.0) 1061 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:38.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:38 smithi151 bash[23756]: audit 2024-03-04T15:54:37.042875+0000 mon.a (mon.0) 1062 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:38.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:38 smithi151 bash[23756]: audit 2024-03-04T15:54:37.061826+0000 mon.a (mon.0) 1063 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:38.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:38 smithi151 bash[23756]: audit 2024-03-04T15:54:38.005328+0000 mon.a (mon.0) 1064 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:39.443 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] api_stat: Running main() from gmock_main.cc 2024-03-04T15:54:39.444 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [==========] Running 9 tests from 2 test suites. 2024-03-04T15:54:39.444 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [----------] Global test environment set-up. 2024-03-04T15:54:39.444 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [----------] 5 tests from LibRadosStat 2024-03-04T15:54:39.444 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-03-04T15:54:39.444 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [ OK ] LibRadosStat.Stat (219 ms) 2024-03-04T15:54:39.444 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [ RUN ] LibRadosStat.Stat2 2024-03-04T15:54:39.444 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [ OK ] LibRadosStat.Stat2 (33 ms) 2024-03-04T15:54:39.444 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-03-04T15:54:39.444 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [ OK ] LibRadosStat.StatNS (16 ms) 2024-03-04T15:54:39.444 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-03-04T15:54:39.444 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (0 ms) 2024-03-04T15:54:39.444 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-03-04T15:54:39.444 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (4 ms) 2024-03-04T15:54:39.445 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [----------] 5 tests from LibRadosStat (272 ms total) 2024-03-04T15:54:39.445 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: 2024-03-04T15:54:39.445 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-03-04T15:54:39.445 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-03-04T15:54:39.445 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (2479 ms) 2024-03-04T15:54:39.445 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-03-04T15:54:39.445 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (72 ms) 2024-03-04T15:54:39.445 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-03-04T15:54:39.445 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (0 ms) 2024-03-04T15:54:39.445 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-03-04T15:54:39.445 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (11 ms) 2024-03-04T15:54:39.445 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (2562 ms total) 2024-03-04T15:54:39.445 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: 2024-03-04T15:54:39.445 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [----------] Global test environment tear-down 2024-03-04T15:54:39.445 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [==========] 9 tests from 2 test suites ran. (39438 ms total) 2024-03-04T15:54:39.446 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat: [ PASSED ] 9 tests. 2024-03-04T15:54:39.446 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: Running main() from gmock_main.cc 2024-03-04T15:54:39.446 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-03-04T15:54:39.446 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [----------] Global test environment set-up. 2024-03-04T15:54:39.446 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-03-04T15:54:39.446 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-03-04T15:54:39.446 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (276 ms) 2024-03-04T15:54:39.446 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-03-04T15:54:39.446 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-03-04T15:54:39.446 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-03-04T15:54:39.447 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: no timeout :/ 2024-03-04T15:54:39.447 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: no timeout :/ 2024-03-04T15:54:39.447 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: no timeout :/ 2024-03-04T15:54:39.447 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: no timeout :/ 2024-03-04T15:54:39.447 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: no timeout :/ 2024-03-04T15:54:39.447 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (34 ms) 2024-03-04T15:54:39.447 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-03-04T15:54:39.447 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (10 ms) 2024-03-04T15:54:39.447 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-03-04T15:54:39.447 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIo.Checksum (3 ms) 2024-03-04T15:54:39.447 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-03-04T15:54:39.447 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (7 ms) 2024-03-04T15:54:39.447 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-03-04T15:54:39.447 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (9 ms) 2024-03-04T15:54:39.448 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-03-04T15:54:39.448 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (5 ms) 2024-03-04T15:54:39.448 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-03-04T15:54:39.448 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (2 ms) 2024-03-04T15:54:39.448 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-03-04T15:54:39.448 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIo.TruncTest (6 ms) 2024-03-04T15:54:39.448 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-03-04T15:54:39.448 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (6 ms) 2024-03-04T15:54:39.448 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-03-04T15:54:39.448 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (5 ms) 2024-03-04T15:54:39.448 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-03-04T15:54:39.448 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIo.RmXattr (16 ms) 2024-03-04T15:54:39.448 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-03-04T15:54:39.448 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIo.XattrIter (5 ms) 2024-03-04T15:54:39.449 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [----------] 14 tests from LibRadosIo (384 ms total) 2024-03-04T15:54:39.449 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: 2024-03-04T15:54:39.449 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-03-04T15:54:39.449 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-03-04T15:54:39.449 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (2528 ms) 2024-03-04T15:54:39.449 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-03-04T15:54:39.449 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (32 ms) 2024-03-04T15:54:39.449 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-03-04T15:54:39.449 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (6 ms) 2024-03-04T15:54:39.449 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-03-04T15:54:39.449 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (6 ms) 2024-03-04T15:54:39.449 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-03-04T15:54:39.449 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (9 ms) 2024-03-04T15:54:39.449 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-03-04T15:54:39.449 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (6 ms) 2024-03-04T15:54:39.450 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-03-04T15:54:39.450 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (3 ms) 2024-03-04T15:54:39.450 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-03-04T15:54:39.450 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (4 ms) 2024-03-04T15:54:39.450 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-03-04T15:54:39.450 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (9 ms) 2024-03-04T15:54:39.450 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-03-04T15:54:39.450 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (3 ms) 2024-03-04T15:54:39.450 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [----------] 10 tests from LibRadosIoEC (2606 ms total) 2024-03-04T15:54:39.450 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: 2024-03-04T15:54:39.450 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [----- api_stat_pp: Running main() from gmock_main.cc 2024-03-04T15:54:39.451 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-03-04T15:54:39.451 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-03-04T15:54:39.451 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-03-04T15:54:39.451 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: seed 64189 2024-03-04T15:54:39.451 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-03-04T15:54:39.451 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (267 ms) 2024-03-04T15:54:39.451 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-03-04T15:54:39.451 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (3 ms) 2024-03-04T15:54:39.451 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-03-04T15:54:39.451 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (1 ms) 2024-03-04T15:54:39.451 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-03-04T15:54:39.451 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (2 ms) 2024-03-04T15:54:39.451 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-03-04T15:54:39.451 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (21 ms) 2024-03-04T15:54:39.451 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (295 ms total) 2024-03-04T15:54:39.452 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: 2024-03-04T15:54:39.452 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-03-04T15:54:39.452 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-03-04T15:54:39.452 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1566 ms) 2024-03-04T15:54:39.452 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-03-04T15:54:39.452 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (1 ms) 2024-03-04T15:54:39.452 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-03-04T15:54:39.452 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (6 ms) 2024-03-04T15:54:39.452 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-03-04T15:54:39.452 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (992 ms) 2024-03-04T15:54:39.452 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (2565 ms total) 2024-03-04T15:54:39.452 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: 2024-03-04T15:54:39.452 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-03-04T15:54:39.452 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (39423 ms total) 2024-03-04T15:54:39.452 INFO:tasks.workunit.client.0.smithi117.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-03-04T15:54:39.453 INFO:tasks.workunit.client.0.smithi117.stdout:-----] Global test environment tear-down 2024-03-04T15:54:39.453 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [==========] 24 tests from 2 test suites ran. (39558 ms total) 2024-03-04T15:54:39.453 INFO:tasks.workunit.client.0.smithi117.stdout: api_io: [ PASSED ] 24 tests. 2024-03-04T15:54:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: audit 2024-03-04T15:54:38.029200+0000 mon.a (mon.0) 1065 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7"}]': finished 2024-03-04T15:54:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: audit 2024-03-04T15:54:38.029400+0000 mon.a (mon.0) 1066 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi117-63931-16"}]': finished 2024-03-04T15:54:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: audit 2024-03-04T15:54:38.029556+0000 mon.a (mon.0) 1067 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi117-64169-7"}]': finished 2024-03-04T15:54:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: audit 2024-03-04T15:54:38.029676+0000 mon.a (mon.0) 1068 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: audit 2024-03-04T15:54:38.029789+0000 mon.a (mon.0) 1069 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: audit 2024-03-04T15:54:38.029887+0000 mon.a (mon.0) 1070 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: audit 2024-03-04T15:54:38.030008+0000 mon.a (mon.0) 1071 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: cluster 2024-03-04T15:54:38.038932+0000 mon.a (mon.0) 1072 : cluster 0 osdmap e96: 8 total, 8 up, 8 in 2024-03-04T15:54:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: audit 2024-03-04T15:54:38.040862+0000 mon.c (mon.2) 159 : audit 1 from='client.? 172.21.15.117:0/2687162500' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: audit 2024-03-04T15:54:38.041326+0000 mon.b (mon.1) 51 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app2"}]: dispatch 2024-03-04T15:54:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: audit 2024-03-04T15:54:38.043200+0000 mon.b (mon.1) 52 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: audit 2024-03-04T15:54:38.043569+0000 mon.c (mon.2) 160 : audit 1 from='client.? 172.21.15.117:0/3022824111' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: audit 2024-03-04T15:54:38.044581+0000 mon.c (mon.2) 161 : audit 1 from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: audit 2024-03-04T15:54:38.045060+0000 mon.c (mon.2) 162 : audit 1 from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: audit 2024-03-04T15:54:38.050526+0000 mon.c (mon.2) 163 : audit 1 from='client.? 172.21.15.117:0/1728754262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: cluster 2024-03-04T15:54:38.063438+0000 mon.a (mon.0) 1073 : cluster 3 Health check update: 16 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: cluster 2024-03-04T15:54:38.063856+0000 mon.a (mon.0) 1074 : cluster 3 pool 'PoolQuotaPP_smithi117-63923-3' is full (reached quota's max_bytes: 4 KiB) 2024-03-04T15:54:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: cluster 2024-03-04T15:54:38.065361+0000 mon.a (mon.0) 1075 : cluster 3 Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-04T15:54:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: audit 2024-03-04T15:54:38.068752+0000 mon.a (mon.0) 1076 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: audit 2024-03-04T15:54:38.069098+0000 mon.a (mon.0) 1077 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: audit 2024-03-04T15:54:38.070330+0000 mon.a (mon.0) 1078 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: audit 2024-03-04T15:54:38.070665+0000 mon.a (mon.0) 1079 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: audit 2024-03-04T15:54:38.071382+0000 mon.a (mon.0) 1080 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: audit 2024-03-04T15:54:38.071620+0000 mon.a (mon.0) 1081 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:39 smithi151 bash[23756]: cluster 2024-03-04T15:54:38.078352+0000 mon.a (mon.0) 1082 : cluster 0 osdmap e97: 8 total, 8 up, 8 in 2024-03-04T15:54:39.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: audit 2024-03-04T15:54:38.029200+0000 mon.a (mon.0) 1065 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7"}]': finished 2024-03-04T15:54:39.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: audit 2024-03-04T15:54:38.029400+0000 mon.a (mon.0) 1066 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi117-63931-16"}]': finished 2024-03-04T15:54:39.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: audit 2024-03-04T15:54:38.029556+0000 mon.a (mon.0) 1067 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi117-64169-7"}]': finished 2024-03-04T15:54:39.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: audit 2024-03-04T15:54:38.029676+0000 mon.a (mon.0) 1068 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:39.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: audit 2024-03-04T15:54:38.029789+0000 mon.a (mon.0) 1069 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:39.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: audit 2024-03-04T15:54:38.029887+0000 mon.a (mon.0) 1070 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:39.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: audit 2024-03-04T15:54:38.030008+0000 mon.a (mon.0) 1071 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:39.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: cluster 2024-03-04T15:54:38.038932+0000 mon.a (mon.0) 1072 : cluster 0 osdmap e96: 8 total, 8 up, 8 in 2024-03-04T15:54:39.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: audit 2024-03-04T15:54:38.040862+0000 mon.c (mon.2) 159 : audit 1 from='client.? 172.21.15.117:0/2687162500' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:39.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: audit 2024-03-04T15:54:38.041326+0000 mon.b (mon.1) 51 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app2"}]: dispatch 2024-03-04T15:54:39.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: audit 2024-03-04T15:54:38.043200+0000 mon.b (mon.1) 52 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:39.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: audit 2024-03-04T15:54:38.043569+0000 mon.c (mon.2) 160 : audit 1 from='client.? 172.21.15.117:0/3022824111' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:39.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: audit 2024-03-04T15:54:38.044581+0000 mon.c (mon.2) 161 : audit 1 from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:39.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: audit 2024-03-04T15:54:38.045060+0000 mon.c (mon.2) 162 : audit 1 from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:39.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: audit 2024-03-04T15:54:38.029200+0000 mon.a (mon.0) 1065 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi117-64189-7"}]': finished 2024-03-04T15:54:39.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: audit 2024-03-04T15:54:38.029400+0000 mon.a (mon.0) 1066 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi117-63931-16"}]': finished 2024-03-04T15:54:39.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: audit 2024-03-04T15:54:38.029556+0000 mon.a (mon.0) 1067 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi117-64169-7"}]': finished 2024-03-04T15:54:39.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: audit 2024-03-04T15:54:38.029676+0000 mon.a (mon.0) 1068 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:39.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: audit 2024-03-04T15:54:38.029789+0000 mon.a (mon.0) 1069 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:39.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: audit 2024-03-04T15:54:38.029887+0000 mon.a (mon.0) 1070 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:39.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: audit 2024-03-04T15:54:38.030008+0000 mon.a (mon.0) 1071 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:39.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: cluster 2024-03-04T15:54:38.038932+0000 mon.a (mon.0) 1072 : cluster 0 osdmap e96: 8 total, 8 up, 8 in 2024-03-04T15:54:39.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: audit 2024-03-04T15:54:38.040862+0000 mon.c (mon.2) 159 : audit 1 from='client.? 172.21.15.117:0/2687162500' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:39.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: audit 2024-03-04T15:54:38.041326+0000 mon.b (mon.1) 51 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app2"}]: dispatch 2024-03-04T15:54:39.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: audit 2024-03-04T15:54:38.043200+0000 mon.b (mon.1) 52 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:39.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: audit 2024-03-04T15:54:38.043569+0000 mon.c (mon.2) 160 : audit 1 from='client.? 172.21.15.117:0/3022824111' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:39.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: audit 2024-03-04T15:54:38.044581+0000 mon.c (mon.2) 161 : audit 1 from='client.? 172.21.15.117:0/1746895391' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:39.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: audit 2024-03-04T15:54:38.045060+0000 mon.c (mon.2) 162 : audit 1 from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:39.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: audit 2024-03-04T15:54:38.050526+0000 mon.c (mon.2) 163 : audit 1 from='client.? 172.21.15.117:0/1728754262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:39.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: cluster 2024-03-04T15:54:38.063438+0000 mon.a (mon.0) 1073 : cluster 3 Health check update: 16 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:39.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: cluster 2024-03-04T15:54:38.063856+0000 mon.a (mon.0) 1074 : cluster 3 pool 'PoolQuotaPP_smithi117-63923-3' is full (reached quota's max_bytes: 4 KiB) 2024-03-04T15:54:39.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: cluster 2024-03-04T15:54:38.065361+0000 mon.a (mon.0) 1075 : cluster 3 Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-04T15:54:39.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: audit 2024-03-04T15:54:38.068752+0000 mon.a (mon.0) 1076 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:39.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: audit 2024-03-04T15:54:38.069098+0000 mon.a (mon.0) 1077 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:39.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: audit 2024-03-04T15:54:38.070330+0000 mon.a (mon.0) 1078 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:39.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: audit 2024-03-04T15:54:38.070665+0000 mon.a (mon.0) 1079 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:39.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: audit 2024-03-04T15:54:38.071382+0000 mon.a (mon.0) 1080 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:39.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: audit 2024-03-04T15:54:38.071620+0000 mon.a (mon.0) 1081 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:39.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[28843]: cluster 2024-03-04T15:54:38.078352+0000 mon.a (mon.0) 1082 : cluster 0 osdmap e97: 8 total, 8 up, 8 in 2024-03-04T15:54:39.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: audit 2024-03-04T15:54:38.050526+0000 mon.c (mon.2) 163 : audit 1 from='client.? 172.21.15.117:0/1728754262' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:39.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: cluster 2024-03-04T15:54:38.063438+0000 mon.a (mon.0) 1073 : cluster 3 Health check update: 16 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:39.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: cluster 2024-03-04T15:54:38.063856+0000 mon.a (mon.0) 1074 : cluster 3 pool 'PoolQuotaPP_smithi117-63923-3' is full (reached quota's max_bytes: 4 KiB) 2024-03-04T15:54:39.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: cluster 2024-03-04T15:54:38.065361+0000 mon.a (mon.0) 1075 : cluster 3 Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-04T15:54:39.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: audit 2024-03-04T15:54:38.068752+0000 mon.a (mon.0) 1076 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:39.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: audit 2024-03-04T15:54:38.069098+0000 mon.a (mon.0) 1077 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi117-64189-7"}]: dispatch 2024-03-04T15:54:39.823 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: audit 2024-03-04T15:54:38.070330+0000 mon.a (mon.0) 1078 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi117-64169-7"}]: dispatch 2024-03-04T15:54:39.823 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: audit 2024-03-04T15:54:38.070665+0000 mon.a (mon.0) 1079 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi117-63931-16"}]: dispatch 2024-03-04T15:54:39.823 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: audit 2024-03-04T15:54:38.071382+0000 mon.a (mon.0) 1080 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:39.823 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: audit 2024-03-04T15:54:38.071620+0000 mon.a (mon.0) 1081 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:39.823 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:39 smithi117 bash[21480]: cluster 2024-03-04T15:54:38.078352+0000 mon.a (mon.0) 1082 : cluster 0 osdmap e97: 8 total, 8 up, 8 in 2024-03-04T15:54:40.841 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: Running main() from gmock_main.cc 2024-03-04T15:54:40.841 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-03-04T15:54:40.841 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [----------] Global test environment set-up. 2024-03-04T15:54:40.841 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-03-04T15:54:40.841 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-03-04T15:54:40.842 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (228 ms) 2024-03-04T15:54:40.842 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-03-04T15:54:40.842 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ OK ] LibRadosLock.LockShared (5 ms) 2024-03-04T15:54:40.842 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-03-04T15:54:40.842 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1023 ms) 2024-03-04T15:54:40.842 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-03-04T15:54:40.842 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1007 ms) 2024-03-04T15:54:40.842 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-03-04T15:54:40.842 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (4 ms) 2024-03-04T15:54:40.842 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-03-04T15:54:40.842 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ OK ] LibRadosLock.Unlock (8 ms) 2024-03-04T15:54:40.842 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-03-04T15:54:40.842 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (10 ms) 2024-03-04T15:54:40.842 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-03-04T15:54:40.842 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (5 ms) 2024-03-04T15:54:40.842 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [----------] 8 tests from LibRadosLock (2290 ms total) 2024-03-04T15:54:40.843 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: 2024-03-04T15:54:40.843 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-03-04T15:54:40.843 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-03-04T15:54:40.843 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1555 ms) 2024-03-04T15:54:40.843 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-03-04T15:54:40.843 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (13 ms) 2024-03-04T15:54:40.843 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-03-04T15:54:40.843 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1980 ms) 2024-03-04T15:54:40.843 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-03-04T15:54:40.843 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1012 ms) 2024-03-04T15:54:40.843 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-03-04T15:54:40.843 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (8 ms) 2024-03-04T15:54:40.843 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-03-04T15:54:40.843 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (10 ms) 2024-03-04T15:54:40.843 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-03-04T15:54:40.844 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (17 ms) 2024-03-04T15:54:40.844 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-03-04T15:54:40.844 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (8 ms) 2024-03-04T15:54:40.844 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (4603 ms total) 2024-03-04T15:54:40.844 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: 2024-03-04T15:54:40.844 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [----------] Global test environment tear-down 2024-03-04T15:54:40.844 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (40933 ms total) 2024-03-04T15:54:40.844 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock: [ PASSED ] 16 tests. 2024-03-04T15:54:40.890 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-03-04T15:54:40.891 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-03-04T15:54:40.891 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-03-04T15:54:40.891 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-03-04T15:54:40.891 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: seed 63986 2024-03-04T15:54:40.891 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-03-04T15:54:40.891 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (278 ms) 2024-03-04T15:54:40.891 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-03-04T15:54:40.891 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (7 ms) 2024-03-04T15:54:40.891 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-03-04T15:54:40.891 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1009 ms) 2024-03-04T15:54:40.891 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-03-04T15:54:40.891 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1009 ms) 2024-03-04T15:54:40.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-03-04T15:54:40.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (6 ms) 2024-03-04T15:54:40.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-03-04T15:54:40.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (6 ms) 2024-03-04T15:54:40.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-03-04T15:54:40.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (8 ms) 2024-03-04T15:54:40.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-03-04T15:54:40.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (7 ms) 2024-03-04T15:54:40.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2331 ms total) 2024-03-04T15:54:40.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: 2024-03-04T15:54:40.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-03-04T15:54:40.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-03-04T15:54:40.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (1521 ms) 2024-03-04T15:54:40.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-03-04T15:54:40.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (1027 ms) 2024-03-04T15:54:40.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-03-04T15:54:40.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1017 ms) 2024-03-04T15:54:40.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-03-04T15:54:40.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1010 ms) 2024-03-04T15:54:40.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-03-04T15:54:40.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (12 ms) 2024-03-04T15:54:40.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-03-04T15:54:40.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (13 ms) 2024-03-04T15:54:40.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-03-04T15:54:40.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (15 ms) 2024-03-04T15:54:40.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-03-04T15:54:40.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (5 ms) 2024-03-04T15:54:40.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (4620 ms total) 2024-03-04T15:54:40.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: 2024-03-04T15:54:40.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-03-04T15:54:40.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (40955 ms total) 2024-03-04T15:54:40.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-03-04T15:54:41.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: cluster 2024-03-04T15:54:37.665340+0000 mgr.y (mgr.24428) 195 : cluster 0 pgmap v145: 876 pgs: 177 creating+peering, 72 unknown, 627 active+clean; 145 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 7.3 KiB/s rd, 8.5 KiB/s wr, 23 op/s 2024-03-04T15:54:41.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:38.998042+0000 mon.a (mon.0) 1083 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64254-6", "pg_num": 4}]: dispatch 2024-03-04T15:54:41.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:39.430165+0000 mon.a (mon.0) 1084 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:41.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:39.431967+0000 mon.a (mon.0) 1085 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:41.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:39.432102+0000 mon.a (mon.0) 1086 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi117-64189-7"}]': finished 2024-03-04T15:54:41.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:39.432167+0000 mon.a (mon.0) 1087 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi117-64169-7"}]': finished 2024-03-04T15:54:41.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:39.432250+0000 mon.a (mon.0) 1088 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi117-63931-16"}]': finished 2024-03-04T15:54:41.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:39.432330+0000 mon.a (mon.0) 1089 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi117-63960-10"}]': finished 2024-03-04T15:54:41.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:39.432411+0000 mon.a (mon.0) 1090 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10"}]': finished 2024-03-04T15:54:41.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:39.432501+0000 mon.a (mon.0) 1091 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64254-6", "pg_num": 4}]': finished 2024-03-04T15:54:41.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: cluster 2024-03-04T15:54:39.436862+0000 mon.a (mon.0) 1092 : cluster 0 osdmap e98: 8 total, 8 up, 8 in 2024-03-04T15:54:41.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:39.439357+0000 mon.c (mon.2) 164 : audit 1 from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:41.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:39.439798+0000 mon.c (mon.2) 165 : audit 1 from='client.? 172.21.15.117:0/433345339' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi117-63915-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:41.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:39.441524+0000 mon.c (mon.2) 166 : audit 1 from='client.? 172.21.15.117:0/1728754262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:41.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:39.443559+0000 mon.a (mon.0) 1093 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4", "tierpool": "test-rados-api-smithi117-64254-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:54:41.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: cluster 2024-03-04T15:54:37.665340+0000 mgr.y (mgr.24428) 195 : cluster 0 pgmap v145: 876 pgs: 177 creating+peering, 72 unknown, 627 active+clean; 145 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 7.3 KiB/s rd, 8.5 KiB/s wr, 23 op/s 2024-03-04T15:54:41.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:38.998042+0000 mon.a (mon.0) 1083 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64254-6", "pg_num": 4}]: dispatch 2024-03-04T15:54:41.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:39.430165+0000 mon.a (mon.0) 1084 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:41.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:39.431967+0000 mon.a (mon.0) 1085 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:41.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:39.432102+0000 mon.a (mon.0) 1086 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi117-64189-7"}]': finished 2024-03-04T15:54:41.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:39.432167+0000 mon.a (mon.0) 1087 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi117-64169-7"}]': finished 2024-03-04T15:54:41.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:39.432250+0000 mon.a (mon.0) 1088 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi117-63931-16"}]': finished 2024-03-04T15:54:41.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:39.432330+0000 mon.a (mon.0) 1089 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi117-63960-10"}]': finished 2024-03-04T15:54:41.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:39.432411+0000 mon.a (mon.0) 1090 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10"}]': finished 2024-03-04T15:54:41.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:39.432501+0000 mon.a (mon.0) 1091 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64254-6", "pg_num": 4}]': finished 2024-03-04T15:54:41.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: cluster 2024-03-04T15:54:39.436862+0000 mon.a (mon.0) 1092 : cluster 0 osdmap e98: 8 total, 8 up, 8 in 2024-03-04T15:54:41.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:39.439357+0000 mon.c (mon.2) 164 : audit 1 from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:41.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:39.439798+0000 mon.c (mon.2) 165 : audit 1 from='client.? 172.21.15.117:0/433345339' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi117-63915-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:41.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:39.441524+0000 mon.c (mon.2) 166 : audit 1 from='client.? 172.21.15.117:0/1728754262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:41.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:39.443559+0000 mon.a (mon.0) 1093 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4", "tierpool": "test-rados-api-smithi117-64254-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:54:41.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:39.445800+0000 mon.b (mon.1) 53 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-03-04T15:54:41.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:39.447265+0000 mon.b (mon.1) 54 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-04T15:54:41.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:40.365951+0000 mon.a (mon.0) 1094 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:41.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:40.366876+0000 mon.a (mon.0) 1095 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi117-63915-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:41.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:40.372193+0000 mon.a (mon.0) 1096 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:54:41.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:40.372931+0000 mon.a (mon.0) 1097 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:41.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:40.374227+0000 mon.c (mon.2) 167 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:54:41.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[21480]: audit 2024-03-04T15:54:40.380944+0000 mon.a (mon.0) 1098 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-04T15:54:41.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:39.445800+0000 mon.b (mon.1) 53 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-03-04T15:54:41.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:39.447265+0000 mon.b (mon.1) 54 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-04T15:54:41.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:40.365951+0000 mon.a (mon.0) 1094 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:41.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:40.366876+0000 mon.a (mon.0) 1095 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi117-63915-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:41.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:40.372193+0000 mon.a (mon.0) 1096 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:54:41.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:40.372931+0000 mon.a (mon.0) 1097 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:41.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:40.374227+0000 mon.c (mon.2) 167 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:54:41.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:40 smithi117 bash[28843]: audit 2024-03-04T15:54:40.380944+0000 mon.a (mon.0) 1098 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-04T15:54:41.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: cluster 2024-03-04T15:54:37.665340+0000 mgr.y (mgr.24428) 195 : cluster 0 pgmap v145: 876 pgs: 177 creating+peering, 72 unknown, 627 active+clean; 145 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 7.3 KiB/s rd, 8.5 KiB/s wr, 23 op/s 2024-03-04T15:54:41.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:38.998042+0000 mon.a (mon.0) 1083 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64254-6", "pg_num": 4}]: dispatch 2024-03-04T15:54:41.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:39.430165+0000 mon.a (mon.0) 1084 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:41.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:39.431967+0000 mon.a (mon.0) 1085 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi117-63990-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:41.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:39.432102+0000 mon.a (mon.0) 1086 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi117-64189-7"}]': finished 2024-03-04T15:54:41.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:39.432167+0000 mon.a (mon.0) 1087 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi117-64169-7"}]': finished 2024-03-04T15:54:41.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:39.432250+0000 mon.a (mon.0) 1088 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi117-63931-16"}]': finished 2024-03-04T15:54:41.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:39.432330+0000 mon.a (mon.0) 1089 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi117-63960-10"}]': finished 2024-03-04T15:54:41.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:39.432411+0000 mon.a (mon.0) 1090 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi117-63986-10"}]': finished 2024-03-04T15:54:41.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:39.432501+0000 mon.a (mon.0) 1091 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64254-6", "pg_num": 4}]': finished 2024-03-04T15:54:41.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: cluster 2024-03-04T15:54:39.436862+0000 mon.a (mon.0) 1092 : cluster 0 osdmap e98: 8 total, 8 up, 8 in 2024-03-04T15:54:41.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:39.439357+0000 mon.c (mon.2) 164 : audit 1 from='client.? 172.21.15.117:0/4270213921' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:41.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:39.439798+0000 mon.c (mon.2) 165 : audit 1 from='client.? 172.21.15.117:0/433345339' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi117-63915-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:41.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:39.441524+0000 mon.c (mon.2) 166 : audit 1 from='client.? 172.21.15.117:0/1728754262' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:41.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:39.443559+0000 mon.a (mon.0) 1093 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4", "tierpool": "test-rados-api-smithi117-64254-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:54:41.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:39.445800+0000 mon.b (mon.1) 53 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-03-04T15:54:41.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:39.447265+0000 mon.b (mon.1) 54 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-04T15:54:41.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:40.365951+0000 mon.a (mon.0) 1094 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi117-63960-10"}]: dispatch 2024-03-04T15:54:41.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:40.366876+0000 mon.a (mon.0) 1095 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi117-63915-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:41.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:40.372193+0000 mon.a (mon.0) 1096 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:54:41.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:40.372931+0000 mon.a (mon.0) 1097 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi117-63986-10"}]: dispatch 2024-03-04T15:54:41.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:40.374227+0000 mon.c (mon.2) 167 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:54:41.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:40 smithi151 bash[23756]: audit 2024-03-04T15:54:40.380944+0000 mon.a (mon.0) 1098 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-04T15:54:42.199 INFO:tasks.workunit.client.0.smithi117.stdout: pool: Running main() from gmock_main.cc 2024-03-04T15:54:42.199 INFO:tasks.workunit.client.0.smithi117.stdout: pool: [==========] Running 6 tests from 1 test suite. 2024-03-04T15:54:42.199 INFO:tasks.workunit.client.0.smithi117.stdout: pool: [----------] Global test environment set-up. 2024-03-04T15:54:42.199 INFO:tasks.workunit.client.0.smithi117.stdout: pool: [----------] 6 tests from NeoRadosPools 2024-03-04T15:54:42.200 INFO:tasks.workunit.client.0.smithi117.stdout: pool: [ RUN ] NeoRadosPools.PoolList 2024-03-04T15:54:42.200 INFO:tasks.workunit.client.0.smithi117.stdout: pool: [ OK ] NeoRadosPools.PoolList (18421 ms) 2024-03-04T15:54:42.200 INFO:tasks.workunit.client.0.smithi117.stdout: pool: [ RUN ] NeoRadosPools.PoolLookup 2024-03-04T15:54:42.200 INFO:tasks.workunit.client.0.smithi117.stdout: pool: [ OK ] NeoRadosPools.PoolLookup (6552 ms) 2024-03-04T15:54:42.200 INFO:tasks.workunit.client.0.smithi117.stdout: pool: [ RUN ] NeoRadosPools.PoolLookupOtherInstance 2024-03-04T15:54:42.200 INFO:tasks.workunit.client.0.smithi117.stdout: pool: [ OK ] NeoRadosPools.PoolLookupOtherInstance (4900 ms) 2024-03-04T15:54:42.200 INFO:tasks.workunit.client.0.smithi117.stdout: pool: [ RUN ] NeoRadosPools.PoolDelete 2024-03-04T15:54:42.200 INFO:tasks.workunit.client.0.smithi117.stdout: pool: [ OK ] NeoRadosPools.PoolDelete (6886 ms) 2024-03-04T15:54:42.200 INFO:tasks.workunit.client.0.smithi117.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateDelete 2024-03-04T15:54:42.200 INFO:tasks.workunit.client.0.smithi117.stdout: pool: [ OK ] NeoRadosPools.PoolCreateDelete (2417 ms) 2024-03-04T15:54:42.200 INFO:tasks.workunit.client.0.smithi117.stdout: pool: [ RUN ] NeoRadosPools.PoolCreateWithCrushRule 2024-03-04T15:54:42.200 INFO:tasks.workunit.client.0.smithi117.stdout: pool: [ OK ] NeoRadosPools.PoolCreateWithCrushRule (2743 ms) 2024-03-04T15:54:42.200 INFO:tasks.workunit.client.0.smithi117.stdout: pool: [----------] 6 tests from NeoRadosPools (41919 ms total) 2024-03-04T15:54:42.200 INFO:tasks.workunit.client.0.smithi117.stdout: pool: 2024-03-04T15:54:42.200 INFO:tasks.workunit.client.0.smithi117.stdout: pool: [----------] Global test environment tear-down 2024-03-04T15:54:42.201 INFO:tasks.workunit.client.0.smithi117.stdout: pool: [==========] 6 tests from 1 test suite ran. (41919 ms total) 2024-03-04T15:54:42.201 INFO:tasks.workunit.client.0.smithi117.stdout: pool: [ PASSED ] 6 tests. 2024-03-04T15:54:42.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:42 smithi151 bash[23756]: cluster 2024-03-04T15:54:39.666595+0000 mgr.y (mgr.24428) 196 : cluster 0 pgmap v149: 704 pgs: 112 creating+peering, 172 unknown, 420 active+clean; 145 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 7.1 KiB/s wr, 5 op/s 2024-03-04T15:54:42.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:42 smithi151 bash[23756]: audit 2024-03-04T15:54:40.806087+0000 mon.a (mon.0) 1099 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:42.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:42 smithi151 bash[23756]: audit 2024-03-04T15:54:40.816231+0000 mon.a (mon.0) 1100 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4", "tierpool": "test-rados-api-smithi117-64254-6", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:54:42.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:42 smithi151 bash[23756]: audit 2024-03-04T15:54:40.816312+0000 mon.a (mon.0) 1101 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi117-63960-10"}]': finished 2024-03-04T15:54:42.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:42 smithi151 bash[23756]: audit 2024-03-04T15:54:40.816454+0000 mon.a (mon.0) 1102 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi117-63915-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:42.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:42 smithi151 bash[23756]: audit 2024-03-04T15:54:40.816559+0000 mon.a (mon.0) 1103 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi117-63986-10"}]': finished 2024-03-04T15:54:42.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:42 smithi151 bash[23756]: audit 2024-03-04T15:54:40.816639+0000 mon.a (mon.0) 1104 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-04T15:54:42.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:42 smithi151 bash[23756]: cluster 2024-03-04T15:54:40.822378+0000 mon.a (mon.0) 1105 : cluster 0 osdmap e99: 8 total, 8 up, 8 in 2024-03-04T15:54:42.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:42 smithi151 bash[23756]: audit 2024-03-04T15:54:40.826520+0000 mon.b (mon.1) 55 : audit 1 from='client.? 172.21.15.117:0/807910892' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:42.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:42 smithi151 bash[23756]: audit 2024-03-04T15:54:40.830702+0000 mon.b (mon.1) 56 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-04T15:54:42.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:42 smithi151 bash[23756]: audit 2024-03-04T15:54:40.834601+0000 mon.a (mon.0) 1106 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4", "overlaypool": "test-rados-api-smithi117-64254-6"}]: dispatch 2024-03-04T15:54:42.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:42 smithi151 bash[23756]: audit 2024-03-04T15:54:40.837714+0000 mon.c (mon.2) 168 : audit 1 from='client.? 172.21.15.117:0/3387966464' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:42.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:42 smithi151 bash[23756]: audit 2024-03-04T15:54:40.838211+0000 mon.a (mon.0) 1107 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:42.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:42 smithi151 bash[23756]: audit 2024-03-04T15:54:40.838311+0000 mon.c (mon.2) 169 : audit 1 from='client.? 172.21.15.117:0/3288255947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:42.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:42 smithi151 bash[23756]: audit 2024-03-04T15:54:40.838688+0000 mon.a (mon.0) 1108 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:42.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:42 smithi151 bash[23756]: audit 2024-03-04T15:54:40.845304+0000 mon.c (mon.2) 170 : audit 1 from='client.? 172.21.15.117:0/3879512807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi117-64656-2"}]: dispatch 2024-03-04T15:54:42.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:42 smithi151 bash[23756]: audit 2024-03-04T15:54:40.855032+0000 mon.a (mon.0) 1109 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi117-64656-2"}]: dispatch 2024-03-04T15:54:42.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:42 smithi151 bash[23756]: audit 2024-03-04T15:54:40.856743+0000 mon.a (mon.0) 1110 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:42.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:42 smithi151 bash[23756]: audit 2024-03-04T15:54:40.857044+0000 mon.a (mon.0) 1111 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-04T15:54:42.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[21480]: cluster 2024-03-04T15:54:39.666595+0000 mgr.y (mgr.24428) 196 : cluster 0 pgmap v149: 704 pgs: 112 creating+peering, 172 unknown, 420 active+clean; 145 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 7.1 KiB/s wr, 5 op/s 2024-03-04T15:54:42.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[21480]: audit 2024-03-04T15:54:40.806087+0000 mon.a (mon.0) 1099 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:42.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[21480]: audit 2024-03-04T15:54:40.816231+0000 mon.a (mon.0) 1100 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4", "tierpool": "test-rados-api-smithi117-64254-6", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:54:42.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[21480]: audit 2024-03-04T15:54:40.816312+0000 mon.a (mon.0) 1101 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi117-63960-10"}]': finished 2024-03-04T15:54:42.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[21480]: audit 2024-03-04T15:54:40.816454+0000 mon.a (mon.0) 1102 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi117-63915-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:42.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[21480]: audit 2024-03-04T15:54:40.816559+0000 mon.a (mon.0) 1103 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi117-63986-10"}]': finished 2024-03-04T15:54:42.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[21480]: audit 2024-03-04T15:54:40.816639+0000 mon.a (mon.0) 1104 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-04T15:54:42.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[21480]: cluster 2024-03-04T15:54:40.822378+0000 mon.a (mon.0) 1105 : cluster 0 osdmap e99: 8 total, 8 up, 8 in 2024-03-04T15:54:42.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[21480]: audit 2024-03-04T15:54:40.826520+0000 mon.b (mon.1) 55 : audit 1 from='client.? 172.21.15.117:0/807910892' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:42.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[21480]: audit 2024-03-04T15:54:40.830702+0000 mon.b (mon.1) 56 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-04T15:54:42.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[21480]: audit 2024-03-04T15:54:40.834601+0000 mon.a (mon.0) 1106 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4", "overlaypool": "test-rados-api-smithi117-64254-6"}]: dispatch 2024-03-04T15:54:42.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[21480]: audit 2024-03-04T15:54:40.837714+0000 mon.c (mon.2) 168 : audit 1 from='client.? 172.21.15.117:0/3387966464' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:42.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[21480]: audit 2024-03-04T15:54:40.838211+0000 mon.a (mon.0) 1107 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:42.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[21480]: audit 2024-03-04T15:54:40.838311+0000 mon.c (mon.2) 169 : audit 1 from='client.? 172.21.15.117:0/3288255947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:42.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[28843]: cluster 2024-03-04T15:54:39.666595+0000 mgr.y (mgr.24428) 196 : cluster 0 pgmap v149: 704 pgs: 112 creating+peering, 172 unknown, 420 active+clean; 145 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 7.1 KiB/s wr, 5 op/s 2024-03-04T15:54:42.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[28843]: audit 2024-03-04T15:54:40.806087+0000 mon.a (mon.0) 1099 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:42.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[28843]: audit 2024-03-04T15:54:40.816231+0000 mon.a (mon.0) 1100 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4", "tierpool": "test-rados-api-smithi117-64254-6", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:54:42.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[28843]: audit 2024-03-04T15:54:40.816312+0000 mon.a (mon.0) 1101 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi117-63960-10"}]': finished 2024-03-04T15:54:42.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[28843]: audit 2024-03-04T15:54:40.816454+0000 mon.a (mon.0) 1102 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi117-63915-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:42.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[28843]: audit 2024-03-04T15:54:40.816559+0000 mon.a (mon.0) 1103 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi117-63986-10"}]': finished 2024-03-04T15:54:42.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[28843]: audit 2024-03-04T15:54:40.816639+0000 mon.a (mon.0) 1104 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-04T15:54:42.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[28843]: cluster 2024-03-04T15:54:40.822378+0000 mon.a (mon.0) 1105 : cluster 0 osdmap e99: 8 total, 8 up, 8 in 2024-03-04T15:54:42.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[28843]: audit 2024-03-04T15:54:40.826520+0000 mon.b (mon.1) 55 : audit 1 from='client.? 172.21.15.117:0/807910892' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:42.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[28843]: audit 2024-03-04T15:54:40.830702+0000 mon.b (mon.1) 56 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-04T15:54:42.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[28843]: audit 2024-03-04T15:54:40.834601+0000 mon.a (mon.0) 1106 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4", "overlaypool": "test-rados-api-smithi117-64254-6"}]: dispatch 2024-03-04T15:54:42.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[28843]: audit 2024-03-04T15:54:40.837714+0000 mon.c (mon.2) 168 : audit 1 from='client.? 172.21.15.117:0/3387966464' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:42.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[28843]: audit 2024-03-04T15:54:40.838211+0000 mon.a (mon.0) 1107 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:42.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[28843]: audit 2024-03-04T15:54:40.838311+0000 mon.c (mon.2) 169 : audit 1 from='client.? 172.21.15.117:0/3288255947' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:42.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[28843]: audit 2024-03-04T15:54:40.838688+0000 mon.a (mon.0) 1108 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:42.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[28843]: audit 2024-03-04T15:54:40.845304+0000 mon.c (mon.2) 170 : audit 1 from='client.? 172.21.15.117:0/3879512807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi117-64656-2"}]: dispatch 2024-03-04T15:54:42.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[28843]: audit 2024-03-04T15:54:40.855032+0000 mon.a (mon.0) 1109 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi117-64656-2"}]: dispatch 2024-03-04T15:54:42.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[28843]: audit 2024-03-04T15:54:40.856743+0000 mon.a (mon.0) 1110 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:42.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[28843]: audit 2024-03-04T15:54:40.857044+0000 mon.a (mon.0) 1111 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-04T15:54:42.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[21480]: audit 2024-03-04T15:54:40.838688+0000 mon.a (mon.0) 1108 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:42.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[21480]: audit 2024-03-04T15:54:40.845304+0000 mon.c (mon.2) 170 : audit 1 from='client.? 172.21.15.117:0/3879512807' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi117-64656-2"}]: dispatch 2024-03-04T15:54:42.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[21480]: audit 2024-03-04T15:54:40.855032+0000 mon.a (mon.0) 1109 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi117-64656-2"}]: dispatch 2024-03-04T15:54:42.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[21480]: audit 2024-03-04T15:54:40.856743+0000 mon.a (mon.0) 1110 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:42.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:42 smithi117 bash[21480]: audit 2024-03-04T15:54:40.857044+0000 mon.a (mon.0) 1111 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-04T15:54:43.218 INFO:tasks.workunit.client.0.smithi117.stdout: LibRadosIoECPP.OverlappingWriteRoundTripPP 2024-03-04T15:54:43.218 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (4 ms) 2024-03-04T15:54:43.218 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-03-04T15:54:43.218 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (7 ms) 2024-03-04T15:54:43.218 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-03-04T15:54:43.219 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (4 ms) 2024-03-04T15:54:43.219 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-03-04T15:54:43.219 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (5 ms) 2024-03-04T15:54:43.219 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-03-04T15:54:43.219 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (4 ms) 2024-03-04T15:54:43.219 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-03-04T15:54:43.219 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (3 ms) 2024-03-04T15:54:43.219 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-03-04T15:54:43.219 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (3 ms) 2024-03-04T15:54:43.219 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-03-04T15:54:43.219 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (10 ms) 2024-03-04T15:54:43.219 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-03-04T15:54:43.219 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (4 ms) 2024-03-04T15:54:43.220 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-03-04T15:54:43.220 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (10 ms) 2024-03-04T15:54:43.220 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-03-04T15:54:43.220 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (3 ms) 2024-03-04T15:54:43.220 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-03-04T15:54:43.220 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (6 ms) 2024-03-04T15:54:43.220 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (1350 ms total) 2024-03-04T15:54:43.220 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: 2024-03-04T15:54:43.221 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [----------] Global test environment tear-down 2024-03-04T15:54:43.221 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (43319 ms total) 2024-03-04T15:54:43.221 INFO:tasks.workunit.client.0.smithi117.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-03-04T15:54:43.257 INFO:tasks.workunit.client.0.smithi117.stdout: list: Running main() from gmock_main.cc 2024-03-04T15:54:43.258 INFO:tasks.workunit.client.0.smithi117.stdout: list: [==========] Running 3 tests from 1 test suite. 2024-03-04T15:54:43.258 INFO:tasks.workunit.client.0.smithi117.stdout: list: [----------] Global test environment set-up. 2024-03-04T15:54:43.258 INFO:tasks.workunit.client.0.smithi117.stdout: list: [----------] 3 tests from NeoradosList 2024-03-04T15:54:43.258 INFO:tasks.workunit.client.0.smithi117.stdout: list: [ RUN ] NeoradosList.ListObjects 2024-03-04T15:54:43.258 INFO:tasks.workunit.client.0.smithi117.stdout: list: [ OK ] NeoradosList.ListObjects (26439 ms) 2024-03-04T15:54:43.258 INFO:tasks.workunit.client.0.smithi117.stdout: list: [ RUN ] NeoradosList.ListObjectsNS 2024-03-04T15:54:43.258 INFO:tasks.workunit.client.0.smithi117.stdout: list: [ OK ] NeoradosList.ListObjectsNS (10363 ms) 2024-03-04T15:54:43.258 INFO:tasks.workunit.client.0.smithi117.stdout: list: [ RUN ] NeoradosList.ListObjectsMany 2024-03-04T15:54:43.258 INFO:tasks.workunit.client.0.smithi117.stdout: list: [ OK ] NeoradosList.ListObjectsMany (6174 ms) 2024-03-04T15:54:43.258 INFO:tasks.workunit.client.0.smithi117.stdout: list: [----------] 3 tests from NeoradosList (42976 ms total) 2024-03-04T15:54:43.258 INFO:tasks.workunit.client.0.smithi117.stdout: list: 2024-03-04T15:54:43.258 INFO:tasks.workunit.client.0.smithi117.stdout: list: [----------] Global test environment tear-down 2024-03-04T15:54:43.258 INFO:tasks.workunit.client.0.smithi117.stdout: list: [==========] 3 tests from 1 test suite ran. (42976 ms total) 2024-03-04T15:54:43.259 INFO:tasks.workunit.client.0.smithi117.stdout: list: [ PASSED ] 3 tests. 2024-03-04T15:54:43.288 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: Running main() from gmock_main.cc 2024-03-04T15:54:43.288 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-03-04T15:54:43.289 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-03-04T15:54:43.289 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-03-04T15:54:43.289 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-03-04T15:54:43.289 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: watch_notify_test_cb 2024-03-04T15:54:43.289 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (259 ms) 2024-03-04T15:54:43.289 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-03-04T15:54:43.289 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94383330759568 err -107 2024-03-04T15:54:43.289 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-03-04T15:54:43.289 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (17 ms) 2024-03-04T15:54:43.289 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-03-04T15:54:43.289 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94383330852560 err -107 2024-03-04T15:54:43.289 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-03-04T15:54:43.289 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (11 ms) 2024-03-04T15:54:43.289 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-03-04T15:54:43.290 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: watch_notify2_test_cb from 24634 notify_id 343597383680 cookie 94383330866736 2024-03-04T15:54:43.290 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (13 ms) 2024-03-04T15:54:43.290 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-03-04T15:54:43.290 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: watch_notify2_test_cb from 24634 notify_id 343597383680 cookie 94383330904720 2024-03-04T15:54:43.290 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (5 ms) 2024-03-04T15:54:43.290 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-03-04T15:54:43.290 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: watch_notify2_test_cb from 24634 notify_id 343597383681 cookie 94383330904720 2024-03-04T15:54:43.290 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (6 ms) 2024-03-04T15:54:43.290 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-03-04T15:54:43.290 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: watch_notify2_test_cb from 24634 notify_id 343597383680 cookie 94383330889856 2024-03-04T15:54:43.290 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: watch_notify2_test_cb from 24634 notify_id 343597383680 cookie 94383330892192 2024-03-04T15:54:43.290 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (13 ms) 2024-03-04T15:54:43.290 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-03-04T15:54:43.290 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: watch_notify2_test_cb from 24634 notify_id 343597383681 cookie 94383330885808 2024-03-04T15:54:43.290 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: watch_notify2_test_cb from 24634 notify_id 343597383682 cookie 94383330885808 2024-03-04T15:54:43.290 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3010 ms) 2024-03-04T15:54:43.291 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-03-04T15:54:43.291 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-03-04T15:54:43.291 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94383330885808 err -107 2024-03-04T15:54:43.291 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: watch_notify2_test_cb from 24634 notify_id 365072220162 cookie 94383330885808 2024-03-04T15:54:43.291 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (4018 ms) 2024-03-04T15:54:43.291 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-03-04T15:54:43.291 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-03-04T15:54:43.291 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94383330885808 err -107 2024-03-04T15:54:43.291 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1010 ms) 2024-03-04T15:54:43.291 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (8362 ms total) 2024-03-04T15:54:43.291 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: 2024-03-04T15:54:43.291 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-03-04T15:54:43.292 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-03-04T15:54:43.292 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: watch_notify_test_cb 2024-03-04T15:54:43.292 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (2548 ms) 2024-03-04T15:54:43.292 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (2548 ms total) 2024-03-04T15:54:43.292 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: 2024-03-04T15:54:43.292 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-03-04T15:54:43.292 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (43234 ms total) 2024-03-04T15:54:43.306 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-03-04T15:54:43.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: cluster 2024-03-04T15:54:41.671771+0000 mgr.y (mgr.24428) 197 : cluster 0 pgmap v151: 648 pgs: 123 creating+peering, 525 active+clean; 145 MiB data, 767 MiB used, 714 GiB / 715 GiB avail 2024-03-04T15:54:43.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: audit 2024-03-04T15:54:42.167475+0000 mon.a (mon.0) 1112 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: audit 2024-03-04T15:54:42.175603+0000 mon.a (mon.0) 1113 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4", "overlaypool": "test-rados-api-smithi117-64254-6"}]': finished 2024-03-04T15:54:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: audit 2024-03-04T15:54:42.175663+0000 mon.a (mon.0) 1114 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: audit 2024-03-04T15:54:42.175709+0000 mon.a (mon.0) 1115 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]': finished 2024-03-04T15:54:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: audit 2024-03-04T15:54:42.175753+0000 mon.a (mon.0) 1116 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi117-64656-2"}]': finished 2024-03-04T15:54:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: audit 2024-03-04T15:54:42.175796+0000 mon.a (mon.0) 1117 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22"}]': finished 2024-03-04T15:54:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: audit 2024-03-04T15:54:42.175848+0000 mon.a (mon.0) 1118 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-04T15:54:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: audit 2024-03-04T15:54:42.188160+0000 mon.b (mon.1) 57 : audit 1 from='client.? 172.21.15.117:0/807910892' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: audit 2024-03-04T15:54:42.191878+0000 mon.c (mon.2) 171 : audit 1 from='client.? 172.21.15.117:0/3879512807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi117-64656-2"}]: dispatch 2024-03-04T15:54:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: audit 2024-03-04T15:54:42.192446+0000 mon.c (mon.2) 172 : audit 1 from='client.? 172.21.15.117:0/3288255947' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: audit 2024-03-04T15:54:42.192836+0000 mon.c (mon.2) 173 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: audit 2024-03-04T15:54:42.198979+0000 mon.b (mon.1) 58 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key1"}]: dispatch 2024-03-04T15:54:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: cluster 2024-03-04T15:54:42.640580+0000 mon.a (mon.0) 1119 : cluster 0 osdmap e100: 8 total, 8 up, 8 in 2024-03-04T15:54:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: audit 2024-03-04T15:54:42.641868+0000 mon.a (mon.0) 1120 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64254-6", "mode": "writeback"}]: dispatch 2024-03-04T15:54:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: cluster 2024-03-04T15:54:41.671771+0000 mgr.y (mgr.24428) 197 : cluster 0 pgmap v151: 648 pgs: 123 creating+peering, 525 active+clean; 145 MiB data, 767 MiB used, 714 GiB / 715 GiB avail 2024-03-04T15:54:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: audit 2024-03-04T15:54:42.167475+0000 mon.a (mon.0) 1112 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: audit 2024-03-04T15:54:42.175603+0000 mon.a (mon.0) 1113 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4", "overlaypool": "test-rados-api-smithi117-64254-6"}]': finished 2024-03-04T15:54:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: audit 2024-03-04T15:54:42.175663+0000 mon.a (mon.0) 1114 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: audit 2024-03-04T15:54:42.175709+0000 mon.a (mon.0) 1115 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]': finished 2024-03-04T15:54:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: audit 2024-03-04T15:54:42.175753+0000 mon.a (mon.0) 1116 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi117-64656-2"}]': finished 2024-03-04T15:54:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: audit 2024-03-04T15:54:42.175796+0000 mon.a (mon.0) 1117 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22"}]': finished 2024-03-04T15:54:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: audit 2024-03-04T15:54:42.175848+0000 mon.a (mon.0) 1118 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-04T15:54:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: audit 2024-03-04T15:54:42.188160+0000 mon.b (mon.1) 57 : audit 1 from='client.? 172.21.15.117:0/807910892' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: audit 2024-03-04T15:54:42.191878+0000 mon.c (mon.2) 171 : audit 1 from='client.? 172.21.15.117:0/3879512807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi117-64656-2"}]: dispatch 2024-03-04T15:54:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: audit 2024-03-04T15:54:42.192446+0000 mon.c (mon.2) 172 : audit 1 from='client.? 172.21.15.117:0/3288255947' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: audit 2024-03-04T15:54:42.192836+0000 mon.c (mon.2) 173 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:43.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: audit 2024-03-04T15:54:42.198979+0000 mon.b (mon.1) 58 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key1"}]: dispatch 2024-03-04T15:54:43.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: cluster 2024-03-04T15:54:42.640580+0000 mon.a (mon.0) 1119 : cluster 0 osdmap e100: 8 total, 8 up, 8 in 2024-03-04T15:54:43.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: audit 2024-03-04T15:54:42.641868+0000 mon.a (mon.0) 1120 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64254-6", "mode": "writeback"}]: dispatch 2024-03-04T15:54:43.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: audit 2024-03-04T15:54:42.642075+0000 mon.a (mon.0) 1121 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:43.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: audit 2024-03-04T15:54:42.642318+0000 mon.a (mon.0) 1122 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi117-64656-2"}]: dispatch 2024-03-04T15:54:43.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: audit 2024-03-04T15:54:42.642659+0000 mon.a (mon.0) 1123 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key1"}]: dispatch 2024-03-04T15:54:43.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: audit 2024-03-04T15:54:42.642889+0000 mon.a (mon.0) 1124 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:43.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: audit 2024-03-04T15:54:42.643333+0000 mon.a (mon.0) 1125 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:43.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: cluster 2024-03-04T15:54:43.067266+0000 mon.a (mon.0) 1126 : cluster 3 Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:43.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[21480]: audit 2024-03-04T15:54:43.169364+0000 mon.a (mon.0) 1127 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:43.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: audit 2024-03-04T15:54:42.642075+0000 mon.a (mon.0) 1121 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:43.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: audit 2024-03-04T15:54:42.642318+0000 mon.a (mon.0) 1122 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi117-64656-2"}]: dispatch 2024-03-04T15:54:43.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: audit 2024-03-04T15:54:42.642659+0000 mon.a (mon.0) 1123 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key1"}]: dispatch 2024-03-04T15:54:43.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: audit 2024-03-04T15:54:42.642889+0000 mon.a (mon.0) 1124 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:43.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: audit 2024-03-04T15:54:42.643333+0000 mon.a (mon.0) 1125 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:43.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: cluster 2024-03-04T15:54:43.067266+0000 mon.a (mon.0) 1126 : cluster 3 Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:43.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:43 smithi117 bash[28843]: audit 2024-03-04T15:54:43.169364+0000 mon.a (mon.0) 1127 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: cluster 2024-03-04T15:54:41.671771+0000 mgr.y (mgr.24428) 197 : cluster 0 pgmap v151: 648 pgs: 123 creating+peering, 525 active+clean; 145 MiB data, 767 MiB used, 714 GiB / 715 GiB avail 2024-03-04T15:54:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: audit 2024-03-04T15:54:42.167475+0000 mon.a (mon.0) 1112 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: audit 2024-03-04T15:54:42.175603+0000 mon.a (mon.0) 1113 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4", "overlaypool": "test-rados-api-smithi117-64254-6"}]': finished 2024-03-04T15:54:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: audit 2024-03-04T15:54:42.175663+0000 mon.a (mon.0) 1114 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: audit 2024-03-04T15:54:42.175709+0000 mon.a (mon.0) 1115 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi117-64235-12"}]': finished 2024-03-04T15:54:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: audit 2024-03-04T15:54:42.175753+0000 mon.a (mon.0) 1116 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsNSsmithi117-64656-2"}]': finished 2024-03-04T15:54:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: audit 2024-03-04T15:54:42.175796+0000 mon.a (mon.0) 1117 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi117-63937-22"}]': finished 2024-03-04T15:54:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: audit 2024-03-04T15:54:42.175848+0000 mon.a (mon.0) 1118 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-04T15:54:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: audit 2024-03-04T15:54:42.188160+0000 mon.b (mon.1) 57 : audit 1 from='client.? 172.21.15.117:0/807910892' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: audit 2024-03-04T15:54:42.191878+0000 mon.c (mon.2) 171 : audit 1 from='client.? 172.21.15.117:0/3879512807' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi117-64656-2"}]: dispatch 2024-03-04T15:54:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: audit 2024-03-04T15:54:42.192446+0000 mon.c (mon.2) 172 : audit 1 from='client.? 172.21.15.117:0/3288255947' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: audit 2024-03-04T15:54:42.192836+0000 mon.c (mon.2) 173 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: audit 2024-03-04T15:54:42.198979+0000 mon.b (mon.1) 58 : audit 1 from='client.? 172.21.15.117:0/3068727900' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key1"}]: dispatch 2024-03-04T15:54:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: cluster 2024-03-04T15:54:42.640580+0000 mon.a (mon.0) 1119 : cluster 0 osdmap e100: 8 total, 8 up, 8 in 2024-03-04T15:54:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: audit 2024-03-04T15:54:42.641868+0000 mon.a (mon.0) 1120 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64254-6", "mode": "writeback"}]: dispatch 2024-03-04T15:54:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: audit 2024-03-04T15:54:42.642075+0000 mon.a (mon.0) 1121 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi117-63937-22"}]: dispatch 2024-03-04T15:54:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: audit 2024-03-04T15:54:42.642318+0000 mon.a (mon.0) 1122 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi117-64656-2"}]: dispatch 2024-03-04T15:54:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: audit 2024-03-04T15:54:42.642659+0000 mon.a (mon.0) 1123 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key1"}]: dispatch 2024-03-04T15:54:43.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: audit 2024-03-04T15:54:42.642889+0000 mon.a (mon.0) 1124 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi117-64235-12"}]: dispatch 2024-03-04T15:54:43.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: audit 2024-03-04T15:54:42.643333+0000 mon.a (mon.0) 1125 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:43.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: cluster 2024-03-04T15:54:43.067266+0000 mon.a (mon.0) 1126 : cluster 3 Health check update: 11 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:43.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:43 smithi151 bash[23756]: audit 2024-03-04T15:54:43.169364+0000 mon.a (mon.0) 1127 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:45.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:44 smithi151 bash[23756]: cluster 2024-03-04T15:54:43.181356+0000 mon.a (mon.0) 1128 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:54:45.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:44 smithi151 bash[23756]: audit 2024-03-04T15:54:43.184420+0000 mon.a (mon.0) 1129 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64254-6", "mode": "writeback"}]': finished 2024-03-04T15:54:45.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:44 smithi151 bash[23756]: audit 2024-03-04T15:54:43.184615+0000 mon.a (mon.0) 1130 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi117-63937-22"}]': finished 2024-03-04T15:54:45.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:44 smithi151 bash[23756]: audit 2024-03-04T15:54:43.184809+0000 mon.a (mon.0) 1131 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi117-64656-2"}]': finished 2024-03-04T15:54:45.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:44 smithi151 bash[23756]: audit 2024-03-04T15:54:43.185000+0000 mon.a (mon.0) 1132 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key1"}]': finished 2024-03-04T15:54:45.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:44 smithi151 bash[23756]: audit 2024-03-04T15:54:43.185242+0000 mon.a (mon.0) 1133 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi117-64235-12"}]': finished 2024-03-04T15:54:45.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:44 smithi151 bash[23756]: audit 2024-03-04T15:54:43.185450+0000 mon.a (mon.0) 1134 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:45.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:44 smithi151 bash[23756]: cluster 2024-03-04T15:54:43.197982+0000 mon.a (mon.0) 1135 : cluster 0 osdmap e101: 8 total, 8 up, 8 in 2024-03-04T15:54:45.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:44 smithi151 bash[23756]: audit 2024-03-04T15:54:43.205280+0000 mon.c (mon.2) 174 : audit 1 from='client.? 172.21.15.117:0/3434822709' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:45.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:44 smithi151 bash[23756]: audit 2024-03-04T15:54:43.210181+0000 mon.a (mon.0) 1136 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:45.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:44 smithi151 bash[23756]: audit 2024-03-04T15:54:43.314972+0000 mon.c (mon.2) 175 : audit 1 from='client.? 172.21.15.117:0/2761101374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi117-64656-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:45.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:44 smithi151 bash[23756]: audit 2024-03-04T15:54:43.791597+0000 mon.a (mon.0) 1137 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi117-64656-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:45.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[21480]: cluster 2024-03-04T15:54:43.181356+0000 mon.a (mon.0) 1128 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:54:45.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[21480]: audit 2024-03-04T15:54:43.184420+0000 mon.a (mon.0) 1129 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64254-6", "mode": "writeback"}]': finished 2024-03-04T15:54:45.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[21480]: audit 2024-03-04T15:54:43.184615+0000 mon.a (mon.0) 1130 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi117-63937-22"}]': finished 2024-03-04T15:54:45.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[21480]: audit 2024-03-04T15:54:43.184809+0000 mon.a (mon.0) 1131 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi117-64656-2"}]': finished 2024-03-04T15:54:45.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[21480]: audit 2024-03-04T15:54:43.185000+0000 mon.a (mon.0) 1132 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key1"}]': finished 2024-03-04T15:54:45.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[21480]: audit 2024-03-04T15:54:43.185242+0000 mon.a (mon.0) 1133 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi117-64235-12"}]': finished 2024-03-04T15:54:45.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[21480]: audit 2024-03-04T15:54:43.185450+0000 mon.a (mon.0) 1134 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:45.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[21480]: cluster 2024-03-04T15:54:43.197982+0000 mon.a (mon.0) 1135 : cluster 0 osdmap e101: 8 total, 8 up, 8 in 2024-03-04T15:54:45.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[21480]: audit 2024-03-04T15:54:43.205280+0000 mon.c (mon.2) 174 : audit 1 from='client.? 172.21.15.117:0/3434822709' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:45.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[21480]: audit 2024-03-04T15:54:43.210181+0000 mon.a (mon.0) 1136 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:45.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[21480]: audit 2024-03-04T15:54:43.314972+0000 mon.c (mon.2) 175 : audit 1 from='client.? 172.21.15.117:0/2761101374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi117-64656-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:45.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[21480]: audit 2024-03-04T15:54:43.791597+0000 mon.a (mon.0) 1137 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi117-64656-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:45.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[28843]: cluster 2024-03-04T15:54:43.181356+0000 mon.a (mon.0) 1128 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:54:45.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[28843]: audit 2024-03-04T15:54:43.184420+0000 mon.a (mon.0) 1129 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64254-6", "mode": "writeback"}]': finished 2024-03-04T15:54:45.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[28843]: audit 2024-03-04T15:54:43.184615+0000 mon.a (mon.0) 1130 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi117-63937-22"}]': finished 2024-03-04T15:54:45.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[28843]: audit 2024-03-04T15:54:43.184809+0000 mon.a (mon.0) 1131 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsNSsmithi117-64656-2"}]': finished 2024-03-04T15:54:45.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[28843]: audit 2024-03-04T15:54:43.185000+0000 mon.a (mon.0) 1132 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi117-63990-1","app":"app1","key":"key1"}]': finished 2024-03-04T15:54:45.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[28843]: audit 2024-03-04T15:54:43.185242+0000 mon.a (mon.0) 1133 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi117-64235-12"}]': finished 2024-03-04T15:54:45.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[28843]: audit 2024-03-04T15:54:43.185450+0000 mon.a (mon.0) 1134 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:45.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[28843]: cluster 2024-03-04T15:54:43.197982+0000 mon.a (mon.0) 1135 : cluster 0 osdmap e101: 8 total, 8 up, 8 in 2024-03-04T15:54:45.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[28843]: audit 2024-03-04T15:54:43.205280+0000 mon.c (mon.2) 174 : audit 1 from='client.? 172.21.15.117:0/3434822709' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:45.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[28843]: audit 2024-03-04T15:54:43.210181+0000 mon.a (mon.0) 1136 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:45.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[28843]: audit 2024-03-04T15:54:43.314972+0000 mon.c (mon.2) 175 : audit 1 from='client.? 172.21.15.117:0/2761101374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi117-64656-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:45.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:44 smithi117 bash[28843]: audit 2024-03-04T15:54:43.791597+0000 mon.a (mon.0) 1137 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi117-64656-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:54:46.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:54:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:54:46.575 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-03-04T15:54:46.575 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-03-04T15:54:46.575 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-03-04T15:54:46.575 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-03-04T15:54:46.576 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-03-04T15:54:46.576 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: notify 2024-03-04T15:54:46.576 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (7660 ms) 2024-03-04T15:54:46.576 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-03-04T15:54:46.576 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (4 ms) 2024-03-04T15:54:46.576 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (7664 ms total) 2024-03-04T15:54:46.576 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: 2024-03-04T15:54:46.576 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-03-04T15:54:46.576 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-03-04T15:54:46.576 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: notify 2024-03-04T15:54:46.576 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (188 ms) 2024-03-04T15:54:46.576 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-03-04T15:54:46.577 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: notify 2024-03-04T15:54:46.577 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (5155 ms) 2024-03-04T15:54:46.577 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-03-04T15:54:46.577 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (18 ms) 2024-03-04T15:54:46.577 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-03-04T15:54:46.577 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (19 ms) 2024-03-04T15:54:46.577 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-03-04T15:54:46.577 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: handle_notify cookie 94643924887632 notify_id 433791696898 notifier_gid 15291 2024-03-04T15:54:46.577 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (8 ms) 2024-03-04T15:54:46.577 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-03-04T15:54:46.577 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: handle_notify cookie 94643924887632 notify_id 433791696899 notifier_gid 15291 2024-03-04T15:54:46.577 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (9 ms) 2024-03-04T15:54:46.577 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-03-04T15:54:46.577 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: handle_notify cookie 94643924903760 notify_id 433791696900 notifier_gid 15291 2024-03-04T15:54:46.578 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (14 ms) 2024-03-04T15:54:46.578 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-03-04T15:54:46.578 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: handle_notify cookie 94643924903760 notify_id 433791696899 notifier_gid 15291 2024-03-04T15:54:46.578 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (23 ms) 2024-03-04T15:54:46.578 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-03-04T15:54:46.578 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: handle_notify cookie 94643924903760 notify_id 433791696897 notifier_gid 15291 2024-03-04T15:54:46.578 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (34 ms) 2024-03-04T15:54:46.578 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-03-04T15:54:46.578 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: handle_notify cookie 94643924903760 notify_id 433791696900 notifier_gid 15291 2024-03-04T15:54:46.578 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (25 ms) 2024-03-04T15:54:46.578 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-03-04T15:54:46.578 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: trying... 2024-03-04T15:54:46.578 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: handle_notify cookie 94643924903760 notify_id 433791696901 notifier_gid 15291 2024-03-04T15:54:46.578 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: timed out 2024-03-04T15:54:46.579 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: flushing 2024-03-04T15:54:46.932 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:54:46 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:54:47.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[28843]: cluster 2024-03-04T15:54:43.672573+0000 mgr.y (mgr.24428) 198 : cluster 0 pgmap v154: 584 pgs: 128 unknown, 33 creating+peering, 423 active+clean; 145 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 2.8 KiB/s wr, 2 op/s 2024-03-04T15:54:47.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[28843]: audit 2024-03-04T15:54:44.300870+0000 mon.a (mon.0) 1138 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:47.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[28843]: audit 2024-03-04T15:54:44.307912+0000 mon.a (mon.0) 1139 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:47.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[28843]: audit 2024-03-04T15:54:44.308013+0000 mon.a (mon.0) 1140 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi117-64656-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:47.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[28843]: audit 2024-03-04T15:54:44.337551+0000 mon.c (mon.2) 176 : audit 1 from='client.? 172.21.15.117:0/2761101374' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi117-64656-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi117-64656-3"}]: dispatch 2024-03-04T15:54:47.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[28843]: audit 2024-03-04T15:54:44.337886+0000 mon.c (mon.2) 177 : audit 1 from='client.? 172.21.15.117:0/4188757668' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:47.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[28843]: audit 2024-03-04T15:54:44.340134+0000 mon.c (mon.2) 178 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:47.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[28843]: cluster 2024-03-04T15:54:44.835981+0000 mon.a (mon.0) 1141 : cluster 0 osdmap e102: 8 total, 8 up, 8 in 2024-03-04T15:54:47.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[28843]: audit 2024-03-04T15:54:44.862898+0000 mon.a (mon.0) 1142 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi117-64656-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi117-64656-3"}]: dispatch 2024-03-04T15:54:47.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[28843]: audit 2024-03-04T15:54:44.863662+0000 mon.a (mon.0) 1143 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:47.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[28843]: audit 2024-03-04T15:54:44.864271+0000 mon.a (mon.0) 1144 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:47.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[28843]: cluster 2024-03-04T15:54:45.375693+0000 mon.a (mon.0) 1145 : cluster 1 Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-04T15:54:47.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[28843]: audit 2024-03-04T15:54:45.377775+0000 mon.a (mon.0) 1146 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:47.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[21480]: cluster 2024-03-04T15:54:43.672573+0000 mgr.y (mgr.24428) 198 : cluster 0 pgmap v154: 584 pgs: 128 unknown, 33 creating+peering, 423 active+clean; 145 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 2.8 KiB/s wr, 2 op/s 2024-03-04T15:54:47.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[21480]: audit 2024-03-04T15:54:44.300870+0000 mon.a (mon.0) 1138 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:47.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[21480]: audit 2024-03-04T15:54:44.307912+0000 mon.a (mon.0) 1139 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:47.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[21480]: audit 2024-03-04T15:54:44.308013+0000 mon.a (mon.0) 1140 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi117-64656-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:47.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[21480]: audit 2024-03-04T15:54:44.337551+0000 mon.c (mon.2) 176 : audit 1 from='client.? 172.21.15.117:0/2761101374' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi117-64656-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi117-64656-3"}]: dispatch 2024-03-04T15:54:47.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[21480]: audit 2024-03-04T15:54:44.337886+0000 mon.c (mon.2) 177 : audit 1 from='client.? 172.21.15.117:0/4188757668' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:47.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[21480]: audit 2024-03-04T15:54:44.340134+0000 mon.c (mon.2) 178 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:47.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[21480]: cluster 2024-03-04T15:54:44.835981+0000 mon.a (mon.0) 1141 : cluster 0 osdmap e102: 8 total, 8 up, 8 in 2024-03-04T15:54:47.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[21480]: audit 2024-03-04T15:54:44.862898+0000 mon.a (mon.0) 1142 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi117-64656-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi117-64656-3"}]: dispatch 2024-03-04T15:54:47.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[21480]: audit 2024-03-04T15:54:44.863662+0000 mon.a (mon.0) 1143 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:47.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[21480]: audit 2024-03-04T15:54:44.864271+0000 mon.a (mon.0) 1144 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:47.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[21480]: cluster 2024-03-04T15:54:45.375693+0000 mon.a (mon.0) 1145 : cluster 1 Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-04T15:54:47.321 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:46 smithi117 bash[21480]: audit 2024-03-04T15:54:45.377775+0000 mon.a (mon.0) 1146 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:47.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:46 smithi151 bash[23756]: cluster 2024-03-04T15:54:43.672573+0000 mgr.y (mgr.24428) 198 : cluster 0 pgmap v154: 584 pgs: 128 unknown, 33 creating+peering, 423 active+clean; 145 MiB data, 767 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 2.8 KiB/s wr, 2 op/s 2024-03-04T15:54:47.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:46 smithi151 bash[23756]: audit 2024-03-04T15:54:44.300870+0000 mon.a (mon.0) 1138 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:47.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:46 smithi151 bash[23756]: audit 2024-03-04T15:54:44.307912+0000 mon.a (mon.0) 1139 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:47.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:46 smithi151 bash[23756]: audit 2024-03-04T15:54:44.308013+0000 mon.a (mon.0) 1140 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ListObjectsManysmithi117-64656-3", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:54:47.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:46 smithi151 bash[23756]: audit 2024-03-04T15:54:44.337551+0000 mon.c (mon.2) 176 : audit 1 from='client.? 172.21.15.117:0/2761101374' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi117-64656-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi117-64656-3"}]: dispatch 2024-03-04T15:54:47.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:46 smithi151 bash[23756]: audit 2024-03-04T15:54:44.337886+0000 mon.c (mon.2) 177 : audit 1 from='client.? 172.21.15.117:0/4188757668' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:47.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:46 smithi151 bash[23756]: audit 2024-03-04T15:54:44.340134+0000 mon.c (mon.2) 178 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:47.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:46 smithi151 bash[23756]: cluster 2024-03-04T15:54:44.835981+0000 mon.a (mon.0) 1141 : cluster 0 osdmap e102: 8 total, 8 up, 8 in 2024-03-04T15:54:47.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:46 smithi151 bash[23756]: audit 2024-03-04T15:54:44.862898+0000 mon.a (mon.0) 1142 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi117-64656-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi117-64656-3"}]: dispatch 2024-03-04T15:54:47.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:46 smithi151 bash[23756]: audit 2024-03-04T15:54:44.863662+0000 mon.a (mon.0) 1143 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:47.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:46 smithi151 bash[23756]: audit 2024-03-04T15:54:44.864271+0000 mon.a (mon.0) 1144 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:47.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:46 smithi151 bash[23756]: cluster 2024-03-04T15:54:45.375693+0000 mon.a (mon.0) 1145 : cluster 1 Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-04T15:54:47.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:46 smithi151 bash[23756]: audit 2024-03-04T15:54:45.377775+0000 mon.a (mon.0) 1146 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:48.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:48 smithi151 bash[23756]: audit 2024-03-04T15:54:45.388898+0000 mon.a (mon.0) 1147 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:48.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:48 smithi151 bash[23756]: audit 2024-03-04T15:54:45.388998+0000 mon.a (mon.0) 1148 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:48.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:48 smithi151 bash[23756]: cluster 2024-03-04T15:54:46.943068+0000 mon.a (mon.0) 1149 : cluster 0 osdmap e103: 8 total, 8 up, 8 in 2024-03-04T15:54:48.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:48 smithi151 bash[23756]: audit 2024-03-04T15:54:46.950961+0000 mon.a (mon.0) 1150 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:48.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:48 smithi151 bash[23756]: audit 2024-03-04T15:54:46.961803+0000 mon.a (mon.0) 1151 : audit 1 from='client.? 172.21.15.117:0/3985467235' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi117-63915-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:48.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:48 smithi151 bash[23756]: audit 2024-03-04T15:54:46.973946+0000 mon.a (mon.0) 1152 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi117-64656-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi117-64656-3"}]': finished 2024-03-04T15:54:48.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:48 smithi151 bash[23756]: cluster 2024-03-04T15:54:46.984295+0000 mon.a (mon.0) 1153 : cluster 0 osdmap e104: 8 total, 8 up, 8 in 2024-03-04T15:54:48.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:48 smithi151 bash[23756]: audit 2024-03-04T15:54:47.076612+0000 mon.c (mon.2) 179 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:54:48.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:48 smithi117 bash[21480]: audit 2024-03-04T15:54:45.388898+0000 mon.a (mon.0) 1147 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:48.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:48 smithi117 bash[21480]: audit 2024-03-04T15:54:45.388998+0000 mon.a (mon.0) 1148 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:48.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:48 smithi117 bash[21480]: cluster 2024-03-04T15:54:46.943068+0000 mon.a (mon.0) 1149 : cluster 0 osdmap e103: 8 total, 8 up, 8 in 2024-03-04T15:54:48.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:48 smithi117 bash[21480]: audit 2024-03-04T15:54:46.950961+0000 mon.a (mon.0) 1150 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:48.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:48 smithi117 bash[21480]: audit 2024-03-04T15:54:46.961803+0000 mon.a (mon.0) 1151 : audit 1 from='client.? 172.21.15.117:0/3985467235' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi117-63915-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:48.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:48 smithi117 bash[21480]: audit 2024-03-04T15:54:46.973946+0000 mon.a (mon.0) 1152 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi117-64656-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi117-64656-3"}]': finished 2024-03-04T15:54:48.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:48 smithi117 bash[21480]: cluster 2024-03-04T15:54:46.984295+0000 mon.a (mon.0) 1153 : cluster 0 osdmap e104: 8 total, 8 up, 8 in 2024-03-04T15:54:48.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:48 smithi117 bash[21480]: audit 2024-03-04T15:54:47.076612+0000 mon.c (mon.2) 179 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:54:48.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:48 smithi117 bash[28843]: audit 2024-03-04T15:54:45.388898+0000 mon.a (mon.0) 1147 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:48.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:48 smithi117 bash[28843]: audit 2024-03-04T15:54:45.388998+0000 mon.a (mon.0) 1148 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:48.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:48 smithi117 bash[28843]: cluster 2024-03-04T15:54:46.943068+0000 mon.a (mon.0) 1149 : cluster 0 osdmap e103: 8 total, 8 up, 8 in 2024-03-04T15:54:48.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:48 smithi117 bash[28843]: audit 2024-03-04T15:54:46.950961+0000 mon.a (mon.0) 1150 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:48.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:48 smithi117 bash[28843]: audit 2024-03-04T15:54:46.961803+0000 mon.a (mon.0) 1151 : audit 1 from='client.? 172.21.15.117:0/3985467235' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi117-63915-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:48.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:48 smithi117 bash[28843]: audit 2024-03-04T15:54:46.973946+0000 mon.a (mon.0) 1152 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ListObjectsManysmithi117-64656-3", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ListObjectsManysmithi117-64656-3"}]': finished 2024-03-04T15:54:48.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:48 smithi117 bash[28843]: cluster 2024-03-04T15:54:46.984295+0000 mon.a (mon.0) 1153 : cluster 0 osdmap e104: 8 total, 8 up, 8 in 2024-03-04T15:54:48.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:48 smithi117 bash[28843]: audit 2024-03-04T15:54:47.076612+0000 mon.c (mon.2) 179 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:54:50.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:50 smithi117 bash[21480]: cluster 2024-03-04T15:54:45.673835+0000 mgr.y (mgr.24428) 199 : cluster 0 pgmap v157: 584 pgs: 192 unknown, 1 creating+peering, 391 active+clean; 145 MiB data, 767 MiB used, 714 GiB / 715 GiB avail 2024-03-04T15:54:50.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:50 smithi117 bash[21480]: audit 2024-03-04T15:54:46.494783+0000 mgr.y (mgr.24428) 200 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:54:50.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:50 smithi117 bash[21480]: cluster 2024-03-04T15:54:47.674638+0000 mgr.y (mgr.24428) 201 : cluster 0 pgmap v159: 624 pgs: 102 creating+peering, 90 unknown, 432 active+clean; 145 MiB data, 770 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-04T15:54:50.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:50 smithi117 bash[21480]: cluster 2024-03-04T15:54:48.113691+0000 mon.a (mon.0) 1154 : cluster 3 Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:50.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:50 smithi117 bash[21480]: audit 2024-03-04T15:54:48.116164+0000 mon.a (mon.0) 1155 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:50.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:50 smithi117 bash[21480]: audit 2024-03-04T15:54:48.120982+0000 mon.a (mon.0) 1156 : audit 1 from='client.? 172.21.15.117:0/3985467235' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi117-63915-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:50.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:50 smithi117 bash[21480]: cluster 2024-03-04T15:54:48.128371+0000 mon.a (mon.0) 1157 : cluster 0 osdmap e105: 8 total, 8 up, 8 in 2024-03-04T15:54:50.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:50 smithi117 bash[21480]: audit 2024-03-04T15:54:48.211031+0000 mon.a (mon.0) 1158 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:54:50.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:50 smithi117 bash[28843]: cluster 2024-03-04T15:54:45.673835+0000 mgr.y (mgr.24428) 199 : cluster 0 pgmap v157: 584 pgs: 192 unknown, 1 creating+peering, 391 active+clean; 145 MiB data, 767 MiB used, 714 GiB / 715 GiB avail 2024-03-04T15:54:50.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:50 smithi117 bash[28843]: audit 2024-03-04T15:54:46.494783+0000 mgr.y (mgr.24428) 200 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:54:50.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:50 smithi117 bash[28843]: cluster 2024-03-04T15:54:47.674638+0000 mgr.y (mgr.24428) 201 : cluster 0 pgmap v159: 624 pgs: 102 creating+peering, 90 unknown, 432 active+clean; 145 MiB data, 770 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-04T15:54:50.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:50 smithi117 bash[28843]: cluster 2024-03-04T15:54:48.113691+0000 mon.a (mon.0) 1154 : cluster 3 Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:50.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:50 smithi117 bash[28843]: audit 2024-03-04T15:54:48.116164+0000 mon.a (mon.0) 1155 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:50.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:50 smithi117 bash[28843]: audit 2024-03-04T15:54:48.120982+0000 mon.a (mon.0) 1156 : audit 1 from='client.? 172.21.15.117:0/3985467235' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi117-63915-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:50.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:50 smithi117 bash[28843]: cluster 2024-03-04T15:54:48.128371+0000 mon.a (mon.0) 1157 : cluster 0 osdmap e105: 8 total, 8 up, 8 in 2024-03-04T15:54:50.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:50 smithi117 bash[28843]: audit 2024-03-04T15:54:48.211031+0000 mon.a (mon.0) 1158 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:54:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:50 smithi151 bash[23756]: cluster 2024-03-04T15:54:45.673835+0000 mgr.y (mgr.24428) 199 : cluster 0 pgmap v157: 584 pgs: 192 unknown, 1 creating+peering, 391 active+clean; 145 MiB data, 767 MiB used, 714 GiB / 715 GiB avail 2024-03-04T15:54:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:50 smithi151 bash[23756]: audit 2024-03-04T15:54:46.494783+0000 mgr.y (mgr.24428) 200 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:54:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:50 smithi151 bash[23756]: cluster 2024-03-04T15:54:47.674638+0000 mgr.y (mgr.24428) 201 : cluster 0 pgmap v159: 624 pgs: 102 creating+peering, 90 unknown, 432 active+clean; 145 MiB data, 770 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 4 op/s 2024-03-04T15:54:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:50 smithi151 bash[23756]: cluster 2024-03-04T15:54:48.113691+0000 mon.a (mon.0) 1154 : cluster 3 Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:50 smithi151 bash[23756]: audit 2024-03-04T15:54:48.116164+0000 mon.a (mon.0) 1155 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:50 smithi151 bash[23756]: audit 2024-03-04T15:54:48.120982+0000 mon.a (mon.0) 1156 : audit 1 from='client.? 172.21.15.117:0/3985467235' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi117-63915-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:50 smithi151 bash[23756]: cluster 2024-03-04T15:54:48.128371+0000 mon.a (mon.0) 1157 : cluster 0 osdmap e105: 8 total, 8 up, 8 in 2024-03-04T15:54:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:50 smithi151 bash[23756]: audit 2024-03-04T15:54:48.211031+0000 mon.a (mon.0) 1158 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:54:51.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[21480]: audit 2024-03-04T15:54:49.323071+0000 mon.a (mon.0) 1159 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:51.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[21480]: audit 2024-03-04T15:54:49.347111+0000 mon.a (mon.0) 1160 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-7", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:54:51.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[21480]: audit 2024-03-04T15:54:49.365404+0000 mon.b (mon.1) 59 : audit 1 from='client.? 172.21.15.117:0/3174965344' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:51.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[21480]: audit 2024-03-04T15:54:49.375521+0000 mon.c (mon.2) 180 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-7"}]: dispatch 2024-03-04T15:54:51.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[21480]: cluster 2024-03-04T15:54:49.675765+0000 mgr.y (mgr.24428) 202 : cluster 0 pgmap v162: 656 pgs: 76 creating+peering, 218 unknown, 362 active+clean; 145 MiB data, 770 MiB used, 714 GiB / 715 GiB avail; 956 B/s rd, 1.4 KiB/s wr, 3 op/s 2024-03-04T15:54:51.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[21480]: cluster 2024-03-04T15:54:50.536030+0000 mon.a (mon.0) 1161 : cluster 0 osdmap e106: 8 total, 8 up, 8 in 2024-03-04T15:54:51.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[21480]: audit 2024-03-04T15:54:50.564022+0000 mon.a (mon.0) 1162 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:51.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[21480]: audit 2024-03-04T15:54:50.564304+0000 mon.a (mon.0) 1163 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-7"}]: dispatch 2024-03-04T15:54:51.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[21480]: audit 2024-03-04T15:54:50.564714+0000 mon.a (mon.0) 1164 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:51.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[21480]: cluster 2024-03-04T15:54:50.577232+0000 mon.a (mon.0) 1165 : cluster 0 osdmap e107: 8 total, 8 up, 8 in 2024-03-04T15:54:51.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[21480]: audit 2024-03-04T15:54:50.582154+0000 mon.c (mon.2) 181 : audit 1 from='client.? 172.21.15.117:0/2694688658' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi117-63923-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:51.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[21480]: audit 2024-03-04T15:54:50.595261+0000 mon.b (mon.1) 60 : audit 1 from='client.? 172.21.15.117:0/4068152467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi117-64156-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:51.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[21480]: audit 2024-03-04T15:54:50.672502+0000 mon.a (mon.0) 1166 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi117-63923-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:51.821 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[21480]: audit 2024-03-04T15:54:50.686695+0000 mon.a (mon.0) 1167 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi117-64156-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:51.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[28843]: audit 2024-03-04T15:54:49.323071+0000 mon.a (mon.0) 1159 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:51.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[28843]: audit 2024-03-04T15:54:49.347111+0000 mon.a (mon.0) 1160 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-7", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:54:51.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[28843]: audit 2024-03-04T15:54:49.365404+0000 mon.b (mon.1) 59 : audit 1 from='client.? 172.21.15.117:0/3174965344' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:51.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[28843]: audit 2024-03-04T15:54:49.375521+0000 mon.c (mon.2) 180 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-7"}]: dispatch 2024-03-04T15:54:51.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[28843]: cluster 2024-03-04T15:54:49.675765+0000 mgr.y (mgr.24428) 202 : cluster 0 pgmap v162: 656 pgs: 76 creating+peering, 218 unknown, 362 active+clean; 145 MiB data, 770 MiB used, 714 GiB / 715 GiB avail; 956 B/s rd, 1.4 KiB/s wr, 3 op/s 2024-03-04T15:54:51.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[28843]: cluster 2024-03-04T15:54:50.536030+0000 mon.a (mon.0) 1161 : cluster 0 osdmap e106: 8 total, 8 up, 8 in 2024-03-04T15:54:51.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[28843]: audit 2024-03-04T15:54:50.564022+0000 mon.a (mon.0) 1162 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:51.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[28843]: audit 2024-03-04T15:54:50.564304+0000 mon.a (mon.0) 1163 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-7"}]: dispatch 2024-03-04T15:54:51.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[28843]: audit 2024-03-04T15:54:50.564714+0000 mon.a (mon.0) 1164 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:51.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[28843]: cluster 2024-03-04T15:54:50.577232+0000 mon.a (mon.0) 1165 : cluster 0 osdmap e107: 8 total, 8 up, 8 in 2024-03-04T15:54:51.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[28843]: audit 2024-03-04T15:54:50.582154+0000 mon.c (mon.2) 181 : audit 1 from='client.? 172.21.15.117:0/2694688658' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi117-63923-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:51.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[28843]: audit 2024-03-04T15:54:50.595261+0000 mon.b (mon.1) 60 : audit 1 from='client.? 172.21.15.117:0/4068152467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi117-64156-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:51.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[28843]: audit 2024-03-04T15:54:50.672502+0000 mon.a (mon.0) 1166 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi117-63923-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:51.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:51 smithi117 bash[28843]: audit 2024-03-04T15:54:50.686695+0000 mon.a (mon.0) 1167 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi117-64156-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:51.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:51 smithi151 bash[23756]: audit 2024-03-04T15:54:49.323071+0000 mon.a (mon.0) 1159 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:51.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:51 smithi151 bash[23756]: audit 2024-03-04T15:54:49.347111+0000 mon.a (mon.0) 1160 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-7", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:54:51.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:51 smithi151 bash[23756]: audit 2024-03-04T15:54:49.365404+0000 mon.b (mon.1) 59 : audit 1 from='client.? 172.21.15.117:0/3174965344' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:51.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:51 smithi151 bash[23756]: audit 2024-03-04T15:54:49.375521+0000 mon.c (mon.2) 180 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-7"}]: dispatch 2024-03-04T15:54:51.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:51 smithi151 bash[23756]: cluster 2024-03-04T15:54:49.675765+0000 mgr.y (mgr.24428) 202 : cluster 0 pgmap v162: 656 pgs: 76 creating+peering, 218 unknown, 362 active+clean; 145 MiB data, 770 MiB used, 714 GiB / 715 GiB avail; 956 B/s rd, 1.4 KiB/s wr, 3 op/s 2024-03-04T15:54:51.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:51 smithi151 bash[23756]: cluster 2024-03-04T15:54:50.536030+0000 mon.a (mon.0) 1161 : cluster 0 osdmap e106: 8 total, 8 up, 8 in 2024-03-04T15:54:51.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:51 smithi151 bash[23756]: audit 2024-03-04T15:54:50.564022+0000 mon.a (mon.0) 1162 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:51.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:51 smithi151 bash[23756]: audit 2024-03-04T15:54:50.564304+0000 mon.a (mon.0) 1163 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-7"}]: dispatch 2024-03-04T15:54:51.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:51 smithi151 bash[23756]: audit 2024-03-04T15:54:50.564714+0000 mon.a (mon.0) 1164 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:51.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:51 smithi151 bash[23756]: cluster 2024-03-04T15:54:50.577232+0000 mon.a (mon.0) 1165 : cluster 0 osdmap e107: 8 total, 8 up, 8 in 2024-03-04T15:54:51.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:51 smithi151 bash[23756]: audit 2024-03-04T15:54:50.582154+0000 mon.c (mon.2) 181 : audit 1 from='client.? 172.21.15.117:0/2694688658' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi117-63923-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:51.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:51 smithi151 bash[23756]: audit 2024-03-04T15:54:50.595261+0000 mon.b (mon.1) 60 : audit 1 from='client.? 172.21.15.117:0/4068152467' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi117-64156-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:51.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:51 smithi151 bash[23756]: audit 2024-03-04T15:54:50.672502+0000 mon.a (mon.0) 1166 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi117-63923-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:51.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:51 smithi151 bash[23756]: audit 2024-03-04T15:54:50.686695+0000 mon.a (mon.0) 1167 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi117-64156-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:53.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:52 smithi151 bash[23756]: audit 2024-03-04T15:54:51.572540+0000 mon.a (mon.0) 1168 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:53.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:52 smithi151 bash[23756]: audit 2024-03-04T15:54:51.581923+0000 mon.a (mon.0) 1169 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:53.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:52 smithi151 bash[23756]: audit 2024-03-04T15:54:51.582012+0000 mon.a (mon.0) 1170 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-7"}]': finished 2024-03-04T15:54:53.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:52 smithi151 bash[23756]: audit 2024-03-04T15:54:51.582072+0000 mon.a (mon.0) 1171 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi117-63923-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:53.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:52 smithi151 bash[23756]: audit 2024-03-04T15:54:51.582134+0000 mon.a (mon.0) 1172 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi117-64156-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:53.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:52 smithi151 bash[23756]: cluster 2024-03-04T15:54:51.586084+0000 mon.a (mon.0) 1173 : cluster 0 osdmap e108: 8 total, 8 up, 8 in 2024-03-04T15:54:53.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:52 smithi151 bash[23756]: audit 2024-03-04T15:54:51.866991+0000 mon.c (mon.2) 182 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:54:53.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:52 smithi117 bash[28843]: audit 2024-03-04T15:54:51.572540+0000 mon.a (mon.0) 1168 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:53.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:52 smithi117 bash[28843]: audit 2024-03-04T15:54:51.581923+0000 mon.a (mon.0) 1169 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:53.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:52 smithi117 bash[28843]: audit 2024-03-04T15:54:51.582012+0000 mon.a (mon.0) 1170 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-7"}]': finished 2024-03-04T15:54:53.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:52 smithi117 bash[28843]: audit 2024-03-04T15:54:51.582072+0000 mon.a (mon.0) 1171 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi117-63923-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:53.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:52 smithi117 bash[28843]: audit 2024-03-04T15:54:51.582134+0000 mon.a (mon.0) 1172 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi117-64156-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:53.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:52 smithi117 bash[28843]: cluster 2024-03-04T15:54:51.586084+0000 mon.a (mon.0) 1173 : cluster 0 osdmap e108: 8 total, 8 up, 8 in 2024-03-04T15:54:53.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:52 smithi117 bash[28843]: audit 2024-03-04T15:54:51.866991+0000 mon.c (mon.2) 182 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:54:53.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:52 smithi117 bash[21480]: audit 2024-03-04T15:54:51.572540+0000 mon.a (mon.0) 1168 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:53.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:52 smithi117 bash[21480]: audit 2024-03-04T15:54:51.581923+0000 mon.a (mon.0) 1169 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:53.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:52 smithi117 bash[21480]: audit 2024-03-04T15:54:51.582012+0000 mon.a (mon.0) 1170 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-7"}]': finished 2024-03-04T15:54:53.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:52 smithi117 bash[21480]: audit 2024-03-04T15:54:51.582072+0000 mon.a (mon.0) 1171 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi117-63923-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:53.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:52 smithi117 bash[21480]: audit 2024-03-04T15:54:51.582134+0000 mon.a (mon.0) 1172 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi117-64156-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:53.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:52 smithi117 bash[21480]: cluster 2024-03-04T15:54:51.586084+0000 mon.a (mon.0) 1173 : cluster 0 osdmap e108: 8 total, 8 up, 8 in 2024-03-04T15:54:53.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:52 smithi117 bash[21480]: audit 2024-03-04T15:54:51.866991+0000 mon.c (mon.2) 182 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:54:55.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:55 smithi151 bash[23756]: audit 2024-03-04T15:54:52.925262+0000 mon.a (mon.0) 1174 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:55.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:55 smithi151 bash[23756]: audit 2024-03-04T15:54:52.928051+0000 mon.a (mon.0) 1175 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:54:55.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:55 smithi117 bash[21480]: audit 2024-03-04T15:54:52.925262+0000 mon.a (mon.0) 1174 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:55.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:55 smithi117 bash[21480]: audit 2024-03-04T15:54:52.928051+0000 mon.a (mon.0) 1175 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:54:55.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:55 smithi117 bash[28843]: audit 2024-03-04T15:54:52.925262+0000 mon.a (mon.0) 1174 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:55.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:55 smithi117 bash[28843]: audit 2024-03-04T15:54:52.928051+0000 mon.a (mon.0) 1175 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:54:56.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:54:56 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:54:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:54:56.932 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:54:56 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:54:57.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:56 smithi117 bash[28843]: cluster 2024-03-04T15:54:51.679130+0000 mgr.y (mgr.24428) 203 : cluster 0 pgmap v165: 496 pgs: 96 creating+peering, 400 active+clean; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T15:54:57.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:56 smithi117 bash[28843]: audit 2024-03-04T15:54:52.952163+0000 mon.b (mon.1) 61 : audit 1 from='client.? 172.21.15.117:0/4048072412' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi117-64133-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:57.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:56 smithi117 bash[28843]: cluster 2024-03-04T15:54:53.679861+0000 mgr.y (mgr.24428) 204 : cluster 0 pgmap v167: 464 pgs: 64 unknown, 32 creating+peering, 368 active+clean; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 948 B/s rd, 1 op/s 2024-03-04T15:54:57.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:56 smithi117 bash[28843]: cluster 2024-03-04T15:54:55.392250+0000 mon.a (mon.0) 1176 : cluster 0 osdmap e109: 8 total, 8 up, 8 in 2024-03-04T15:54:57.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:56 smithi117 bash[28843]: cluster 2024-03-04T15:54:55.392926+0000 mon.a (mon.0) 1177 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:57.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:56 smithi117 bash[28843]: audit 2024-03-04T15:54:55.417750+0000 mon.a (mon.0) 1178 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.0"}]: dispatch 2024-03-04T15:54:57.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:56 smithi117 bash[28843]: audit 2024-03-04T15:54:55.417971+0000 mon.a (mon.0) 1179 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:57.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:56 smithi117 bash[28843]: audit 2024-03-04T15:54:55.426193+0000 mon.a (mon.0) 1180 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi117-64133-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:57.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:56 smithi117 bash[28843]: audit 2024-03-04T15:54:55.426413+0000 mon.a (mon.0) 1181 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.1"}]: dispatch 2024-03-04T15:54:57.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:56 smithi117 bash[21480]: cluster 2024-03-04T15:54:51.679130+0000 mgr.y (mgr.24428) 203 : cluster 0 pgmap v165: 496 pgs: 96 creating+peering, 400 active+clean; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T15:54:57.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:56 smithi117 bash[21480]: audit 2024-03-04T15:54:52.952163+0000 mon.b (mon.1) 61 : audit 1 from='client.? 172.21.15.117:0/4048072412' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi117-64133-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:57.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:56 smithi117 bash[21480]: cluster 2024-03-04T15:54:53.679861+0000 mgr.y (mgr.24428) 204 : cluster 0 pgmap v167: 464 pgs: 64 unknown, 32 creating+peering, 368 active+clean; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 948 B/s rd, 1 op/s 2024-03-04T15:54:57.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:56 smithi117 bash[21480]: cluster 2024-03-04T15:54:55.392250+0000 mon.a (mon.0) 1176 : cluster 0 osdmap e109: 8 total, 8 up, 8 in 2024-03-04T15:54:57.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:56 smithi117 bash[21480]: cluster 2024-03-04T15:54:55.392926+0000 mon.a (mon.0) 1177 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:57.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:56 smithi117 bash[21480]: audit 2024-03-04T15:54:55.417750+0000 mon.a (mon.0) 1178 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.0"}]: dispatch 2024-03-04T15:54:57.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:56 smithi117 bash[21480]: audit 2024-03-04T15:54:55.417971+0000 mon.a (mon.0) 1179 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:57.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:56 smithi117 bash[21480]: audit 2024-03-04T15:54:55.426193+0000 mon.a (mon.0) 1180 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi117-64133-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:57.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:56 smithi117 bash[21480]: audit 2024-03-04T15:54:55.426413+0000 mon.a (mon.0) 1181 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.1"}]: dispatch 2024-03-04T15:54:57.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:56 smithi151 bash[23756]: cluster 2024-03-04T15:54:51.679130+0000 mgr.y (mgr.24428) 203 : cluster 0 pgmap v165: 496 pgs: 96 creating+peering, 400 active+clean; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-03-04T15:54:57.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:56 smithi151 bash[23756]: audit 2024-03-04T15:54:52.952163+0000 mon.b (mon.1) 61 : audit 1 from='client.? 172.21.15.117:0/4048072412' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi117-64133-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:57.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:56 smithi151 bash[23756]: cluster 2024-03-04T15:54:53.679861+0000 mgr.y (mgr.24428) 204 : cluster 0 pgmap v167: 464 pgs: 64 unknown, 32 creating+peering, 368 active+clean; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 948 B/s rd, 1 op/s 2024-03-04T15:54:57.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:56 smithi151 bash[23756]: cluster 2024-03-04T15:54:55.392250+0000 mon.a (mon.0) 1176 : cluster 0 osdmap e109: 8 total, 8 up, 8 in 2024-03-04T15:54:57.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:56 smithi151 bash[23756]: cluster 2024-03-04T15:54:55.392926+0000 mon.a (mon.0) 1177 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:54:57.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:56 smithi151 bash[23756]: audit 2024-03-04T15:54:55.417750+0000 mon.a (mon.0) 1178 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.0"}]: dispatch 2024-03-04T15:54:57.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:56 smithi151 bash[23756]: audit 2024-03-04T15:54:55.417971+0000 mon.a (mon.0) 1179 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:57.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:56 smithi151 bash[23756]: audit 2024-03-04T15:54:55.426193+0000 mon.a (mon.0) 1180 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi117-64133-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:57.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:56 smithi151 bash[23756]: audit 2024-03-04T15:54:55.426413+0000 mon.a (mon.0) 1181 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.1"}]: dispatch 2024-03-04T15:54:58.327 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:55.418050+0000 mgr.y (mgr.24428) 205 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.0"}]: dispatch 2024-03-04T15:54:58.327 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:55.430632+0000 mgr.y (mgr.24428) 206 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.1"}]: dispatch 2024-03-04T15:54:58.327 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:55.434881+0000 mon.a (mon.0) 1182 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:54:58.327 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:55.454625+0000 mon.c (mon.2) 183 : audit 1 from='client.? 172.21.15.117:0/346154086' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi117-63915-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.327 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:55.455767+0000 mon.c (mon.2) 184 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-7"}]: dispatch 2024-03-04T15:54:58.327 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: cluster 2024-03-04T15:54:55.547807+0000 osd.1 (osd.1) 9 : cluster 0 37.0 deep-scrub starts 2024-03-04T15:54:58.327 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: cluster 2024-03-04T15:54:55.560080+0000 osd.1 (osd.1) 10 : cluster 0 37.0 deep-scrub ok 2024-03-04T15:54:58.328 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: cluster 2024-03-04T15:54:55.681239+0000 mgr.y (mgr.24428) 207 : cluster 0 pgmap v169: 624 pgs: 224 unknown, 32 creating+peering, 368 active+clean; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 802 B/s rd, 1 op/s 2024-03-04T15:54:58.328 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: cluster 2024-03-04T15:54:56.280843+0000 osd.5 (osd.5) 7 : cluster 0 37.1 deep-scrub starts 2024-03-04T15:54:58.328 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: cluster 2024-03-04T15:54:56.338153+0000 osd.5 (osd.5) 8 : cluster 0 37.1 deep-scrub ok 2024-03-04T15:54:58.328 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:56.502759+0000 mgr.y (mgr.24428) 208 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:54:58.328 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: cluster 2024-03-04T15:54:56.930868+0000 mon.a (mon.0) 1183 : cluster 0 osdmap e110: 8 total, 8 up, 8 in 2024-03-04T15:54:58.328 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:56.939499+0000 mon.a (mon.0) 1184 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.2"}]: dispatch 2024-03-04T15:54:58.328 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:56.939747+0000 mon.a (mon.0) 1185 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:58.328 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:56.939973+0000 mgr.y (mgr.24428) 209 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.2"}]: dispatch 2024-03-04T15:54:58.328 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:56.940063+0000 mon.a (mon.0) 1186 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4"}]: dispatch 2024-03-04T15:54:58.328 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:56.945445+0000 mon.a (mon.0) 1187 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.3"}]: dispatch 2024-03-04T15:54:58.328 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:56.945583+0000 mon.a (mon.0) 1188 : audit 1 from='client.? 172.21.15.117:0/1823875201' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.328 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:56.950435+0000 mgr.y (mgr.24428) 210 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.3"}]: dispatch 2024-03-04T15:54:58.328 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:56.966615+0000 mon.a (mon.0) 1189 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi117-64133-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:58.328 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:56.973002+0000 mon.a (mon.0) 1190 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:54:58.329 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: cluster 2024-03-04T15:54:56.974283+0000 mon.a (mon.0) 1191 : cluster 0 osdmap e111: 8 total, 8 up, 8 in 2024-03-04T15:54:58.329 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:56.980861+0000 mon.a (mon.0) 1192 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.4"}]: dispatch 2024-03-04T15:54:58.329 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:56.986915+0000 mgr.y (mgr.24428) 211 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.4"}]: dispatch 2024-03-04T15:54:58.329 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:56.987687+0000 mon.c (mon.2) 185 : audit 1 from='client.? 172.21.15.117:0/2761101374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi117-64656-3"}]: dispatch 2024-03-04T15:54:58.329 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:56.995652+0000 mon.b (mon.1) 62 : audit 1 from='client.? 172.21.15.117:0/2004835808' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.329 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:56.996115+0000 mon.c (mon.2) 186 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:54:58.329 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:56.996469+0000 mon.c (mon.2) 187 : audit 1 from='client.? 172.21.15.117:0/1771343743' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi117-63923-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.329 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:57.020336+0000 mon.a (mon.0) 1193 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.5"}]: dispatch 2024-03-04T15:54:58.329 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:57.020548+0000 mgr.y (mgr.24428) 212 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.5"}]: dispatch 2024-03-04T15:54:58.329 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:57.023259+0000 mon.a (mon.0) 1194 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi117-63915-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.329 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:57.027185+0000 mon.a (mon.0) 1195 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-7"}]: dispatch 2024-03-04T15:54:58.329 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:57.033218+0000 mon.a (mon.0) 1196 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.6"}]: dispatch 2024-03-04T15:54:58.329 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:57.033421+0000 mon.a (mon.0) 1197 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.329 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:57.033748+0000 mon.a (mon.0) 1198 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi117-64656-3"}]: dispatch 2024-03-04T15:54:58.330 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:57.034094+0000 mon.a (mon.0) 1199 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi117-63923-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.330 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:57.038296+0000 mgr.y (mgr.24428) 213 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.6"}]: dispatch 2024-03-04T15:54:58.330 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:57.042002+0000 mon.a (mon.0) 1200 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.7"}]: dispatch 2024-03-04T15:54:58.330 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:57.042729+0000 mgr.y (mgr.24428) 214 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.7"}]: dispatch 2024-03-04T15:54:58.330 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:57.045586+0000 mon.a (mon.0) 1201 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.8"}]: dispatch 2024-03-04T15:54:58.330 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:57.046107+0000 mgr.y (mgr.24428) 215 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.8"}]: dispatch 2024-03-04T15:54:58.330 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:57.054442+0000 mon.a (mon.0) 1202 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.9"}]: dispatch 2024-03-04T15:54:58.330 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:57.056040+0000 mgr.y (mgr.24428) 216 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.9"}]: dispatch 2024-03-04T15:54:58.330 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: cluster 2024-03-04T15:54:57.233803+0000 osd.5 (osd.5) 9 : cluster 0 37.5 deep-scrub starts 2024-03-04T15:54:58.330 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: cluster 2024-03-04T15:54:57.236221+0000 osd.5 (osd.5) 10 : cluster 0 37.5 deep-scrub ok 2024-03-04T15:54:58.330 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: cluster 2024-03-04T15:54:57.501712+0000 osd.1 (osd.1) 11 : cluster 0 37.8 deep-scrub starts 2024-03-04T15:54:58.330 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: cluster 2024-03-04T15:54:57.506134+0000 osd.1 (osd.1) 12 : cluster 0 37.8 deep-scrub ok 2024-03-04T15:54:58.330 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: cluster 2024-03-04T15:54:57.618829+0000 osd.3 (osd.3) 5 : cluster 0 37.6 deep-scrub starts 2024-03-04T15:54:58.330 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: cluster 2024-03-04T15:54:57.619986+0000 osd.3 (osd.3) 6 : cluster 0 37.6 deep-scrub ok 2024-03-04T15:54:58.330 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: cluster 2024-03-04T15:54:57.807328+0000 osd.6 (osd.6) 5 : cluster 0 37.7 deep-scrub starts 2024-03-04T15:54:58.331 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: cluster 2024-03-04T15:54:57.811147+0000 osd.6 (osd.6) 6 : cluster 0 37.7 deep-scrub ok 2024-03-04T15:54:58.331 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[21480]: audit 2024-03-04T15:54:57.941736+0000 mon.a (mon.0) 1203 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:58.331 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:55.418050+0000 mgr.y (mgr.24428) 205 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.0"}]: dispatch 2024-03-04T15:54:58.331 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:55.430632+0000 mgr.y (mgr.24428) 206 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.1"}]: dispatch 2024-03-04T15:54:58.331 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:55.434881+0000 mon.a (mon.0) 1182 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:54:58.331 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:55.454625+0000 mon.c (mon.2) 183 : audit 1 from='client.? 172.21.15.117:0/346154086' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi117-63915-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.331 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:55.455767+0000 mon.c (mon.2) 184 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-7"}]: dispatch 2024-03-04T15:54:58.331 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: cluster 2024-03-04T15:54:55.547807+0000 osd.1 (osd.1) 9 : cluster 0 37.0 deep-scrub starts 2024-03-04T15:54:58.331 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: cluster 2024-03-04T15:54:55.560080+0000 osd.1 (osd.1) 10 : cluster 0 37.0 deep-scrub ok 2024-03-04T15:54:58.331 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: cluster 2024-03-04T15:54:55.681239+0000 mgr.y (mgr.24428) 207 : cluster 0 pgmap v169: 624 pgs: 224 unknown, 32 creating+peering, 368 active+clean; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 802 B/s rd, 1 op/s 2024-03-04T15:54:58.331 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: cluster 2024-03-04T15:54:56.280843+0000 osd.5 (osd.5) 7 : cluster 0 37.1 deep-scrub starts 2024-03-04T15:54:58.331 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: cluster 2024-03-04T15:54:56.338153+0000 osd.5 (osd.5) 8 : cluster 0 37.1 deep-scrub ok 2024-03-04T15:54:58.332 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:56.502759+0000 mgr.y (mgr.24428) 208 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:54:58.332 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: cluster 2024-03-04T15:54:56.930868+0000 mon.a (mon.0) 1183 : cluster 0 osdmap e110: 8 total, 8 up, 8 in 2024-03-04T15:54:58.332 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:56.939499+0000 mon.a (mon.0) 1184 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.2"}]: dispatch 2024-03-04T15:54:58.332 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:56.939747+0000 mon.a (mon.0) 1185 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:58.332 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:56.939973+0000 mgr.y (mgr.24428) 209 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.2"}]: dispatch 2024-03-04T15:54:58.332 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:56.940063+0000 mon.a (mon.0) 1186 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4"}]: dispatch 2024-03-04T15:54:58.332 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:56.945445+0000 mon.a (mon.0) 1187 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.3"}]: dispatch 2024-03-04T15:54:58.332 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:56.945583+0000 mon.a (mon.0) 1188 : audit 1 from='client.? 172.21.15.117:0/1823875201' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.332 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:56.950435+0000 mgr.y (mgr.24428) 210 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.3"}]: dispatch 2024-03-04T15:54:58.332 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:56.966615+0000 mon.a (mon.0) 1189 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi117-64133-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:58.332 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:56.973002+0000 mon.a (mon.0) 1190 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:54:58.332 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: cluster 2024-03-04T15:54:56.974283+0000 mon.a (mon.0) 1191 : cluster 0 osdmap e111: 8 total, 8 up, 8 in 2024-03-04T15:54:58.332 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:56.980861+0000 mon.a (mon.0) 1192 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.4"}]: dispatch 2024-03-04T15:54:58.332 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:56.986915+0000 mgr.y (mgr.24428) 211 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.4"}]: dispatch 2024-03-04T15:54:58.333 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:56.987687+0000 mon.c (mon.2) 185 : audit 1 from='client.? 172.21.15.117:0/2761101374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi117-64656-3"}]: dispatch 2024-03-04T15:54:58.333 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:56.995652+0000 mon.b (mon.1) 62 : audit 1 from='client.? 172.21.15.117:0/2004835808' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.333 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:56.996115+0000 mon.c (mon.2) 186 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:54:58.333 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:56.996469+0000 mon.c (mon.2) 187 : audit 1 from='client.? 172.21.15.117:0/1771343743' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi117-63923-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.333 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:57.020336+0000 mon.a (mon.0) 1193 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.5"}]: dispatch 2024-03-04T15:54:58.333 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:57.020548+0000 mgr.y (mgr.24428) 212 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.5"}]: dispatch 2024-03-04T15:54:58.333 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:57.023259+0000 mon.a (mon.0) 1194 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi117-63915-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.333 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:57.027185+0000 mon.a (mon.0) 1195 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-7"}]: dispatch 2024-03-04T15:54:58.333 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:57.033218+0000 mon.a (mon.0) 1196 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.6"}]: dispatch 2024-03-04T15:54:58.333 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:57.033421+0000 mon.a (mon.0) 1197 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.333 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:57.033748+0000 mon.a (mon.0) 1198 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi117-64656-3"}]: dispatch 2024-03-04T15:54:58.333 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:57.034094+0000 mon.a (mon.0) 1199 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi117-63923-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.334 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:57.038296+0000 mgr.y (mgr.24428) 213 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.6"}]: dispatch 2024-03-04T15:54:58.334 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:57.042002+0000 mon.a (mon.0) 1200 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.7"}]: dispatch 2024-03-04T15:54:58.334 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:57.042729+0000 mgr.y (mgr.24428) 214 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.7"}]: dispatch 2024-03-04T15:54:58.334 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:57.045586+0000 mon.a (mon.0) 1201 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.8"}]: dispatch 2024-03-04T15:54:58.334 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:57.046107+0000 mgr.y (mgr.24428) 215 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.8"}]: dispatch 2024-03-04T15:54:58.334 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:57.054442+0000 mon.a (mon.0) 1202 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.9"}]: dispatch 2024-03-04T15:54:58.334 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:57.056040+0000 mgr.y (mgr.24428) 216 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.9"}]: dispatch 2024-03-04T15:54:58.334 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: cluster 2024-03-04T15:54:57.233803+0000 osd.5 (osd.5) 9 : cluster 0 37.5 deep-scrub starts 2024-03-04T15:54:58.334 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: cluster 2024-03-04T15:54:57.236221+0000 osd.5 (osd.5) 10 : cluster 0 37.5 deep-scrub ok 2024-03-04T15:54:58.334 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: cluster 2024-03-04T15:54:57.501712+0000 osd.1 (osd.1) 11 : cluster 0 37.8 deep-scrub starts 2024-03-04T15:54:58.334 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: cluster 2024-03-04T15:54:57.506134+0000 osd.1 (osd.1) 12 : cluster 0 37.8 deep-scrub ok 2024-03-04T15:54:58.334 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: cluster 2024-03-04T15:54:57.618829+0000 osd.3 (osd.3) 5 : cluster 0 37.6 deep-scrub starts 2024-03-04T15:54:58.334 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: cluster 2024-03-04T15:54:57.619986+0000 osd.3 (osd.3) 6 : cluster 0 37.6 deep-scrub ok 2024-03-04T15:54:58.334 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: cluster 2024-03-04T15:54:57.807328+0000 osd.6 (osd.6) 5 : cluster 0 37.7 deep-scrub starts 2024-03-04T15:54:58.334 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: cluster 2024-03-04T15:54:57.811147+0000 osd.6 (osd.6) 6 : cluster 0 37.7 deep-scrub ok 2024-03-04T15:54:58.335 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:57 smithi117 bash[28843]: audit 2024-03-04T15:54:57.941736+0000 mon.a (mon.0) 1203 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:58.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:55.418050+0000 mgr.y (mgr.24428) 205 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.0"}]: dispatch 2024-03-04T15:54:58.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:55.430632+0000 mgr.y (mgr.24428) 206 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.1"}]: dispatch 2024-03-04T15:54:58.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:55.434881+0000 mon.a (mon.0) 1182 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:54:58.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:55.454625+0000 mon.c (mon.2) 183 : audit 1 from='client.? 172.21.15.117:0/346154086' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi117-63915-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:55.455767+0000 mon.c (mon.2) 184 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-7"}]: dispatch 2024-03-04T15:54:58.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: cluster 2024-03-04T15:54:55.547807+0000 osd.1 (osd.1) 9 : cluster 0 37.0 deep-scrub starts 2024-03-04T15:54:58.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: cluster 2024-03-04T15:54:55.560080+0000 osd.1 (osd.1) 10 : cluster 0 37.0 deep-scrub ok 2024-03-04T15:54:58.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: cluster 2024-03-04T15:54:55.681239+0000 mgr.y (mgr.24428) 207 : cluster 0 pgmap v169: 624 pgs: 224 unknown, 32 creating+peering, 368 active+clean; 145 MiB data, 738 MiB used, 714 GiB / 715 GiB avail; 802 B/s rd, 1 op/s 2024-03-04T15:54:58.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: cluster 2024-03-04T15:54:56.280843+0000 osd.5 (osd.5) 7 : cluster 0 37.1 deep-scrub starts 2024-03-04T15:54:58.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: cluster 2024-03-04T15:54:56.338153+0000 osd.5 (osd.5) 8 : cluster 0 37.1 deep-scrub ok 2024-03-04T15:54:58.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:56.502759+0000 mgr.y (mgr.24428) 208 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:54:58.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: cluster 2024-03-04T15:54:56.930868+0000 mon.a (mon.0) 1183 : cluster 0 osdmap e110: 8 total, 8 up, 8 in 2024-03-04T15:54:58.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:56.939499+0000 mon.a (mon.0) 1184 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.2"}]: dispatch 2024-03-04T15:54:58.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:56.939747+0000 mon.a (mon.0) 1185 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:58.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:56.939973+0000 mgr.y (mgr.24428) 209 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.2"}]: dispatch 2024-03-04T15:54:58.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:56.940063+0000 mon.a (mon.0) 1186 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4"}]: dispatch 2024-03-04T15:54:58.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:56.945445+0000 mon.a (mon.0) 1187 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.3"}]: dispatch 2024-03-04T15:54:58.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:56.945583+0000 mon.a (mon.0) 1188 : audit 1 from='client.? 172.21.15.117:0/1823875201' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:56.950435+0000 mgr.y (mgr.24428) 210 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.3"}]: dispatch 2024-03-04T15:54:58.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:56.966615+0000 mon.a (mon.0) 1189 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi117-64133-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:58.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:56.973002+0000 mon.a (mon.0) 1190 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:54:58.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: cluster 2024-03-04T15:54:56.974283+0000 mon.a (mon.0) 1191 : cluster 0 osdmap e111: 8 total, 8 up, 8 in 2024-03-04T15:54:58.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:56.980861+0000 mon.a (mon.0) 1192 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.4"}]: dispatch 2024-03-04T15:54:58.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:56.986915+0000 mgr.y (mgr.24428) 211 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.4"}]: dispatch 2024-03-04T15:54:58.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:56.987687+0000 mon.c (mon.2) 185 : audit 1 from='client.? 172.21.15.117:0/2761101374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi117-64656-3"}]: dispatch 2024-03-04T15:54:58.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:56.995652+0000 mon.b (mon.1) 62 : audit 1 from='client.? 172.21.15.117:0/2004835808' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:56.996115+0000 mon.c (mon.2) 186 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:54:58.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:56.996469+0000 mon.c (mon.2) 187 : audit 1 from='client.? 172.21.15.117:0/1771343743' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi117-63923-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:57.020336+0000 mon.a (mon.0) 1193 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.5"}]: dispatch 2024-03-04T15:54:58.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:57.020548+0000 mgr.y (mgr.24428) 212 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.5"}]: dispatch 2024-03-04T15:54:58.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:57.023259+0000 mon.a (mon.0) 1194 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi117-63915-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:57.027185+0000 mon.a (mon.0) 1195 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-7"}]: dispatch 2024-03-04T15:54:58.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:57.033218+0000 mon.a (mon.0) 1196 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.6"}]: dispatch 2024-03-04T15:54:58.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:57.033421+0000 mon.a (mon.0) 1197 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:57.033748+0000 mon.a (mon.0) 1198 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi117-64656-3"}]: dispatch 2024-03-04T15:54:58.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:57.034094+0000 mon.a (mon.0) 1199 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi117-63923-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:54:58.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:57.038296+0000 mgr.y (mgr.24428) 213 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.6"}]: dispatch 2024-03-04T15:54:58.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:57.042002+0000 mon.a (mon.0) 1200 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.7"}]: dispatch 2024-03-04T15:54:58.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:57.042729+0000 mgr.y (mgr.24428) 214 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.7"}]: dispatch 2024-03-04T15:54:58.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:57.045586+0000 mon.a (mon.0) 1201 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.8"}]: dispatch 2024-03-04T15:54:58.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:57.046107+0000 mgr.y (mgr.24428) 215 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.8"}]: dispatch 2024-03-04T15:54:58.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:57.054442+0000 mon.a (mon.0) 1202 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.9"}]: dispatch 2024-03-04T15:54:58.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:57.056040+0000 mgr.y (mgr.24428) 216 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "37.9"}]: dispatch 2024-03-04T15:54:58.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: cluster 2024-03-04T15:54:57.233803+0000 osd.5 (osd.5) 9 : cluster 0 37.5 deep-scrub starts 2024-03-04T15:54:58.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: cluster 2024-03-04T15:54:57.236221+0000 osd.5 (osd.5) 10 : cluster 0 37.5 deep-scrub ok 2024-03-04T15:54:58.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: cluster 2024-03-04T15:54:57.501712+0000 osd.1 (osd.1) 11 : cluster 0 37.8 deep-scrub starts 2024-03-04T15:54:58.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: cluster 2024-03-04T15:54:57.506134+0000 osd.1 (osd.1) 12 : cluster 0 37.8 deep-scrub ok 2024-03-04T15:54:58.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: cluster 2024-03-04T15:54:57.618829+0000 osd.3 (osd.3) 5 : cluster 0 37.6 deep-scrub starts 2024-03-04T15:54:58.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: cluster 2024-03-04T15:54:57.619986+0000 osd.3 (osd.3) 6 : cluster 0 37.6 deep-scrub ok 2024-03-04T15:54:58.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: cluster 2024-03-04T15:54:57.807328+0000 osd.6 (osd.6) 5 : cluster 0 37.7 deep-scrub starts 2024-03-04T15:54:58.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: cluster 2024-03-04T15:54:57.811147+0000 osd.6 (osd.6) 6 : cluster 0 37.7 deep-scrub ok 2024-03-04T15:54:58.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:57 smithi151 bash[23756]: audit 2024-03-04T15:54:57.941736+0000 mon.a (mon.0) 1203 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:54:59.248 INFO:tasks.workunit.client.0.smithi117.stdout: ap ec_list: Running main() from gmock_main.cc 2024-03-04T15:54:59.249 INFO:tasks.workunit.client.0.smithi117.stdout: ec_list: [==========] Running 3 tests from 1 test suite. 2024-03-04T15:54:59.249 INFO:tasks.workunit.client.0.smithi117.stdout: ec_list: [----------] Global test environment set-up. 2024-03-04T15:54:59.249 INFO:tasks.workunit.client.0.smithi117.stdout: ec_list: [----------] 3 tests from NeoradosECList 2024-03-04T15:54:59.249 INFO:tasks.workunit.client.0.smithi117.stdout: ec_list: [ RUN ] NeoradosECList.ListObjects 2024-03-04T15:54:59.249 INFO:tasks.workunit.client.0.smithi117.stdout: ec_list: [ OK ] NeoradosECList.ListObjects (27317 ms) 2024-03-04T15:54:59.249 INFO:tasks.workunit.client.0.smithi117.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsNS 2024-03-04T15:54:59.249 INFO:tasks.workunit.client.0.smithi117.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsNS (15629 ms) 2024-03-04T15:54:59.249 INFO:tasks.workunit.client.0.smithi117.stdout: ec_list: [ RUN ] NeoradosECList.ListObjectsMany 2024-03-04T15:54:59.249 INFO:tasks.workunit.client.0.smithi117.stdout: ec_list: [ OK ] NeoradosECList.ListObjectsMany (15998 ms) 2024-03-04T15:54:59.249 INFO:tasks.workunit.client.0.smithi117.stdout: ec_list: [----------] 3 tests from NeoradosECList (58945 ms total) 2024-03-04T15:54:59.249 INFO:tasks.workunit.client.0.smithi117.stdout: ec_list: 2024-03-04T15:54:59.249 INFO:tasks.workunit.client.0.smithi117.stdout: ec_list: [----------] Global test environment tear-down 2024-03-04T15:54:59.249 INFO:tasks.workunit.client.0.smithi117.stdout: ec_list: [==========] 3 tests from 1 test suite ran. (58945 ms total) 2024-03-04T15:54:59.249 INFO:tasks.workunit.client.0.smithi117.stdout: ec_list: [ PASSED ] 3 tests. 2024-03-04T15:54:59.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[28843]: cluster 2024-03-04T15:54:57.430627+0000 osd.2 (osd.2) 5 : cluster 0 37.3 deep-scrub starts 2024-03-04T15:54:59.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[28843]: cluster 2024-03-04T15:54:57.432253+0000 osd.2 (osd.2) 6 : cluster 0 37.3 deep-scrub ok 2024-03-04T15:54:59.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[28843]: audit 2024-03-04T15:54:57.971464+0000 mon.a (mon.0) 1204 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4"}]': finished 2024-03-04T15:54:59.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[28843]: audit 2024-03-04T15:54:57.971540+0000 mon.a (mon.0) 1205 : audit 1 from='client.? 172.21.15.117:0/1823875201' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:59.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[28843]: audit 2024-03-04T15:54:57.971602+0000 mon.a (mon.0) 1206 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi117-63915-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:59.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[28843]: audit 2024-03-04T15:54:57.971654+0000 mon.a (mon.0) 1207 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-7"}]': finished 2024-03-04T15:54:59.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[28843]: audit 2024-03-04T15:54:57.971701+0000 mon.a (mon.0) 1208 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:59.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[28843]: audit 2024-03-04T15:54:57.971751+0000 mon.a (mon.0) 1209 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi117-64656-3"}]': finished 2024-03-04T15:54:59.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[28843]: audit 2024-03-04T15:54:57.971809+0000 mon.a (mon.0) 1210 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi117-63923-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:59.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[28843]: cluster 2024-03-04T15:54:57.975811+0000 mon.a (mon.0) 1211 : cluster 0 osdmap e112: 8 total, 8 up, 8 in 2024-03-04T15:54:59.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[28843]: audit 2024-03-04T15:54:57.978477+0000 mon.a (mon.0) 1212 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4", "tierpool": "test-rados-api-smithi117-64254-6"}]: dispatch 2024-03-04T15:54:59.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[28843]: audit 2024-03-04T15:54:57.995054+0000 mon.c (mon.2) 188 : audit 1 from='client.? 172.21.15.117:0/2761101374' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi117-64656-3"}]: dispatch 2024-03-04T15:54:59.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[28843]: audit 2024-03-04T15:54:58.017070+0000 mon.a (mon.0) 1213 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi117-64656-3"}]: dispatch 2024-03-04T15:54:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[21480]: cluster 2024-03-04T15:54:57.430627+0000 osd.2 (osd.2) 5 : cluster 0 37.3 deep-scrub starts 2024-03-04T15:54:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[21480]: cluster 2024-03-04T15:54:57.432253+0000 osd.2 (osd.2) 6 : cluster 0 37.3 deep-scrub ok 2024-03-04T15:54:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[21480]: audit 2024-03-04T15:54:57.971464+0000 mon.a (mon.0) 1204 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4"}]': finished 2024-03-04T15:54:59.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[21480]: audit 2024-03-04T15:54:57.971540+0000 mon.a (mon.0) 1205 : audit 1 from='client.? 172.21.15.117:0/1823875201' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:59.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[21480]: audit 2024-03-04T15:54:57.971602+0000 mon.a (mon.0) 1206 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi117-63915-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:59.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[21480]: audit 2024-03-04T15:54:57.971654+0000 mon.a (mon.0) 1207 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-7"}]': finished 2024-03-04T15:54:59.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[21480]: audit 2024-03-04T15:54:57.971701+0000 mon.a (mon.0) 1208 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:59.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[21480]: audit 2024-03-04T15:54:57.971751+0000 mon.a (mon.0) 1209 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi117-64656-3"}]': finished 2024-03-04T15:54:59.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[21480]: audit 2024-03-04T15:54:57.971809+0000 mon.a (mon.0) 1210 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi117-63923-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:59.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[21480]: cluster 2024-03-04T15:54:57.975811+0000 mon.a (mon.0) 1211 : cluster 0 osdmap e112: 8 total, 8 up, 8 in 2024-03-04T15:54:59.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[21480]: audit 2024-03-04T15:54:57.978477+0000 mon.a (mon.0) 1212 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4", "tierpool": "test-rados-api-smithi117-64254-6"}]: dispatch 2024-03-04T15:54:59.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[21480]: audit 2024-03-04T15:54:57.995054+0000 mon.c (mon.2) 188 : audit 1 from='client.? 172.21.15.117:0/2761101374' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi117-64656-3"}]: dispatch 2024-03-04T15:54:59.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:54:59 smithi117 bash[21480]: audit 2024-03-04T15:54:58.017070+0000 mon.a (mon.0) 1213 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi117-64656-3"}]: dispatch 2024-03-04T15:54:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:59 smithi151 bash[23756]: cluster 2024-03-04T15:54:57.430627+0000 osd.2 (osd.2) 5 : cluster 0 37.3 deep-scrub starts 2024-03-04T15:54:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:59 smithi151 bash[23756]: cluster 2024-03-04T15:54:57.432253+0000 osd.2 (osd.2) 6 : cluster 0 37.3 deep-scrub ok 2024-03-04T15:54:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:59 smithi151 bash[23756]: audit 2024-03-04T15:54:57.971464+0000 mon.a (mon.0) 1204 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4"}]': finished 2024-03-04T15:54:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:59 smithi151 bash[23756]: audit 2024-03-04T15:54:57.971540+0000 mon.a (mon.0) 1205 : audit 1 from='client.? 172.21.15.117:0/1823875201' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:59.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:59 smithi151 bash[23756]: audit 2024-03-04T15:54:57.971602+0000 mon.a (mon.0) 1206 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi117-63915-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:59.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:59 smithi151 bash[23756]: audit 2024-03-04T15:54:57.971654+0000 mon.a (mon.0) 1207 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-7"}]': finished 2024-03-04T15:54:59.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:59 smithi151 bash[23756]: audit 2024-03-04T15:54:57.971701+0000 mon.a (mon.0) 1208 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:59.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:59 smithi151 bash[23756]: audit 2024-03-04T15:54:57.971751+0000 mon.a (mon.0) 1209 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ListObjectsManysmithi117-64656-3"}]': finished 2024-03-04T15:54:59.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:59 smithi151 bash[23756]: audit 2024-03-04T15:54:57.971809+0000 mon.a (mon.0) 1210 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi117-63923-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:54:59.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:59 smithi151 bash[23756]: cluster 2024-03-04T15:54:57.975811+0000 mon.a (mon.0) 1211 : cluster 0 osdmap e112: 8 total, 8 up, 8 in 2024-03-04T15:54:59.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:59 smithi151 bash[23756]: audit 2024-03-04T15:54:57.978477+0000 mon.a (mon.0) 1212 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4", "tierpool": "test-rados-api-smithi117-64254-6"}]: dispatch 2024-03-04T15:54:59.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:59 smithi151 bash[23756]: audit 2024-03-04T15:54:57.995054+0000 mon.c (mon.2) 188 : audit 1 from='client.? 172.21.15.117:0/2761101374' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi117-64656-3"}]: dispatch 2024-03-04T15:54:59.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:54:59 smithi151 bash[23756]: audit 2024-03-04T15:54:58.017070+0000 mon.a (mon.0) 1213 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi117-64656-3"}]: dispatch 2024-03-04T15:55:00.575 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[21480]: cluster 2024-03-04T15:54:57.490556+0000 osd.0 (osd.0) 5 : cluster 0 37.2 deep-scrub starts 2024-03-04T15:55:00.575 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[21480]: cluster 2024-03-04T15:54:57.495646+0000 osd.0 (osd.0) 6 : cluster 0 37.2 deep-scrub ok 2024-03-04T15:55:00.575 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[21480]: cluster 2024-03-04T15:54:57.672464+0000 osd.4 (osd.4) 5 : cluster 0 37.4 deep-scrub starts 2024-03-04T15:55:00.576 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[21480]: cluster 2024-03-04T15:54:57.674570+0000 osd.4 (osd.4) 6 : cluster 0 37.4 deep-scrub ok 2024-03-04T15:55:00.576 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[21480]: cluster 2024-03-04T15:54:57.682025+0000 mgr.y (mgr.24428) 217 : cluster 0 pgmap v171: 712 pgs: 176 creating+peering, 1 active+clean+scrubbing+deep, 144 unknown, 391 active+clean; 217 MiB data, 964 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 12 MiB/s wr, 2 op/s 2024-03-04T15:55:00.576 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[21480]: cluster 2024-03-04T15:54:58.532969+0000 osd.1 (osd.1) 13 : cluster 0 37.9 deep-scrub starts 2024-03-04T15:55:00.576 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[21480]: cluster 2024-03-04T15:54:58.593590+0000 osd.1 (osd.1) 14 : cluster 0 37.9 deep-scrub ok 2024-03-04T15:55:00.576 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[21480]: audit 2024-03-04T15:54:59.184629+0000 mon.a (mon.0) 1214 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:00.576 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[21480]: cluster 2024-03-04T15:54:59.188363+0000 mon.a (mon.0) 1215 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:55:00.576 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[21480]: audit 2024-03-04T15:54:59.192803+0000 mon.a (mon.0) 1216 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4", "tierpool": "test-rados-api-smithi117-64254-6"}]': finished 2024-03-04T15:55:00.576 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[21480]: audit 2024-03-04T15:54:59.192888+0000 mon.a (mon.0) 1217 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi117-64656-3"}]': finished 2024-03-04T15:55:00.576 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[21480]: cluster 2024-03-04T15:54:59.199300+0000 mon.a (mon.0) 1218 : cluster 0 osdmap e113: 8 total, 8 up, 8 in 2024-03-04T15:55:00.576 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[21480]: audit 2024-03-04T15:54:59.210612+0000 mon.a (mon.0) 1219 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi117-64254-6", "pool2": "test-rados-api-smithi117-64254-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T15:55:00.576 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[21480]: audit 2024-03-04T15:55:00.189537+0000 mon.a (mon.0) 1220 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:00.576 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[21480]: audit 2024-03-04T15:55:00.198832+0000 mon.a (mon.0) 1221 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi117-64254-6", "pool2": "test-rados-api-smithi117-64254-6", "yes_i_really_really_mean_it": true}]': finished 2024-03-04T15:55:00.577 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[21480]: cluster 2024-03-04T15:55:00.207296+0000 mon.a (mon.0) 1222 : cluster 0 osdmap e114: 8 total, 8 up, 8 in 2024-03-04T15:55:00.577 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[28843]: cluster 2024-03-04T15:54:57.490556+0000 osd.0 (osd.0) 5 : cluster 0 37.2 deep-scrub starts 2024-03-04T15:55:00.577 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[28843]: cluster 2024-03-04T15:54:57.495646+0000 osd.0 (osd.0) 6 : cluster 0 37.2 deep-scrub ok 2024-03-04T15:55:00.577 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[28843]: cluster 2024-03-04T15:54:57.672464+0000 osd.4 (osd.4) 5 : cluster 0 37.4 deep-scrub starts 2024-03-04T15:55:00.577 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[28843]: cluster 2024-03-04T15:54:57.674570+0000 osd.4 (osd.4) 6 : cluster 0 37.4 deep-scrub ok 2024-03-04T15:55:00.577 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[28843]: cluster 2024-03-04T15:54:57.682025+0000 mgr.y (mgr.24428) 217 : cluster 0 pgmap v171: 712 pgs: 176 creating+peering, 1 active+clean+scrubbing+deep, 144 unknown, 391 active+clean; 217 MiB data, 964 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 12 MiB/s wr, 2 op/s 2024-03-04T15:55:00.577 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[28843]: cluster 2024-03-04T15:54:58.532969+0000 osd.1 (osd.1) 13 : cluster 0 37.9 deep-scrub starts 2024-03-04T15:55:00.577 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[28843]: cluster 2024-03-04T15:54:58.593590+0000 osd.1 (osd.1) 14 : cluster 0 37.9 deep-scrub ok 2024-03-04T15:55:00.577 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[28843]: audit 2024-03-04T15:54:59.184629+0000 mon.a (mon.0) 1214 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:00.577 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[28843]: cluster 2024-03-04T15:54:59.188363+0000 mon.a (mon.0) 1215 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:55:00.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[28843]: audit 2024-03-04T15:54:59.192803+0000 mon.a (mon.0) 1216 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4", "tierpool": "test-rados-api-smithi117-64254-6"}]': finished 2024-03-04T15:55:00.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[28843]: audit 2024-03-04T15:54:59.192888+0000 mon.a (mon.0) 1217 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi117-64656-3"}]': finished 2024-03-04T15:55:00.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[28843]: cluster 2024-03-04T15:54:59.199300+0000 mon.a (mon.0) 1218 : cluster 0 osdmap e113: 8 total, 8 up, 8 in 2024-03-04T15:55:00.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[28843]: audit 2024-03-04T15:54:59.210612+0000 mon.a (mon.0) 1219 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi117-64254-6", "pool2": "test-rados-api-smithi117-64254-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T15:55:00.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[28843]: audit 2024-03-04T15:55:00.189537+0000 mon.a (mon.0) 1220 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:00.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[28843]: audit 2024-03-04T15:55:00.198832+0000 mon.a (mon.0) 1221 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi117-64254-6", "pool2": "test-rados-api-smithi117-64254-6", "yes_i_really_really_mean_it": true}]': finished 2024-03-04T15:55:00.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:00 smithi117 bash[28843]: cluster 2024-03-04T15:55:00.207296+0000 mon.a (mon.0) 1222 : cluster 0 osdmap e114: 8 total, 8 up, 8 in 2024-03-04T15:55:00.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:00 smithi151 bash[23756]: cluster 2024-03-04T15:54:57.490556+0000 osd.0 (osd.0) 5 : cluster 0 37.2 deep-scrub starts 2024-03-04T15:55:00.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:00 smithi151 bash[23756]: cluster 2024-03-04T15:54:57.495646+0000 osd.0 (osd.0) 6 : cluster 0 37.2 deep-scrub ok 2024-03-04T15:55:00.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:00 smithi151 bash[23756]: cluster 2024-03-04T15:54:57.672464+0000 osd.4 (osd.4) 5 : cluster 0 37.4 deep-scrub starts 2024-03-04T15:55:00.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:00 smithi151 bash[23756]: cluster 2024-03-04T15:54:57.674570+0000 osd.4 (osd.4) 6 : cluster 0 37.4 deep-scrub ok 2024-03-04T15:55:00.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:00 smithi151 bash[23756]: cluster 2024-03-04T15:54:57.682025+0000 mgr.y (mgr.24428) 217 : cluster 0 pgmap v171: 712 pgs: 176 creating+peering, 1 active+clean+scrubbing+deep, 144 unknown, 391 active+clean; 217 MiB data, 964 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 12 MiB/s wr, 2 op/s 2024-03-04T15:55:00.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:00 smithi151 bash[23756]: cluster 2024-03-04T15:54:58.532969+0000 osd.1 (osd.1) 13 : cluster 0 37.9 deep-scrub starts 2024-03-04T15:55:00.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:00 smithi151 bash[23756]: cluster 2024-03-04T15:54:58.593590+0000 osd.1 (osd.1) 14 : cluster 0 37.9 deep-scrub ok 2024-03-04T15:55:00.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:00 smithi151 bash[23756]: audit 2024-03-04T15:54:59.184629+0000 mon.a (mon.0) 1214 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:00.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:00 smithi151 bash[23756]: cluster 2024-03-04T15:54:59.188363+0000 mon.a (mon.0) 1215 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:55:00.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:00 smithi151 bash[23756]: audit 2024-03-04T15:54:59.192803+0000 mon.a (mon.0) 1216 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi117-64254-4", "tierpool": "test-rados-api-smithi117-64254-6"}]': finished 2024-03-04T15:55:00.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:00 smithi151 bash[23756]: audit 2024-03-04T15:54:59.192888+0000 mon.a (mon.0) 1217 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ListObjectsManysmithi117-64656-3"}]': finished 2024-03-04T15:55:00.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:00 smithi151 bash[23756]: cluster 2024-03-04T15:54:59.199300+0000 mon.a (mon.0) 1218 : cluster 0 osdmap e113: 8 total, 8 up, 8 in 2024-03-04T15:55:00.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:00 smithi151 bash[23756]: audit 2024-03-04T15:54:59.210612+0000 mon.a (mon.0) 1219 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi117-64254-6", "pool2": "test-rados-api-smithi117-64254-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T15:55:00.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:00 smithi151 bash[23756]: audit 2024-03-04T15:55:00.189537+0000 mon.a (mon.0) 1220 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:00.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:00 smithi151 bash[23756]: audit 2024-03-04T15:55:00.198832+0000 mon.a (mon.0) 1221 : audit 1 from='client.? 172.21.15.117:0/1952388674' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "test-rados-api-smithi117-64254-6", "pool2": "test-rados-api-smithi117-64254-6", "yes_i_really_really_mean_it": true}]': finished 2024-03-04T15:55:00.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:00 smithi151 bash[23756]: cluster 2024-03-04T15:55:00.207296+0000 mon.a (mon.0) 1222 : cluster 0 osdmap e114: 8 total, 8 up, 8 in 2024-03-04T15:55:03.620 INFO:tasks.workunit.client.0.smithi117.stdout:i_watch_notify_pp: flushed 2024-03-04T15:55:03.620 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3012 ms) 2024-03-04T15:55:03.620 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-03-04T15:55:03.621 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: trying... 2024-03-04T15:55:03.621 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: handle_notify cookie 94643924903760 notify_id 442381631492 notifier_gid 15291 2024-03-04T15:55:03.621 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: timed out 2024-03-04T15:55:03.621 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: flushing 2024-03-04T15:55:03.621 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: flushed 2024-03-04T15:55:03.621 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3015 ms) 2024-03-04T15:55:03.621 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-03-04T15:55:03.621 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: List watches 2024-03-04T15:55:03.621 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: notify2 2024-03-04T15:55:03.621 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: handle_notify cookie 94643924903760 notify_id 455266533378 notifier_gid 15291 2024-03-04T15:55:03.621 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: notify2 done 2024-03-04T15:55:03.621 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: watch_check 2024-03-04T15:55:03.621 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: unwatch2 2024-03-04T15:55:03.621 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: flushing 2024-03-04T15:55:03.622 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: done 2024-03-04T15:55:03.622 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3017 ms) 2024-03-04T15:55:03.622 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-03-04T15:55:03.622 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: List watches 2024-03-04T15:55:03.622 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: notify2 2024-03-04T15:55:03.622 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: handle_notify cookie 94643924903760 notify_id 463856467974 notifier_gid 15291 2024-03-04T15:55:03.622 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: notify2 done 2024-03-04T15:55:03.622 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: watch_check 2024-03-04T15:55:03.622 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: unwatch2 2024-03-04T15:55:03.622 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: flushing 2024-03-04T15:55:03.622 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: done 2024-03-04T15:55:03.622 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3010 ms) 2024-03-04T15:55:03.622 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (17547 ms total) 2024-03-04T15:55:03.622 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: 2024-03-04T15:55:03.622 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-03-04T15:55:03.623 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (63577 ms total) 2024-03-04T15:55:03.623 INFO:tasks.workunit.client.0.smithi117.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-03-04T15:55:03.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:03 smithi151 bash[23756]: cluster 2024-03-04T15:55:00.962129+0000 mon.a (mon.0) 1223 : cluster 3 Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:04.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:03 smithi117 bash[21480]: cluster 2024-03-04T15:55:00.962129+0000 mon.a (mon.0) 1223 : cluster 3 Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:04.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:03 smithi117 bash[28843]: cluster 2024-03-04T15:55:00.962129+0000 mon.a (mon.0) 1223 : cluster 3 Health check update: 10 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:04.864 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: Running main() from gmock_main.cc 2024-03-04T15:55:04.864 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [==========] Running 7 tests from 1 test suite. 2024-03-04T15:55:04.865 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [----------] Global test environment set-up. 2024-03-04T15:55:04.865 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps 2024-03-04T15:55:04.865 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertExists 2024-03-04T15:55:04.865 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertExists (17964 ms) 2024-03-04T15:55:04.865 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [ RUN ] NeoRadosWriteOps.AssertVersion 2024-03-04T15:55:04.865 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [ OK ] NeoRadosWriteOps.AssertVersion (17437 ms) 2024-03-04T15:55:04.865 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Xattrs 2024-03-04T15:55:04.865 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [ OK ] NeoRadosWriteOps.Xattrs (5113 ms) 2024-03-04T15:55:04.865 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Write 2024-03-04T15:55:04.865 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [ OK ] NeoRadosWriteOps.Write (4603 ms) 2024-03-04T15:55:04.865 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [ RUN ] NeoRadosWriteOps.Exec 2024-03-04T15:55:04.865 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [ OK ] NeoRadosWriteOps.Exec (6211 ms) 2024-03-04T15:55:04.865 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [ RUN ] NeoRadosWriteOps.WriteSame 2024-03-04T15:55:04.866 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [ OK ] NeoRadosWriteOps.WriteSame (6364 ms) 2024-03-04T15:55:04.866 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [ RUN ] NeoRadosWriteOps.CmpExt 2024-03-04T15:55:04.866 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [ OK ] NeoRadosWriteOps.CmpExt (6849 ms) 2024-03-04T15:55:04.866 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [----------] 7 tests from NeoRadosWriteOps (64541 ms total) 2024-03-04T15:55:04.866 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: 2024-03-04T15:55:04.866 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [----------] Global test environment tear-down 2024-03-04T15:55:04.866 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [==========] 7 tests from 1 test suite ran. (64541 ms total) 2024-03-04T15:55:04.866 INFO:tasks.workunit.client.0.smithi117.stdout: write_operations: [ PASSED ] 7 tests. 2024-03-04T15:55:05.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[21480]: cluster 2024-03-04T15:54:59.682855+0000 mgr.y (mgr.24428) 218 : cluster 0 pgmap v174: 744 pgs: 144 creating+peering, 1 active+clean+scrubbing+deep, 208 unknown, 391 active+clean; 217 MiB data, 964 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 17 MiB/s wr, 3 op/s 2024-03-04T15:55:05.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[21480]: audit 2024-03-04T15:55:01.539960+0000 mon.a (mon.0) 1224 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:05.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[21480]: audit 2024-03-04T15:55:01.598756+0000 mon.c (mon.2) 189 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[21480]: audit 2024-03-04T15:55:01.622472+0000 mon.b (mon.1) 63 : audit 1 from='client.? 172.21.15.117:0/2893458733' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[21480]: cluster 2024-03-04T15:55:01.683762+0000 mgr.y (mgr.24428) 219 : cluster 0 pgmap v177: 484 pgs: 64 unknown, 420 active+clean; 217 MiB data, 977 MiB used, 714 GiB / 715 GiB avail 2024-03-04T15:55:05.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[21480]: cluster 2024-03-04T15:55:03.590020+0000 mon.a (mon.0) 1225 : cluster 0 osdmap e115: 8 total, 8 up, 8 in 2024-03-04T15:55:05.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[21480]: audit 2024-03-04T15:55:03.606380+0000 mon.a (mon.0) 1226 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:05.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[21480]: audit 2024-03-04T15:55:03.623904+0000 mon.b (mon.1) 64 : audit 1 from='client.? 172.21.15.117:0/94642053' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi117-63915-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[21480]: cluster 2024-03-04T15:55:03.685763+0000 mgr.y (mgr.24428) 220 : cluster 0 pgmap v179: 612 pgs: 192 unknown, 420 active+clean; 217 MiB data, 977 MiB used, 714 GiB / 715 GiB avail 2024-03-04T15:55:05.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[21480]: cluster 2024-03-04T15:55:04.144060+0000 mon.a (mon.0) 1227 : cluster 0 osdmap e116: 8 total, 8 up, 8 in 2024-03-04T15:55:05.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[21480]: audit 2024-03-04T15:55:04.172189+0000 mon.a (mon.0) 1228 : audit 1 from='client.? 172.21.15.117:0/3289019776' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi117-63923-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[21480]: audit 2024-03-04T15:55:04.172506+0000 mon.a (mon.0) 1229 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[21480]: audit 2024-03-04T15:55:04.173367+0000 mon.a (mon.0) 1230 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi117-63915-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[21480]: audit 2024-03-04T15:55:04.178238+0000 mon.a (mon.0) 1231 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[21480]: audit 2024-03-04T15:55:04.730376+0000 mon.a (mon.0) 1232 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:05.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[28843]: cluster 2024-03-04T15:54:59.682855+0000 mgr.y (mgr.24428) 218 : cluster 0 pgmap v174: 744 pgs: 144 creating+peering, 1 active+clean+scrubbing+deep, 208 unknown, 391 active+clean; 217 MiB data, 964 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 17 MiB/s wr, 3 op/s 2024-03-04T15:55:05.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[28843]: audit 2024-03-04T15:55:01.539960+0000 mon.a (mon.0) 1224 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:05.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[28843]: audit 2024-03-04T15:55:01.598756+0000 mon.c (mon.2) 189 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[28843]: audit 2024-03-04T15:55:01.622472+0000 mon.b (mon.1) 63 : audit 1 from='client.? 172.21.15.117:0/2893458733' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[28843]: cluster 2024-03-04T15:55:01.683762+0000 mgr.y (mgr.24428) 219 : cluster 0 pgmap v177: 484 pgs: 64 unknown, 420 active+clean; 217 MiB data, 977 MiB used, 714 GiB / 715 GiB avail 2024-03-04T15:55:05.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[28843]: cluster 2024-03-04T15:55:03.590020+0000 mon.a (mon.0) 1225 : cluster 0 osdmap e115: 8 total, 8 up, 8 in 2024-03-04T15:55:05.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[28843]: audit 2024-03-04T15:55:03.606380+0000 mon.a (mon.0) 1226 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:05.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[28843]: audit 2024-03-04T15:55:03.623904+0000 mon.b (mon.1) 64 : audit 1 from='client.? 172.21.15.117:0/94642053' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi117-63915-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[28843]: cluster 2024-03-04T15:55:03.685763+0000 mgr.y (mgr.24428) 220 : cluster 0 pgmap v179: 612 pgs: 192 unknown, 420 active+clean; 217 MiB data, 977 MiB used, 714 GiB / 715 GiB avail 2024-03-04T15:55:05.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[28843]: cluster 2024-03-04T15:55:04.144060+0000 mon.a (mon.0) 1227 : cluster 0 osdmap e116: 8 total, 8 up, 8 in 2024-03-04T15:55:05.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[28843]: audit 2024-03-04T15:55:04.172189+0000 mon.a (mon.0) 1228 : audit 1 from='client.? 172.21.15.117:0/3289019776' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi117-63923-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[28843]: audit 2024-03-04T15:55:04.172506+0000 mon.a (mon.0) 1229 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[28843]: audit 2024-03-04T15:55:04.173367+0000 mon.a (mon.0) 1230 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi117-63915-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[28843]: audit 2024-03-04T15:55:04.178238+0000 mon.a (mon.0) 1231 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:04 smithi117 bash[28843]: audit 2024-03-04T15:55:04.730376+0000 mon.a (mon.0) 1232 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:05.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:04 smithi151 bash[23756]: cluster 2024-03-04T15:54:59.682855+0000 mgr.y (mgr.24428) 218 : cluster 0 pgmap v174: 744 pgs: 144 creating+peering, 1 active+clean+scrubbing+deep, 208 unknown, 391 active+clean; 217 MiB data, 964 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 17 MiB/s wr, 3 op/s 2024-03-04T15:55:05.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:04 smithi151 bash[23756]: audit 2024-03-04T15:55:01.539960+0000 mon.a (mon.0) 1224 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:05.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:04 smithi151 bash[23756]: audit 2024-03-04T15:55:01.598756+0000 mon.c (mon.2) 189 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:04 smithi151 bash[23756]: audit 2024-03-04T15:55:01.622472+0000 mon.b (mon.1) 63 : audit 1 from='client.? 172.21.15.117:0/2893458733' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:04 smithi151 bash[23756]: cluster 2024-03-04T15:55:01.683762+0000 mgr.y (mgr.24428) 219 : cluster 0 pgmap v177: 484 pgs: 64 unknown, 420 active+clean; 217 MiB data, 977 MiB used, 714 GiB / 715 GiB avail 2024-03-04T15:55:05.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:04 smithi151 bash[23756]: cluster 2024-03-04T15:55:03.590020+0000 mon.a (mon.0) 1225 : cluster 0 osdmap e115: 8 total, 8 up, 8 in 2024-03-04T15:55:05.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:04 smithi151 bash[23756]: audit 2024-03-04T15:55:03.606380+0000 mon.a (mon.0) 1226 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:05.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:04 smithi151 bash[23756]: audit 2024-03-04T15:55:03.623904+0000 mon.b (mon.1) 64 : audit 1 from='client.? 172.21.15.117:0/94642053' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi117-63915-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:04 smithi151 bash[23756]: cluster 2024-03-04T15:55:03.685763+0000 mgr.y (mgr.24428) 220 : cluster 0 pgmap v179: 612 pgs: 192 unknown, 420 active+clean; 217 MiB data, 977 MiB used, 714 GiB / 715 GiB avail 2024-03-04T15:55:05.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:04 smithi151 bash[23756]: cluster 2024-03-04T15:55:04.144060+0000 mon.a (mon.0) 1227 : cluster 0 osdmap e116: 8 total, 8 up, 8 in 2024-03-04T15:55:05.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:04 smithi151 bash[23756]: audit 2024-03-04T15:55:04.172189+0000 mon.a (mon.0) 1228 : audit 1 from='client.? 172.21.15.117:0/3289019776' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi117-63923-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:04 smithi151 bash[23756]: audit 2024-03-04T15:55:04.172506+0000 mon.a (mon.0) 1229 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:04 smithi151 bash[23756]: audit 2024-03-04T15:55:04.173367+0000 mon.a (mon.0) 1230 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi117-63915-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:04 smithi151 bash[23756]: audit 2024-03-04T15:55:04.178238+0000 mon.a (mon.0) 1231 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:05.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:04 smithi151 bash[23756]: audit 2024-03-04T15:55:04.730376+0000 mon.a (mon.0) 1232 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:06.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:05 smithi117 bash[21480]: audit 2024-03-04T15:55:04.768013+0000 mon.a (mon.0) 1233 : audit 1 from='client.? 172.21.15.117:0/3289019776' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi117-63923-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:06.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:05 smithi117 bash[21480]: audit 2024-03-04T15:55:04.768081+0000 mon.a (mon.0) 1234 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:06.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:05 smithi117 bash[21480]: audit 2024-03-04T15:55:04.768144+0000 mon.a (mon.0) 1235 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi117-63915-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:06.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:05 smithi117 bash[21480]: audit 2024-03-04T15:55:04.768202+0000 mon.a (mon.0) 1236 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:06.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:05 smithi117 bash[21480]: cluster 2024-03-04T15:55:04.773372+0000 mon.a (mon.0) 1237 : cluster 0 osdmap e117: 8 total, 8 up, 8 in 2024-03-04T15:55:06.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:05 smithi117 bash[21480]: audit 2024-03-04T15:55:04.829710+0000 mon.c (mon.2) 190 : audit 1 from='client.? 172.21.15.117:0/3630438032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:06.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:05 smithi117 bash[21480]: audit 2024-03-04T15:55:04.830435+0000 mon.a (mon.0) 1238 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:06.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:05 smithi117 bash[21480]: audit 2024-03-04T15:55:04.935053+0000 mon.c (mon.2) 191 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:55:06.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:05 smithi117 bash[21480]: audit 2024-03-04T15:55:04.937351+0000 mon.a (mon.0) 1239 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:55:06.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:05 smithi117 bash[21480]: audit 2024-03-04T15:55:05.743071+0000 mon.a (mon.0) 1240 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:06.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:05 smithi117 bash[28843]: audit 2024-03-04T15:55:04.768013+0000 mon.a (mon.0) 1233 : audit 1 from='client.? 172.21.15.117:0/3289019776' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi117-63923-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:06.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:05 smithi117 bash[28843]: audit 2024-03-04T15:55:04.768081+0000 mon.a (mon.0) 1234 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:06.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:05 smithi117 bash[28843]: audit 2024-03-04T15:55:04.768144+0000 mon.a (mon.0) 1235 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi117-63915-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:06.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:05 smithi117 bash[28843]: audit 2024-03-04T15:55:04.768202+0000 mon.a (mon.0) 1236 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:06.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:05 smithi117 bash[28843]: cluster 2024-03-04T15:55:04.773372+0000 mon.a (mon.0) 1237 : cluster 0 osdmap e117: 8 total, 8 up, 8 in 2024-03-04T15:55:06.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:05 smithi117 bash[28843]: audit 2024-03-04T15:55:04.829710+0000 mon.c (mon.2) 190 : audit 1 from='client.? 172.21.15.117:0/3630438032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:06.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:05 smithi117 bash[28843]: audit 2024-03-04T15:55:04.830435+0000 mon.a (mon.0) 1238 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:06.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:05 smithi117 bash[28843]: audit 2024-03-04T15:55:04.935053+0000 mon.c (mon.2) 191 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:55:06.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:05 smithi117 bash[28843]: audit 2024-03-04T15:55:04.937351+0000 mon.a (mon.0) 1239 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:55:06.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:05 smithi117 bash[28843]: audit 2024-03-04T15:55:05.743071+0000 mon.a (mon.0) 1240 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:05 smithi151 bash[23756]: audit 2024-03-04T15:55:04.768013+0000 mon.a (mon.0) 1233 : audit 1 from='client.? 172.21.15.117:0/3289019776' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi117-63923-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:05 smithi151 bash[23756]: audit 2024-03-04T15:55:04.768081+0000 mon.a (mon.0) 1234 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64340-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:05 smithi151 bash[23756]: audit 2024-03-04T15:55:04.768144+0000 mon.a (mon.0) 1235 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi117-63915-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:05 smithi151 bash[23756]: audit 2024-03-04T15:55:04.768202+0000 mon.a (mon.0) 1236 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:05 smithi151 bash[23756]: cluster 2024-03-04T15:55:04.773372+0000 mon.a (mon.0) 1237 : cluster 0 osdmap e117: 8 total, 8 up, 8 in 2024-03-04T15:55:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:05 smithi151 bash[23756]: audit 2024-03-04T15:55:04.829710+0000 mon.c (mon.2) 190 : audit 1 from='client.? 172.21.15.117:0/3630438032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:06.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:05 smithi151 bash[23756]: audit 2024-03-04T15:55:04.830435+0000 mon.a (mon.0) 1238 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:06.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:05 smithi151 bash[23756]: audit 2024-03-04T15:55:04.935053+0000 mon.c (mon.2) 191 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:55:06.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:05 smithi151 bash[23756]: audit 2024-03-04T15:55:04.937351+0000 mon.a (mon.0) 1239 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:55:06.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:05 smithi151 bash[23756]: audit 2024-03-04T15:55:05.743071+0000 mon.a (mon.0) 1240 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:06.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:55:06 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:55:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:55:06.932 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:55:06 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:55:07.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:07 smithi151 bash[23756]: audit 2024-03-04T15:55:05.780551+0000 mon.a (mon.0) 1241 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:07.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:07 smithi151 bash[23756]: audit 2024-03-04T15:55:05.780640+0000 mon.a (mon.0) 1242 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-9", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:55:07.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:07 smithi151 bash[23756]: cluster 2024-03-04T15:55:05.787616+0000 mon.a (mon.0) 1243 : cluster 0 osdmap e118: 8 total, 8 up, 8 in 2024-03-04T15:55:07.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:07 smithi151 bash[23756]: audit 2024-03-04T15:55:05.805339+0000 mon.c (mon.2) 192 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-9"}]: dispatch 2024-03-04T15:55:07.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:07 smithi151 bash[23756]: audit 2024-03-04T15:55:05.806079+0000 mon.a (mon.0) 1244 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-9"}]: dispatch 2024-03-04T15:55:07.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:07 smithi151 bash[23756]: cluster 2024-03-04T15:55:06.439886+0000 mon.a (mon.0) 1245 : cluster 3 Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:07.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:07 smithi151 bash[23756]: audit 2024-03-04T15:55:06.454702+0000 mon.a (mon.0) 1246 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-9"}]': finished 2024-03-04T15:55:07.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:07 smithi151 bash[23756]: cluster 2024-03-04T15:55:06.459263+0000 mon.a (mon.0) 1247 : cluster 0 osdmap e119: 8 total, 8 up, 8 in 2024-03-04T15:55:07.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:07 smithi151 bash[23756]: audit 2024-03-04T15:55:06.485637+0000 mon.c (mon.2) 193 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-9", "mode": "writeback"}]: dispatch 2024-03-04T15:55:08.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:07 smithi117 bash[28843]: audit 2024-03-04T15:55:05.780551+0000 mon.a (mon.0) 1241 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:08.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:07 smithi117 bash[28843]: audit 2024-03-04T15:55:05.780640+0000 mon.a (mon.0) 1242 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-9", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:55:08.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:07 smithi117 bash[28843]: cluster 2024-03-04T15:55:05.787616+0000 mon.a (mon.0) 1243 : cluster 0 osdmap e118: 8 total, 8 up, 8 in 2024-03-04T15:55:08.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:07 smithi117 bash[28843]: audit 2024-03-04T15:55:05.805339+0000 mon.c (mon.2) 192 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-9"}]: dispatch 2024-03-04T15:55:08.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:07 smithi117 bash[28843]: audit 2024-03-04T15:55:05.806079+0000 mon.a (mon.0) 1244 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-9"}]: dispatch 2024-03-04T15:55:08.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:07 smithi117 bash[28843]: cluster 2024-03-04T15:55:06.439886+0000 mon.a (mon.0) 1245 : cluster 3 Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:08.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:07 smithi117 bash[28843]: audit 2024-03-04T15:55:06.454702+0000 mon.a (mon.0) 1246 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-9"}]': finished 2024-03-04T15:55:08.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:07 smithi117 bash[28843]: cluster 2024-03-04T15:55:06.459263+0000 mon.a (mon.0) 1247 : cluster 0 osdmap e119: 8 total, 8 up, 8 in 2024-03-04T15:55:08.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:07 smithi117 bash[28843]: audit 2024-03-04T15:55:06.485637+0000 mon.c (mon.2) 193 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-9", "mode": "writeback"}]: dispatch 2024-03-04T15:55:08.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:07 smithi117 bash[21480]: audit 2024-03-04T15:55:05.780551+0000 mon.a (mon.0) 1241 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:08.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:07 smithi117 bash[21480]: audit 2024-03-04T15:55:05.780640+0000 mon.a (mon.0) 1242 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-9", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:55:08.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:07 smithi117 bash[21480]: cluster 2024-03-04T15:55:05.787616+0000 mon.a (mon.0) 1243 : cluster 0 osdmap e118: 8 total, 8 up, 8 in 2024-03-04T15:55:08.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:07 smithi117 bash[21480]: audit 2024-03-04T15:55:05.805339+0000 mon.c (mon.2) 192 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-9"}]: dispatch 2024-03-04T15:55:08.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:07 smithi117 bash[21480]: audit 2024-03-04T15:55:05.806079+0000 mon.a (mon.0) 1244 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-9"}]: dispatch 2024-03-04T15:55:08.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:07 smithi117 bash[21480]: cluster 2024-03-04T15:55:06.439886+0000 mon.a (mon.0) 1245 : cluster 3 Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:08.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:07 smithi117 bash[21480]: audit 2024-03-04T15:55:06.454702+0000 mon.a (mon.0) 1246 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-9"}]': finished 2024-03-04T15:55:08.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:07 smithi117 bash[21480]: cluster 2024-03-04T15:55:06.459263+0000 mon.a (mon.0) 1247 : cluster 0 osdmap e119: 8 total, 8 up, 8 in 2024-03-04T15:55:08.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:07 smithi117 bash[21480]: audit 2024-03-04T15:55:06.485637+0000 mon.c (mon.2) 193 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-9", "mode": "writeback"}]: dispatch 2024-03-04T15:55:09.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:08 smithi117 bash[21480]: cluster 2024-03-04T15:55:05.686544+0000 mgr.y (mgr.24428) 221 : cluster 0 pgmap v181: 644 pgs: 256 unknown, 388 active+clean; 217 MiB data, 977 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-03-04T15:55:09.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:08 smithi117 bash[21480]: audit 2024-03-04T15:55:06.511850+0000 mgr.y (mgr.24428) 222 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:55:09.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:08 smithi117 bash[21480]: audit 2024-03-04T15:55:07.055872+0000 mon.a (mon.0) 1248 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:09.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:08 smithi117 bash[21480]: audit 2024-03-04T15:55:07.643254+0000 mon.a (mon.0) 1249 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-9", "mode": "writeback"}]: dispatch 2024-03-04T15:55:09.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:08 smithi117 bash[28843]: cluster 2024-03-04T15:55:05.686544+0000 mgr.y (mgr.24428) 221 : cluster 0 pgmap v181: 644 pgs: 256 unknown, 388 active+clean; 217 MiB data, 977 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-03-04T15:55:09.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:08 smithi117 bash[28843]: audit 2024-03-04T15:55:06.511850+0000 mgr.y (mgr.24428) 222 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:55:09.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:08 smithi117 bash[28843]: audit 2024-03-04T15:55:07.055872+0000 mon.a (mon.0) 1248 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:09.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:08 smithi117 bash[28843]: audit 2024-03-04T15:55:07.643254+0000 mon.a (mon.0) 1249 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-9", "mode": "writeback"}]: dispatch 2024-03-04T15:55:09.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:08 smithi151 bash[23756]: cluster 2024-03-04T15:55:05.686544+0000 mgr.y (mgr.24428) 221 : cluster 0 pgmap v181: 644 pgs: 256 unknown, 388 active+clean; 217 MiB data, 977 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.1 KiB/s wr, 2 op/s 2024-03-04T15:55:09.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:08 smithi151 bash[23756]: audit 2024-03-04T15:55:06.511850+0000 mgr.y (mgr.24428) 222 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:55:09.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:08 smithi151 bash[23756]: audit 2024-03-04T15:55:07.055872+0000 mon.a (mon.0) 1248 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:09.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:08 smithi151 bash[23756]: audit 2024-03-04T15:55:07.643254+0000 mon.a (mon.0) 1249 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-9", "mode": "writeback"}]: dispatch 2024-03-04T15:55:09.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-03-04T15:55:09.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-03-04T15:55:09.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-03-04T15:55:09.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-03-04T15:55:09.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-03-04T15:55:09.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-03-04T15:55:09.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-03-04T15:55:09.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: 2024-03-04T15:55:09.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-03-04T15:55:09.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-03-04T15:55:09.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (20514 ms) 2024-03-04T15:55:09.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-03-04T15:55:09.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (12410 ms) 2024-03-04T15:55:09.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-03-04T15:55:09.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (7815 ms) 2024-03-04T15:55:09.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-03-04T15:55:09.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (6165 ms) 2024-03-04T15:55:09.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-03-04T15:55:09.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (5956 ms) 2024-03-04T15:55:09.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-03-04T15:55:09.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (7270 ms) 2024-03-04T15:55:09.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-03-04T15:55:09.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (9648 ms) 2024-03-04T15:55:09.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (69778 ms total) 2024-03-04T15:55:09.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: 2024-03-04T15:55:09.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-03-04T15:55:09.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (69779 ms total) 2024-03-04T15:55:09.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-03-04T15:55:10.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[21480]: cluster 2024-03-04T15:55:07.687280+0000 mgr.y (mgr.24428) 223 : cluster 0 pgmap v185: 580 pgs: 96 unknown, 484 active+clean; 217 MiB data, 990 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-04T15:55:10.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[21480]: cluster 2024-03-04T15:55:08.710313+0000 mon.a (mon.0) 1250 : cluster 0 osdmap e120: 8 total, 8 up, 8 in 2024-03-04T15:55:10.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[21480]: cluster 2024-03-04T15:55:08.716120+0000 mon.a (mon.0) 1251 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:55:10.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[21480]: audit 2024-03-04T15:55:08.730334+0000 mon.a (mon.0) 1252 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:10.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[21480]: audit 2024-03-04T15:55:08.744140+0000 mon.a (mon.0) 1253 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-9", "mode": "writeback"}]': finished 2024-03-04T15:55:10.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[21480]: cluster 2024-03-04T15:55:08.751072+0000 mon.a (mon.0) 1254 : cluster 0 osdmap e121: 8 total, 8 up, 8 in 2024-03-04T15:55:10.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[21480]: audit 2024-03-04T15:55:08.772464+0000 mon.c (mon.2) 194 : audit 1 from='client.? 172.21.15.117:0/3976863546' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi117-63915-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:10.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[21480]: audit 2024-03-04T15:55:08.773921+0000 mon.c (mon.2) 195 : audit 1 from='client.? 172.21.15.117:0/1205612209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi117-63923-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:10.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[21480]: audit 2024-03-04T15:55:08.983313+0000 mon.c (mon.2) 196 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:55:10.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[21480]: audit 2024-03-04T15:55:09.308018+0000 mon.a (mon.0) 1255 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi117-63915-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:10.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[21480]: audit 2024-03-04T15:55:09.310867+0000 mon.a (mon.0) 1256 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi117-63923-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:10.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[21480]: audit 2024-03-04T15:55:09.328064+0000 mon.a (mon.0) 1257 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:55:10.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[28843]: cluster 2024-03-04T15:55:07.687280+0000 mgr.y (mgr.24428) 223 : cluster 0 pgmap v185: 580 pgs: 96 unknown, 484 active+clean; 217 MiB data, 990 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-04T15:55:10.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[28843]: cluster 2024-03-04T15:55:08.710313+0000 mon.a (mon.0) 1250 : cluster 0 osdmap e120: 8 total, 8 up, 8 in 2024-03-04T15:55:10.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[28843]: cluster 2024-03-04T15:55:08.716120+0000 mon.a (mon.0) 1251 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:55:10.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[28843]: audit 2024-03-04T15:55:08.730334+0000 mon.a (mon.0) 1252 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:10.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[28843]: audit 2024-03-04T15:55:08.744140+0000 mon.a (mon.0) 1253 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-9", "mode": "writeback"}]': finished 2024-03-04T15:55:10.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[28843]: cluster 2024-03-04T15:55:08.751072+0000 mon.a (mon.0) 1254 : cluster 0 osdmap e121: 8 total, 8 up, 8 in 2024-03-04T15:55:10.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[28843]: audit 2024-03-04T15:55:08.772464+0000 mon.c (mon.2) 194 : audit 1 from='client.? 172.21.15.117:0/3976863546' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi117-63915-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:10.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[28843]: audit 2024-03-04T15:55:08.773921+0000 mon.c (mon.2) 195 : audit 1 from='client.? 172.21.15.117:0/1205612209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi117-63923-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:10.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[28843]: audit 2024-03-04T15:55:08.983313+0000 mon.c (mon.2) 196 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:55:10.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[28843]: audit 2024-03-04T15:55:09.308018+0000 mon.a (mon.0) 1255 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi117-63915-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:10.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[28843]: audit 2024-03-04T15:55:09.310867+0000 mon.a (mon.0) 1256 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi117-63923-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:10.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:09 smithi117 bash[28843]: audit 2024-03-04T15:55:09.328064+0000 mon.a (mon.0) 1257 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:55:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:09 smithi151 bash[23756]: cluster 2024-03-04T15:55:07.687280+0000 mgr.y (mgr.24428) 223 : cluster 0 pgmap v185: 580 pgs: 96 unknown, 484 active+clean; 217 MiB data, 990 MiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-04T15:55:10.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:09 smithi151 bash[23756]: cluster 2024-03-04T15:55:08.710313+0000 mon.a (mon.0) 1250 : cluster 0 osdmap e120: 8 total, 8 up, 8 in 2024-03-04T15:55:10.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:09 smithi151 bash[23756]: cluster 2024-03-04T15:55:08.716120+0000 mon.a (mon.0) 1251 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:55:10.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:09 smithi151 bash[23756]: audit 2024-03-04T15:55:08.730334+0000 mon.a (mon.0) 1252 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:10.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:09 smithi151 bash[23756]: audit 2024-03-04T15:55:08.744140+0000 mon.a (mon.0) 1253 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-9", "mode": "writeback"}]': finished 2024-03-04T15:55:10.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:09 smithi151 bash[23756]: cluster 2024-03-04T15:55:08.751072+0000 mon.a (mon.0) 1254 : cluster 0 osdmap e121: 8 total, 8 up, 8 in 2024-03-04T15:55:10.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:09 smithi151 bash[23756]: audit 2024-03-04T15:55:08.772464+0000 mon.c (mon.2) 194 : audit 1 from='client.? 172.21.15.117:0/3976863546' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi117-63915-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:10.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:09 smithi151 bash[23756]: audit 2024-03-04T15:55:08.773921+0000 mon.c (mon.2) 195 : audit 1 from='client.? 172.21.15.117:0/1205612209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi117-63923-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:10.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:09 smithi151 bash[23756]: audit 2024-03-04T15:55:08.983313+0000 mon.c (mon.2) 196 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:55:10.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:09 smithi151 bash[23756]: audit 2024-03-04T15:55:09.308018+0000 mon.a (mon.0) 1255 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi117-63915-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:10.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:09 smithi151 bash[23756]: audit 2024-03-04T15:55:09.310867+0000 mon.a (mon.0) 1256 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi117-63923-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:10.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:09 smithi151 bash[23756]: audit 2024-03-04T15:55:09.328064+0000 mon.a (mon.0) 1257 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:55:10.891 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-03-04T15:55:10.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [----------] Global test environment set-up. 2024-03-04T15:55:10.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-03-04T15:55:10.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-03-04T15:55:10.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-03-04T15:55:10.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-03-04T15:55:10.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: 2024-03-04T15:55:10.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-03-04T15:55:10.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-03-04T15:55:10.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-03-04T15:55:10.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: 2024-03-04T15:54:00.001+0000 7f5186296880 -1 failed for service _ceph-mon._tcp 2024-03-04T15:55:10.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: 2024-03-04T15:54:00.001+0000 7f5186296880 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-03-04T15:55:10.892 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (51 ms) 2024-03-04T15:55:10.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-03-04T15:55:10.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (2014 ms) 2024-03-04T15:55:10.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (2065 ms total) 2024-03-04T15:55:10.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: 2024-03-04T15:55:10.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-03-04T15:55:10.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-03-04T15:55:10.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: started 0x55bbf2b742a0 2024-03-04T15:55:10.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: started 0x55bbf2eb0350 2024-03-04T15:55:10.893 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: started 2 aios 2024-03-04T15:55:10.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: waiting 0x55bbf2b742a0 2024-03-04T15:55:10.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: waiting 0x55bbf2eb0350 2024-03-04T15:55:10.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: done. 2024-03-04T15:55:10.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (23232 ms) 2024-03-04T15:55:10.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (23232 ms total) 2024-03-04T15:55:10.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: 2024-03-04T15:55:10.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-03-04T15:55:10.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-03-04T15:55:10.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-03-04T15:55:10.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-03-04T15:55:10.894 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ OK ] LibRadosMisc.Exec (179 ms) 2024-03-04T15:55:10.895 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-03-04T15:55:10.895 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (9 ms) 2024-03-04T15:55:10.895 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-03-04T15:55:10.895 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (3 ms) 2024-03-04T15:55:10.895 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-03-04T15:55:10.895 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ OK ] LibRadosMisc.Applications (10445 ms) 2024-03-04T15:55:10.895 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-03-04T15:55:10.895 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-03-04T15:55:10.895 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-03-04T15:55:10.895 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-03-04T15:55:10.896 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-03-04T15:55:10.896 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (27313 ms) 2024-03-04T15:55:10.896 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [----------] 8 tests from LibRadosMisc (37949 ms total) 2024-03-04T15:55:10.896 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: 2024-03-04T15:55:10.896 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [----------] Global test environment tear-down 2024-03-04T15:55:10.896 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (70897 ms total) 2024-03-04T15:55:10.896 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc: [ PASSED ] 12 tests. 2024-03-04T15:55:11.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:10 smithi117 bash[28843]: cluster 2024-03-04T15:55:09.688358+0000 mgr.y (mgr.24428) 224 : cluster 0 pgmap v187: 580 pgs: 128 unknown, 452 active+clean; 217 MiB data, 990 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-03-04T15:55:11.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:10 smithi117 bash[28843]: audit 2024-03-04T15:55:09.801640+0000 mon.a (mon.0) 1258 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:11.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:10 smithi117 bash[28843]: audit 2024-03-04T15:55:09.814880+0000 mon.a (mon.0) 1259 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi117-63915-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:11.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:10 smithi117 bash[28843]: audit 2024-03-04T15:55:09.814962+0000 mon.a (mon.0) 1260 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi117-63923-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:11.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:10 smithi117 bash[28843]: audit 2024-03-04T15:55:09.815022+0000 mon.a (mon.0) 1261 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:55:11.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:10 smithi117 bash[28843]: cluster 2024-03-04T15:55:09.824255+0000 mon.a (mon.0) 1262 : cluster 0 osdmap e122: 8 total, 8 up, 8 in 2024-03-04T15:55:11.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:10 smithi117 bash[28843]: audit 2024-03-04T15:55:09.831997+0000 mon.c (mon.2) 197 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-9"}]: dispatch 2024-03-04T15:55:11.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:10 smithi117 bash[28843]: audit 2024-03-04T15:55:09.832667+0000 mon.a (mon.0) 1263 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-9"}]: dispatch 2024-03-04T15:55:11.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:10 smithi117 bash[21480]: cluster 2024-03-04T15:55:09.688358+0000 mgr.y (mgr.24428) 224 : cluster 0 pgmap v187: 580 pgs: 128 unknown, 452 active+clean; 217 MiB data, 990 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-03-04T15:55:11.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:10 smithi117 bash[21480]: audit 2024-03-04T15:55:09.801640+0000 mon.a (mon.0) 1258 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:11.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:10 smithi117 bash[21480]: audit 2024-03-04T15:55:09.814880+0000 mon.a (mon.0) 1259 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi117-63915-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:11.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:10 smithi117 bash[21480]: audit 2024-03-04T15:55:09.814962+0000 mon.a (mon.0) 1260 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi117-63923-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:11.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:10 smithi117 bash[21480]: audit 2024-03-04T15:55:09.815022+0000 mon.a (mon.0) 1261 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:55:11.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:10 smithi117 bash[21480]: cluster 2024-03-04T15:55:09.824255+0000 mon.a (mon.0) 1262 : cluster 0 osdmap e122: 8 total, 8 up, 8 in 2024-03-04T15:55:11.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:10 smithi117 bash[21480]: audit 2024-03-04T15:55:09.831997+0000 mon.c (mon.2) 197 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-9"}]: dispatch 2024-03-04T15:55:11.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:10 smithi117 bash[21480]: audit 2024-03-04T15:55:09.832667+0000 mon.a (mon.0) 1263 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-9"}]: dispatch 2024-03-04T15:55:11.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:10 smithi151 bash[23756]: cluster 2024-03-04T15:55:09.688358+0000 mgr.y (mgr.24428) 224 : cluster 0 pgmap v187: 580 pgs: 128 unknown, 452 active+clean; 217 MiB data, 990 MiB used, 714 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-03-04T15:55:11.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:10 smithi151 bash[23756]: audit 2024-03-04T15:55:09.801640+0000 mon.a (mon.0) 1258 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:11.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:10 smithi151 bash[23756]: audit 2024-03-04T15:55:09.814880+0000 mon.a (mon.0) 1259 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi117-63915-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:11.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:10 smithi151 bash[23756]: audit 2024-03-04T15:55:09.814962+0000 mon.a (mon.0) 1260 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi117-63923-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:11.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:10 smithi151 bash[23756]: audit 2024-03-04T15:55:09.815022+0000 mon.a (mon.0) 1261 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:55:11.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:10 smithi151 bash[23756]: cluster 2024-03-04T15:55:09.824255+0000 mon.a (mon.0) 1262 : cluster 0 osdmap e122: 8 total, 8 up, 8 in 2024-03-04T15:55:11.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:10 smithi151 bash[23756]: audit 2024-03-04T15:55:09.831997+0000 mon.c (mon.2) 197 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-9"}]: dispatch 2024-03-04T15:55:11.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:10 smithi151 bash[23756]: audit 2024-03-04T15:55:09.832667+0000 mon.a (mon.0) 1263 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-9"}]: dispatch 2024-03-04T15:55:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:12 smithi151 bash[23756]: audit 2024-03-04T15:55:10.808599+0000 mon.a (mon.0) 1264 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:12 smithi151 bash[23756]: cluster 2024-03-04T15:55:10.815317+0000 mon.a (mon.0) 1265 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:55:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:12 smithi151 bash[23756]: audit 2024-03-04T15:55:10.819686+0000 mon.a (mon.0) 1266 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-9"}]': finished 2024-03-04T15:55:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:12 smithi151 bash[23756]: cluster 2024-03-04T15:55:10.829113+0000 mon.a (mon.0) 1267 : cluster 0 osdmap e123: 8 total, 8 up, 8 in 2024-03-04T15:55:12.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:12 smithi151 bash[23756]: audit 2024-03-04T15:55:10.836814+0000 mon.b (mon.1) 65 : audit 1 from='client.? 172.21.15.117:0/465758787' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:12.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:12 smithi151 bash[23756]: audit 2024-03-04T15:55:10.888234+0000 mon.a (mon.0) 1268 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:12.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:12 smithi151 bash[23756]: cluster 2024-03-04T15:55:11.748044+0000 mon.a (mon.0) 1269 : cluster 3 Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:12.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:12 smithi151 bash[23756]: audit 2024-03-04T15:55:11.753473+0000 mon.a (mon.0) 1270 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:12.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:12 smithi151 bash[23756]: cluster 2024-03-04T15:55:11.762511+0000 mon.a (mon.0) 1271 : cluster 0 osdmap e124: 8 total, 8 up, 8 in 2024-03-04T15:55:13.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:12 smithi117 bash[28843]: audit 2024-03-04T15:55:10.808599+0000 mon.a (mon.0) 1264 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:13.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:12 smithi117 bash[28843]: cluster 2024-03-04T15:55:10.815317+0000 mon.a (mon.0) 1265 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:55:13.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:12 smithi117 bash[28843]: audit 2024-03-04T15:55:10.819686+0000 mon.a (mon.0) 1266 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-9"}]': finished 2024-03-04T15:55:13.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:12 smithi117 bash[28843]: cluster 2024-03-04T15:55:10.829113+0000 mon.a (mon.0) 1267 : cluster 0 osdmap e123: 8 total, 8 up, 8 in 2024-03-04T15:55:13.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:12 smithi117 bash[28843]: audit 2024-03-04T15:55:10.836814+0000 mon.b (mon.1) 65 : audit 1 from='client.? 172.21.15.117:0/465758787' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:13.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:12 smithi117 bash[28843]: audit 2024-03-04T15:55:10.888234+0000 mon.a (mon.0) 1268 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:13.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:12 smithi117 bash[28843]: cluster 2024-03-04T15:55:11.748044+0000 mon.a (mon.0) 1269 : cluster 3 Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:13.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:12 smithi117 bash[28843]: audit 2024-03-04T15:55:11.753473+0000 mon.a (mon.0) 1270 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:13.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:12 smithi117 bash[28843]: cluster 2024-03-04T15:55:11.762511+0000 mon.a (mon.0) 1271 : cluster 0 osdmap e124: 8 total, 8 up, 8 in 2024-03-04T15:55:13.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:12 smithi117 bash[21480]: audit 2024-03-04T15:55:10.808599+0000 mon.a (mon.0) 1264 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:13.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:12 smithi117 bash[21480]: cluster 2024-03-04T15:55:10.815317+0000 mon.a (mon.0) 1265 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:55:13.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:12 smithi117 bash[21480]: audit 2024-03-04T15:55:10.819686+0000 mon.a (mon.0) 1266 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-9"}]': finished 2024-03-04T15:55:13.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:12 smithi117 bash[21480]: cluster 2024-03-04T15:55:10.829113+0000 mon.a (mon.0) 1267 : cluster 0 osdmap e123: 8 total, 8 up, 8 in 2024-03-04T15:55:13.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:12 smithi117 bash[21480]: audit 2024-03-04T15:55:10.836814+0000 mon.b (mon.1) 65 : audit 1 from='client.? 172.21.15.117:0/465758787' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:13.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:12 smithi117 bash[21480]: audit 2024-03-04T15:55:10.888234+0000 mon.a (mon.0) 1268 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:13.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:12 smithi117 bash[21480]: cluster 2024-03-04T15:55:11.748044+0000 mon.a (mon.0) 1269 : cluster 3 Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:13.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:12 smithi117 bash[21480]: audit 2024-03-04T15:55:11.753473+0000 mon.a (mon.0) 1270 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64101-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:13.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:12 smithi117 bash[21480]: cluster 2024-03-04T15:55:11.762511+0000 mon.a (mon.0) 1271 : cluster 0 osdmap e124: 8 total, 8 up, 8 in 2024-03-04T15:55:14.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[21480]: cluster 2024-03-04T15:55:11.691901+0000 mgr.y (mgr.24428) 225 : cluster 0 pgmap v190: 452 pgs: 32 creating+peering, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 763 B/s rd, 2.0 KiB/s wr, 4 op/s 2024-03-04T15:55:14.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[21480]: audit 2024-03-04T15:55:11.782757+0000 mon.c (mon.2) 198 : audit 1 from='client.? 172.21.15.117:0/3312480361' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi117-63915-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:14.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[21480]: audit 2024-03-04T15:55:12.184055+0000 mon.a (mon.0) 1272 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:14.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[21480]: audit 2024-03-04T15:55:12.639034+0000 mon.a (mon.0) 1273 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi117-63915-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:14.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[21480]: audit 2024-03-04T15:55:13.069601+0000 mon.a (mon.0) 1274 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:55:14.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[21480]: audit 2024-03-04T15:55:13.073298+0000 mon.a (mon.0) 1275 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi117-63915-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:14.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[21480]: cluster 2024-03-04T15:55:13.079299+0000 mon.a (mon.0) 1276 : cluster 0 osdmap e125: 8 total, 8 up, 8 in 2024-03-04T15:55:14.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[21480]: audit 2024-03-04T15:55:13.083340+0000 mon.b (mon.1) 66 : audit 1 from='client.? 172.21.15.117:0/465758787' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64101-10", "tierpool":"test-rados-api-smithi117-64101-10-cache", "force_nonempty":""}]: dispatch 2024-03-04T15:55:14.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[21480]: audit 2024-03-04T15:55:13.090304+0000 mon.c (mon.2) 199 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:55:14.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[21480]: audit 2024-03-04T15:55:13.527383+0000 mon.a (mon.0) 1277 : audit 1 from='client.? 172.21.15.117:0/1809438095' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi117-63923-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:14.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[21480]: audit 2024-03-04T15:55:13.527766+0000 mon.a (mon.0) 1278 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64101-10", "tierpool":"test-rados-api-smithi117-64101-10-cache", "force_nonempty":""}]: dispatch 2024-03-04T15:55:14.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[21480]: audit 2024-03-04T15:55:13.528207+0000 mon.a (mon.0) 1279 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:14.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[28843]: cluster 2024-03-04T15:55:11.691901+0000 mgr.y (mgr.24428) 225 : cluster 0 pgmap v190: 452 pgs: 32 creating+peering, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 763 B/s rd, 2.0 KiB/s wr, 4 op/s 2024-03-04T15:55:14.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[28843]: audit 2024-03-04T15:55:11.782757+0000 mon.c (mon.2) 198 : audit 1 from='client.? 172.21.15.117:0/3312480361' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi117-63915-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:14.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[28843]: audit 2024-03-04T15:55:12.184055+0000 mon.a (mon.0) 1272 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:14.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[28843]: audit 2024-03-04T15:55:12.639034+0000 mon.a (mon.0) 1273 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi117-63915-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:14.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[28843]: audit 2024-03-04T15:55:13.069601+0000 mon.a (mon.0) 1274 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:55:14.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[28843]: audit 2024-03-04T15:55:13.073298+0000 mon.a (mon.0) 1275 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi117-63915-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:14.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[28843]: cluster 2024-03-04T15:55:13.079299+0000 mon.a (mon.0) 1276 : cluster 0 osdmap e125: 8 total, 8 up, 8 in 2024-03-04T15:55:14.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[28843]: audit 2024-03-04T15:55:13.083340+0000 mon.b (mon.1) 66 : audit 1 from='client.? 172.21.15.117:0/465758787' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64101-10", "tierpool":"test-rados-api-smithi117-64101-10-cache", "force_nonempty":""}]: dispatch 2024-03-04T15:55:14.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[28843]: audit 2024-03-04T15:55:13.090304+0000 mon.c (mon.2) 199 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:55:14.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[28843]: audit 2024-03-04T15:55:13.527383+0000 mon.a (mon.0) 1277 : audit 1 from='client.? 172.21.15.117:0/1809438095' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi117-63923-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:14.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[28843]: audit 2024-03-04T15:55:13.527766+0000 mon.a (mon.0) 1278 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64101-10", "tierpool":"test-rados-api-smithi117-64101-10-cache", "force_nonempty":""}]: dispatch 2024-03-04T15:55:14.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:13 smithi117 bash[28843]: audit 2024-03-04T15:55:13.528207+0000 mon.a (mon.0) 1279 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:13 smithi151 bash[23756]: cluster 2024-03-04T15:55:11.691901+0000 mgr.y (mgr.24428) 225 : cluster 0 pgmap v190: 452 pgs: 32 creating+peering, 420 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 763 B/s rd, 2.0 KiB/s wr, 4 op/s 2024-03-04T15:55:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:13 smithi151 bash[23756]: audit 2024-03-04T15:55:11.782757+0000 mon.c (mon.2) 198 : audit 1 from='client.? 172.21.15.117:0/3312480361' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi117-63915-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:13 smithi151 bash[23756]: audit 2024-03-04T15:55:12.184055+0000 mon.a (mon.0) 1272 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:14.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:13 smithi151 bash[23756]: audit 2024-03-04T15:55:12.639034+0000 mon.a (mon.0) 1273 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi117-63915-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:14.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:13 smithi151 bash[23756]: audit 2024-03-04T15:55:13.069601+0000 mon.a (mon.0) 1274 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:55:14.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:13 smithi151 bash[23756]: audit 2024-03-04T15:55:13.073298+0000 mon.a (mon.0) 1275 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi117-63915-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:14.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:13 smithi151 bash[23756]: cluster 2024-03-04T15:55:13.079299+0000 mon.a (mon.0) 1276 : cluster 0 osdmap e125: 8 total, 8 up, 8 in 2024-03-04T15:55:14.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:13 smithi151 bash[23756]: audit 2024-03-04T15:55:13.083340+0000 mon.b (mon.1) 66 : audit 1 from='client.? 172.21.15.117:0/465758787' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64101-10", "tierpool":"test-rados-api-smithi117-64101-10-cache", "force_nonempty":""}]: dispatch 2024-03-04T15:55:14.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:13 smithi151 bash[23756]: audit 2024-03-04T15:55:13.090304+0000 mon.c (mon.2) 199 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:55:14.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:13 smithi151 bash[23756]: audit 2024-03-04T15:55:13.527383+0000 mon.a (mon.0) 1277 : audit 1 from='client.? 172.21.15.117:0/1809438095' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi117-63923-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:14.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:13 smithi151 bash[23756]: audit 2024-03-04T15:55:13.527766+0000 mon.a (mon.0) 1278 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64101-10", "tierpool":"test-rados-api-smithi117-64101-10-cache", "force_nonempty":""}]: dispatch 2024-03-04T15:55:14.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:13 smithi151 bash[23756]: audit 2024-03-04T15:55:13.528207+0000 mon.a (mon.0) 1279 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:15.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:15 smithi151 bash[23756]: cluster 2024-03-04T15:55:13.693030+0000 mgr.y (mgr.24428) 226 : cluster 0 pgmap v193: 548 pgs: 160 unknown, 32 creating+peering, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 2.2 KiB/s wr, 3 op/s 2024-03-04T15:55:15.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:15 smithi151 bash[23756]: audit 2024-03-04T15:55:14.078501+0000 mon.a (mon.0) 1280 : audit 1 from='client.? 172.21.15.117:0/1809438095' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi117-63923-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:15.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:15 smithi151 bash[23756]: audit 2024-03-04T15:55:14.078693+0000 mon.a (mon.0) 1281 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64101-10", "tierpool":"test-rados-api-smithi117-64101-10-cache", "force_nonempty":""}]': finished 2024-03-04T15:55:15.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:15 smithi151 bash[23756]: cluster 2024-03-04T15:55:14.089777+0000 mon.a (mon.0) 1282 : cluster 0 osdmap e126: 8 total, 8 up, 8 in 2024-03-04T15:55:15.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:15 smithi151 bash[23756]: audit 2024-03-04T15:55:14.095689+0000 mon.b (mon.1) 67 : audit 1 from='client.? 172.21.15.117:0/465758787' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64101-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:15.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:15 smithi151 bash[23756]: audit 2024-03-04T15:55:14.096555+0000 mon.c (mon.2) 200 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:15.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:15 smithi151 bash[23756]: audit 2024-03-04T15:55:14.106089+0000 mon.a (mon.0) 1283 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:15.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:15 smithi151 bash[23756]: audit 2024-03-04T15:55:14.107739+0000 mon.a (mon.0) 1284 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64101-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:15.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:15 smithi151 bash[23756]: audit 2024-03-04T15:55:14.596356+0000 mon.a (mon.0) 1285 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:15.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:15 smithi117 bash[21480]: cluster 2024-03-04T15:55:13.693030+0000 mgr.y (mgr.24428) 226 : cluster 0 pgmap v193: 548 pgs: 160 unknown, 32 creating+peering, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 2.2 KiB/s wr, 3 op/s 2024-03-04T15:55:15.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:15 smithi117 bash[21480]: audit 2024-03-04T15:55:14.078501+0000 mon.a (mon.0) 1280 : audit 1 from='client.? 172.21.15.117:0/1809438095' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi117-63923-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:15.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:15 smithi117 bash[21480]: audit 2024-03-04T15:55:14.078693+0000 mon.a (mon.0) 1281 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64101-10", "tierpool":"test-rados-api-smithi117-64101-10-cache", "force_nonempty":""}]': finished 2024-03-04T15:55:15.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:15 smithi117 bash[21480]: cluster 2024-03-04T15:55:14.089777+0000 mon.a (mon.0) 1282 : cluster 0 osdmap e126: 8 total, 8 up, 8 in 2024-03-04T15:55:15.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:15 smithi117 bash[21480]: audit 2024-03-04T15:55:14.095689+0000 mon.b (mon.1) 67 : audit 1 from='client.? 172.21.15.117:0/465758787' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64101-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:15.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:15 smithi117 bash[21480]: audit 2024-03-04T15:55:14.096555+0000 mon.c (mon.2) 200 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:15.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:15 smithi117 bash[21480]: audit 2024-03-04T15:55:14.106089+0000 mon.a (mon.0) 1283 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:15.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:15 smithi117 bash[21480]: audit 2024-03-04T15:55:14.107739+0000 mon.a (mon.0) 1284 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64101-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:15.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:15 smithi117 bash[21480]: audit 2024-03-04T15:55:14.596356+0000 mon.a (mon.0) 1285 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:15 smithi117 bash[28843]: cluster 2024-03-04T15:55:13.693030+0000 mgr.y (mgr.24428) 226 : cluster 0 pgmap v193: 548 pgs: 160 unknown, 32 creating+peering, 356 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 2.2 KiB/s wr, 3 op/s 2024-03-04T15:55:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:15 smithi117 bash[28843]: audit 2024-03-04T15:55:14.078501+0000 mon.a (mon.0) 1280 : audit 1 from='client.? 172.21.15.117:0/1809438095' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi117-63923-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:15 smithi117 bash[28843]: audit 2024-03-04T15:55:14.078693+0000 mon.a (mon.0) 1281 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64101-10", "tierpool":"test-rados-api-smithi117-64101-10-cache", "force_nonempty":""}]': finished 2024-03-04T15:55:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:15 smithi117 bash[28843]: cluster 2024-03-04T15:55:14.089777+0000 mon.a (mon.0) 1282 : cluster 0 osdmap e126: 8 total, 8 up, 8 in 2024-03-04T15:55:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:15 smithi117 bash[28843]: audit 2024-03-04T15:55:14.095689+0000 mon.b (mon.1) 67 : audit 1 from='client.? 172.21.15.117:0/465758787' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64101-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:15 smithi117 bash[28843]: audit 2024-03-04T15:55:14.096555+0000 mon.c (mon.2) 200 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:15 smithi117 bash[28843]: audit 2024-03-04T15:55:14.106089+0000 mon.a (mon.0) 1283 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:15 smithi117 bash[28843]: audit 2024-03-04T15:55:14.107739+0000 mon.a (mon.0) 1284 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64101-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:15 smithi117 bash[28843]: audit 2024-03-04T15:55:14.596356+0000 mon.a (mon.0) 1285 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:16.500 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:55:16 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:55:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:55:16.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:16 smithi117 bash[21480]: audit 2024-03-04T15:55:15.084393+0000 mon.a (mon.0) 1286 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:16.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:16 smithi117 bash[21480]: audit 2024-03-04T15:55:15.084728+0000 mon.a (mon.0) 1287 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64101-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:16.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:16 smithi117 bash[21480]: cluster 2024-03-04T15:55:15.094164+0000 mon.a (mon.0) 1288 : cluster 0 osdmap e127: 8 total, 8 up, 8 in 2024-03-04T15:55:16.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:16 smithi117 bash[21480]: audit 2024-03-04T15:55:15.095791+0000 mon.b (mon.1) 68 : audit 1 from='client.? 172.21.15.117:0/2232577004' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi117-63915-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:16.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:16 smithi117 bash[21480]: audit 2024-03-04T15:55:15.103641+0000 mon.b (mon.1) 69 : audit 1 from='client.? 172.21.15.117:0/465758787' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64101-10", "tierpool":"test-rados-api-smithi117-64101-10-cache"}]: dispatch 2024-03-04T15:55:16.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:16 smithi117 bash[21480]: audit 2024-03-04T15:55:15.121894+0000 mon.a (mon.0) 1289 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi117-63915-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:16.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:16 smithi117 bash[21480]: audit 2024-03-04T15:55:15.151174+0000 mon.a (mon.0) 1290 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64101-10", "tierpool":"test-rados-api-smithi117-64101-10-cache"}]: dispatch 2024-03-04T15:55:16.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:16 smithi117 bash[21480]: audit 2024-03-04T15:55:15.604208+0000 mon.a (mon.0) 1291 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:16.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:16 smithi117 bash[28843]: audit 2024-03-04T15:55:15.084393+0000 mon.a (mon.0) 1286 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:16.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:16 smithi117 bash[28843]: audit 2024-03-04T15:55:15.084728+0000 mon.a (mon.0) 1287 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64101-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:16.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:16 smithi117 bash[28843]: cluster 2024-03-04T15:55:15.094164+0000 mon.a (mon.0) 1288 : cluster 0 osdmap e127: 8 total, 8 up, 8 in 2024-03-04T15:55:16.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:16 smithi117 bash[28843]: audit 2024-03-04T15:55:15.095791+0000 mon.b (mon.1) 68 : audit 1 from='client.? 172.21.15.117:0/2232577004' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi117-63915-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:16.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:16 smithi117 bash[28843]: audit 2024-03-04T15:55:15.103641+0000 mon.b (mon.1) 69 : audit 1 from='client.? 172.21.15.117:0/465758787' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64101-10", "tierpool":"test-rados-api-smithi117-64101-10-cache"}]: dispatch 2024-03-04T15:55:16.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:16 smithi117 bash[28843]: audit 2024-03-04T15:55:15.121894+0000 mon.a (mon.0) 1289 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi117-63915-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:16.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:16 smithi117 bash[28843]: audit 2024-03-04T15:55:15.151174+0000 mon.a (mon.0) 1290 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64101-10", "tierpool":"test-rados-api-smithi117-64101-10-cache"}]: dispatch 2024-03-04T15:55:16.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:16 smithi117 bash[28843]: audit 2024-03-04T15:55:15.604208+0000 mon.a (mon.0) 1291 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:16.932 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:55:16 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:55:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:16 smithi151 bash[23756]: audit 2024-03-04T15:55:15.084393+0000 mon.a (mon.0) 1286 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:16 smithi151 bash[23756]: audit 2024-03-04T15:55:15.084728+0000 mon.a (mon.0) 1287 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64101-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:16 smithi151 bash[23756]: cluster 2024-03-04T15:55:15.094164+0000 mon.a (mon.0) 1288 : cluster 0 osdmap e127: 8 total, 8 up, 8 in 2024-03-04T15:55:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:16 smithi151 bash[23756]: audit 2024-03-04T15:55:15.095791+0000 mon.b (mon.1) 68 : audit 1 from='client.? 172.21.15.117:0/2232577004' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi117-63915-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:16.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:16 smithi151 bash[23756]: audit 2024-03-04T15:55:15.103641+0000 mon.b (mon.1) 69 : audit 1 from='client.? 172.21.15.117:0/465758787' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64101-10", "tierpool":"test-rados-api-smithi117-64101-10-cache"}]: dispatch 2024-03-04T15:55:16.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:16 smithi151 bash[23756]: audit 2024-03-04T15:55:15.121894+0000 mon.a (mon.0) 1289 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi117-63915-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:16.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:16 smithi151 bash[23756]: audit 2024-03-04T15:55:15.151174+0000 mon.a (mon.0) 1290 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64101-10", "tierpool":"test-rados-api-smithi117-64101-10-cache"}]: dispatch 2024-03-04T15:55:16.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:16 smithi151 bash[23756]: audit 2024-03-04T15:55:15.604208+0000 mon.a (mon.0) 1291 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:18.167 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: Running main() from gmock_main.cc 2024-03-04T15:55:18.168 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-03-04T15:55:18.168 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [----------] Global test environment set-up. 2024-03-04T15:55:18.168 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-03-04T15:55:18.168 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-03-04T15:55:18.168 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [ OK ] LibRadosPools.PoolList (18298 ms) 2024-03-04T15:55:18.168 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-03-04T15:55:18.168 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (8411 ms) 2024-03-04T15:55:18.168 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-03-04T15:55:18.168 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (6358 ms) 2024-03-04T15:55:18.168 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-03-04T15:55:18.168 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (6414 ms) 2024-03-04T15:55:18.168 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-03-04T15:55:18.169 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3786 ms) 2024-03-04T15:55:18.169 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-03-04T15:55:18.169 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (8392 ms) 2024-03-04T15:55:18.169 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-03-04T15:55:18.169 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (12008 ms) 2024-03-04T15:55:18.169 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-03-04T15:55:18.169 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (6210 ms) 2024-03-04T15:55:18.169 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-03-04T15:55:18.169 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (8309 ms) 2024-03-04T15:55:18.169 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [----------] 9 tests from LibRadosPools (78186 ms total) 2024-03-04T15:55:18.169 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: 2024-03-04T15:55:18.169 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [----------] Global test environment tear-down 2024-03-04T15:55:18.169 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (78187 ms total) 2024-03-04T15:55:18.169 INFO:tasks.workunit.client.0.smithi117.stdout: api_pool: [ PASSED ] 9 tests. 2024-03-04T15:55:18.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:18 smithi151 bash[23756]: cluster 2024-03-04T15:55:15.694299+0000 mgr.y (mgr.24428) 227 : cluster 0 pgmap v196: 516 pgs: 160 unknown, 32 creating+peering, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-03-04T15:55:18.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:18 smithi151 bash[23756]: audit 2024-03-04T15:55:16.481918+0000 mon.a (mon.0) 1292 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi117-63915-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:18.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:18 smithi151 bash[23756]: audit 2024-03-04T15:55:16.481991+0000 mon.a (mon.0) 1293 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64101-10", "tierpool":"test-rados-api-smithi117-64101-10-cache"}]': finished 2024-03-04T15:55:18.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:18 smithi151 bash[23756]: cluster 2024-03-04T15:55:16.488857+0000 mon.a (mon.0) 1294 : cluster 0 osdmap e128: 8 total, 8 up, 8 in 2024-03-04T15:55:18.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:18 smithi151 bash[23756]: audit 2024-03-04T15:55:16.489824+0000 mon.c (mon.2) 201 : audit 1 from='client.? 172.21.15.117:0/2867690153' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi117-63923-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:18.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:18 smithi151 bash[23756]: audit 2024-03-04T15:55:16.502075+0000 mon.a (mon.0) 1295 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi117-63923-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:18.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:18 smithi151 bash[23756]: audit 2024-03-04T15:55:16.605410+0000 mon.a (mon.0) 1296 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:18.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:18 smithi151 bash[23756]: cluster 2024-03-04T15:55:16.750695+0000 mon.a (mon.0) 1297 : cluster 3 Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:18.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:18 smithi151 bash[23756]: audit 2024-03-04T15:55:16.757248+0000 mon.a (mon.0) 1298 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi117-63923-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:18.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:18 smithi151 bash[23756]: cluster 2024-03-04T15:55:16.769912+0000 mon.a (mon.0) 1299 : cluster 0 osdmap e129: 8 total, 8 up, 8 in 2024-03-04T15:55:18.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:18 smithi151 bash[23756]: audit 2024-03-04T15:55:16.901265+0000 mon.c (mon.2) 202 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:55:18.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[21480]: cluster 2024-03-04T15:55:15.694299+0000 mgr.y (mgr.24428) 227 : cluster 0 pgmap v196: 516 pgs: 160 unknown, 32 creating+peering, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-03-04T15:55:18.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[21480]: audit 2024-03-04T15:55:16.481918+0000 mon.a (mon.0) 1292 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi117-63915-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:18.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[21480]: audit 2024-03-04T15:55:16.481991+0000 mon.a (mon.0) 1293 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64101-10", "tierpool":"test-rados-api-smithi117-64101-10-cache"}]': finished 2024-03-04T15:55:18.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[21480]: cluster 2024-03-04T15:55:16.488857+0000 mon.a (mon.0) 1294 : cluster 0 osdmap e128: 8 total, 8 up, 8 in 2024-03-04T15:55:18.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[21480]: audit 2024-03-04T15:55:16.489824+0000 mon.c (mon.2) 201 : audit 1 from='client.? 172.21.15.117:0/2867690153' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi117-63923-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:18.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[21480]: audit 2024-03-04T15:55:16.502075+0000 mon.a (mon.0) 1295 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi117-63923-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:18.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[21480]: audit 2024-03-04T15:55:16.605410+0000 mon.a (mon.0) 1296 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:18.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[21480]: cluster 2024-03-04T15:55:16.750695+0000 mon.a (mon.0) 1297 : cluster 3 Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:18.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[21480]: audit 2024-03-04T15:55:16.757248+0000 mon.a (mon.0) 1298 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi117-63923-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:18.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[21480]: cluster 2024-03-04T15:55:16.769912+0000 mon.a (mon.0) 1299 : cluster 0 osdmap e129: 8 total, 8 up, 8 in 2024-03-04T15:55:18.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[21480]: audit 2024-03-04T15:55:16.901265+0000 mon.c (mon.2) 202 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:55:18.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[28843]: cluster 2024-03-04T15:55:15.694299+0000 mgr.y (mgr.24428) 227 : cluster 0 pgmap v196: 516 pgs: 160 unknown, 32 creating+peering, 324 active+clean; 217 MiB data, 1.0 GiB used, 714 GiB / 715 GiB avail 2024-03-04T15:55:18.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[28843]: audit 2024-03-04T15:55:16.481918+0000 mon.a (mon.0) 1292 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi117-63915-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:18.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[28843]: audit 2024-03-04T15:55:16.481991+0000 mon.a (mon.0) 1293 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64101-10", "tierpool":"test-rados-api-smithi117-64101-10-cache"}]': finished 2024-03-04T15:55:18.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[28843]: cluster 2024-03-04T15:55:16.488857+0000 mon.a (mon.0) 1294 : cluster 0 osdmap e128: 8 total, 8 up, 8 in 2024-03-04T15:55:18.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[28843]: audit 2024-03-04T15:55:16.489824+0000 mon.c (mon.2) 201 : audit 1 from='client.? 172.21.15.117:0/2867690153' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi117-63923-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:18.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[28843]: audit 2024-03-04T15:55:16.502075+0000 mon.a (mon.0) 1295 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi117-63923-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:18.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[28843]: audit 2024-03-04T15:55:16.605410+0000 mon.a (mon.0) 1296 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:18.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[28843]: cluster 2024-03-04T15:55:16.750695+0000 mon.a (mon.0) 1297 : cluster 3 Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:18.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[28843]: audit 2024-03-04T15:55:16.757248+0000 mon.a (mon.0) 1298 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi117-63923-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:18.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[28843]: cluster 2024-03-04T15:55:16.769912+0000 mon.a (mon.0) 1299 : cluster 0 osdmap e129: 8 total, 8 up, 8 in 2024-03-04T15:55:18.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:18 smithi117 bash[28843]: audit 2024-03-04T15:55:16.901265+0000 mon.c (mon.2) 202 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:55:19.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:19 smithi151 bash[23756]: audit 2024-03-04T15:55:16.522899+0000 mgr.y (mgr.24428) 228 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:55:19.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:19 smithi151 bash[23756]: audit 2024-03-04T15:55:17.695265+0000 mon.a (mon.0) 1300 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:19.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:19 smithi151 bash[23756]: cluster 2024-03-04T15:55:17.695505+0000 mgr.y (mgr.24428) 229 : cluster 0 pgmap v199: 484 pgs: 96 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T15:55:19.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:19 smithi151 bash[23756]: audit 2024-03-04T15:55:17.701961+0000 mon.a (mon.0) 1301 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:55:19.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:19 smithi151 bash[23756]: audit 2024-03-04T15:55:18.151430+0000 mon.a (mon.0) 1302 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-11", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:55:19.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:19 smithi151 bash[23756]: cluster 2024-03-04T15:55:18.155159+0000 mon.a (mon.0) 1303 : cluster 0 osdmap e130: 8 total, 8 up, 8 in 2024-03-04T15:55:19.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:19 smithi151 bash[23756]: audit 2024-03-04T15:55:18.164008+0000 mon.c (mon.2) 203 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-11"}]: dispatch 2024-03-04T15:55:19.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:19 smithi151 bash[23756]: audit 2024-03-04T15:55:18.168783+0000 mon.a (mon.0) 1304 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-11"}]: dispatch 2024-03-04T15:55:19.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:19 smithi151 bash[23756]: audit 2024-03-04T15:55:18.197394+0000 mon.c (mon.2) 204 : audit 1 from='client.? 172.21.15.117:0/4120017435' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi117-63915-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:19.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:19 smithi151 bash[23756]: audit 2024-03-04T15:55:18.198079+0000 mon.a (mon.0) 1305 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi117-63915-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:19.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:19 smithi151 bash[23756]: audit 2024-03-04T15:55:19.041670+0000 mon.a (mon.0) 1306 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:19.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[21480]: audit 2024-03-04T15:55:16.522899+0000 mgr.y (mgr.24428) 228 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:55:19.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[21480]: audit 2024-03-04T15:55:17.695265+0000 mon.a (mon.0) 1300 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:19.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[21480]: cluster 2024-03-04T15:55:17.695505+0000 mgr.y (mgr.24428) 229 : cluster 0 pgmap v199: 484 pgs: 96 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T15:55:19.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[21480]: audit 2024-03-04T15:55:17.701961+0000 mon.a (mon.0) 1301 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:55:19.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[21480]: audit 2024-03-04T15:55:18.151430+0000 mon.a (mon.0) 1302 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-11", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:55:19.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[21480]: cluster 2024-03-04T15:55:18.155159+0000 mon.a (mon.0) 1303 : cluster 0 osdmap e130: 8 total, 8 up, 8 in 2024-03-04T15:55:19.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[21480]: audit 2024-03-04T15:55:18.164008+0000 mon.c (mon.2) 203 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-11"}]: dispatch 2024-03-04T15:55:19.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[21480]: audit 2024-03-04T15:55:18.168783+0000 mon.a (mon.0) 1304 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-11"}]: dispatch 2024-03-04T15:55:19.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[21480]: audit 2024-03-04T15:55:18.197394+0000 mon.c (mon.2) 204 : audit 1 from='client.? 172.21.15.117:0/4120017435' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi117-63915-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:19.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[21480]: audit 2024-03-04T15:55:18.198079+0000 mon.a (mon.0) 1305 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi117-63915-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:19.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[21480]: audit 2024-03-04T15:55:19.041670+0000 mon.a (mon.0) 1306 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:19.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[28843]: audit 2024-03-04T15:55:16.522899+0000 mgr.y (mgr.24428) 228 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:55:19.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[28843]: audit 2024-03-04T15:55:17.695265+0000 mon.a (mon.0) 1300 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:19.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[28843]: cluster 2024-03-04T15:55:17.695505+0000 mgr.y (mgr.24428) 229 : cluster 0 pgmap v199: 484 pgs: 96 unknown, 388 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T15:55:19.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[28843]: audit 2024-03-04T15:55:17.701961+0000 mon.a (mon.0) 1301 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:55:19.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[28843]: audit 2024-03-04T15:55:18.151430+0000 mon.a (mon.0) 1302 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-11", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:55:19.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[28843]: cluster 2024-03-04T15:55:18.155159+0000 mon.a (mon.0) 1303 : cluster 0 osdmap e130: 8 total, 8 up, 8 in 2024-03-04T15:55:19.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[28843]: audit 2024-03-04T15:55:18.164008+0000 mon.c (mon.2) 203 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-11"}]: dispatch 2024-03-04T15:55:19.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[28843]: audit 2024-03-04T15:55:18.168783+0000 mon.a (mon.0) 1304 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-11"}]: dispatch 2024-03-04T15:55:19.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[28843]: audit 2024-03-04T15:55:18.197394+0000 mon.c (mon.2) 204 : audit 1 from='client.? 172.21.15.117:0/4120017435' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi117-63915-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:19.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[28843]: audit 2024-03-04T15:55:18.198079+0000 mon.a (mon.0) 1305 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi117-63915-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:19.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:19 smithi117 bash[28843]: audit 2024-03-04T15:55:19.041670+0000 mon.a (mon.0) 1306 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:20.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:20 smithi151 bash[23756]: audit 2024-03-04T15:55:19.156892+0000 mon.a (mon.0) 1307 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-11"}]': finished 2024-03-04T15:55:20.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:20 smithi151 bash[23756]: audit 2024-03-04T15:55:19.157076+0000 mon.a (mon.0) 1308 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi117-63915-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:20.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:20 smithi151 bash[23756]: cluster 2024-03-04T15:55:19.165488+0000 mon.a (mon.0) 1309 : cluster 0 osdmap e131: 8 total, 8 up, 8 in 2024-03-04T15:55:20.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:20 smithi151 bash[23756]: audit 2024-03-04T15:55:19.167941+0000 mon.c (mon.2) 205 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-11", "mode": "writeback"}]: dispatch 2024-03-04T15:55:20.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:20 smithi151 bash[23756]: audit 2024-03-04T15:55:19.169342+0000 mon.c (mon.2) 206 : audit 1 from='client.? 172.21.15.117:0/2637543287' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi117-63923-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:20.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:20 smithi151 bash[23756]: audit 2024-03-04T15:55:19.192796+0000 mon.a (mon.0) 1310 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-11", "mode": "writeback"}]: dispatch 2024-03-04T15:55:20.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:20 smithi151 bash[23756]: audit 2024-03-04T15:55:19.193162+0000 mon.a (mon.0) 1311 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi117-63923-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:20.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:20 smithi151 bash[23756]: audit 2024-03-04T15:55:19.236501+0000 mon.a (mon.0) 1312 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi117-64133-10"}]: dispatch 2024-03-04T15:55:20.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:20 smithi151 bash[23756]: audit 2024-03-04T15:55:19.241128+0000 mon.a (mon.0) 1313 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi117-64133-10"}]: dispatch 2024-03-04T15:55:20.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:20 smithi151 bash[23756]: audit 2024-03-04T15:55:19.241655+0000 mon.a (mon.0) 1314 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi117-64133-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:20.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:20 smithi151 bash[23756]: audit 2024-03-04T15:55:20.042909+0000 mon.a (mon.0) 1315 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:20.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[21480]: audit 2024-03-04T15:55:19.156892+0000 mon.a (mon.0) 1307 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-11"}]': finished 2024-03-04T15:55:20.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[21480]: audit 2024-03-04T15:55:19.157076+0000 mon.a (mon.0) 1308 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi117-63915-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:20.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[21480]: cluster 2024-03-04T15:55:19.165488+0000 mon.a (mon.0) 1309 : cluster 0 osdmap e131: 8 total, 8 up, 8 in 2024-03-04T15:55:20.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[21480]: audit 2024-03-04T15:55:19.167941+0000 mon.c (mon.2) 205 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-11", "mode": "writeback"}]: dispatch 2024-03-04T15:55:20.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[21480]: audit 2024-03-04T15:55:19.169342+0000 mon.c (mon.2) 206 : audit 1 from='client.? 172.21.15.117:0/2637543287' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi117-63923-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:20.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[21480]: audit 2024-03-04T15:55:19.192796+0000 mon.a (mon.0) 1310 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-11", "mode": "writeback"}]: dispatch 2024-03-04T15:55:20.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[21480]: audit 2024-03-04T15:55:19.193162+0000 mon.a (mon.0) 1311 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi117-63923-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:20.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[21480]: audit 2024-03-04T15:55:19.236501+0000 mon.a (mon.0) 1312 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi117-64133-10"}]: dispatch 2024-03-04T15:55:20.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[21480]: audit 2024-03-04T15:55:19.241128+0000 mon.a (mon.0) 1313 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi117-64133-10"}]: dispatch 2024-03-04T15:55:20.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[21480]: audit 2024-03-04T15:55:19.241655+0000 mon.a (mon.0) 1314 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi117-64133-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:20.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[21480]: audit 2024-03-04T15:55:20.042909+0000 mon.a (mon.0) 1315 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:20.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[28843]: audit 2024-03-04T15:55:19.156892+0000 mon.a (mon.0) 1307 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-11"}]': finished 2024-03-04T15:55:20.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[28843]: audit 2024-03-04T15:55:19.157076+0000 mon.a (mon.0) 1308 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi117-63915-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:20.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[28843]: cluster 2024-03-04T15:55:19.165488+0000 mon.a (mon.0) 1309 : cluster 0 osdmap e131: 8 total, 8 up, 8 in 2024-03-04T15:55:20.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[28843]: audit 2024-03-04T15:55:19.167941+0000 mon.c (mon.2) 205 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-11", "mode": "writeback"}]: dispatch 2024-03-04T15:55:20.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[28843]: audit 2024-03-04T15:55:19.169342+0000 mon.c (mon.2) 206 : audit 1 from='client.? 172.21.15.117:0/2637543287' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi117-63923-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:20.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[28843]: audit 2024-03-04T15:55:19.192796+0000 mon.a (mon.0) 1310 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-11", "mode": "writeback"}]: dispatch 2024-03-04T15:55:20.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[28843]: audit 2024-03-04T15:55:19.193162+0000 mon.a (mon.0) 1311 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi117-63923-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:20.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[28843]: audit 2024-03-04T15:55:19.236501+0000 mon.a (mon.0) 1312 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi117-64133-10"}]: dispatch 2024-03-04T15:55:20.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[28843]: audit 2024-03-04T15:55:19.241128+0000 mon.a (mon.0) 1313 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi117-64133-10"}]: dispatch 2024-03-04T15:55:20.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[28843]: audit 2024-03-04T15:55:19.241655+0000 mon.a (mon.0) 1314 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi117-64133-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:20.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:20 smithi117 bash[28843]: audit 2024-03-04T15:55:20.042909+0000 mon.a (mon.0) 1315 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:22.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:21 smithi117 bash[21480]: cluster 2024-03-04T15:55:19.696764+0000 mgr.y (mgr.24428) 230 : cluster 0 pgmap v202: 516 pgs: 192 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-04T15:55:22.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:21 smithi117 bash[21480]: cluster 2024-03-04T15:55:20.158901+0000 mon.a (mon.0) 1316 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:55:22.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:21 smithi117 bash[21480]: audit 2024-03-04T15:55:20.168470+0000 mon.a (mon.0) 1317 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-11", "mode": "writeback"}]': finished 2024-03-04T15:55:22.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:21 smithi117 bash[21480]: audit 2024-03-04T15:55:20.168671+0000 mon.a (mon.0) 1318 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi117-63923-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:22.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:21 smithi117 bash[21480]: audit 2024-03-04T15:55:20.168836+0000 mon.a (mon.0) 1319 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi117-64133-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:55:22.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:21 smithi117 bash[21480]: cluster 2024-03-04T15:55:20.178760+0000 mon.a (mon.0) 1320 : cluster 0 osdmap e132: 8 total, 8 up, 8 in 2024-03-04T15:55:22.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:21 smithi117 bash[21480]: audit 2024-03-04T15:55:20.180901+0000 mon.a (mon.0) 1321 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi117-64133-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi117-64133-10"}]: dispatch 2024-03-04T15:55:22.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:21 smithi117 bash[21480]: audit 2024-03-04T15:55:21.112220+0000 mon.a (mon.0) 1322 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:22.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:21 smithi117 bash[28843]: cluster 2024-03-04T15:55:19.696764+0000 mgr.y (mgr.24428) 230 : cluster 0 pgmap v202: 516 pgs: 192 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-04T15:55:22.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:21 smithi117 bash[28843]: cluster 2024-03-04T15:55:20.158901+0000 mon.a (mon.0) 1316 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:55:22.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:21 smithi117 bash[28843]: audit 2024-03-04T15:55:20.168470+0000 mon.a (mon.0) 1317 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-11", "mode": "writeback"}]': finished 2024-03-04T15:55:22.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:21 smithi117 bash[28843]: audit 2024-03-04T15:55:20.168671+0000 mon.a (mon.0) 1318 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi117-63923-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:22.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:21 smithi117 bash[28843]: audit 2024-03-04T15:55:20.168836+0000 mon.a (mon.0) 1319 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi117-64133-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:55:22.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:21 smithi117 bash[28843]: cluster 2024-03-04T15:55:20.178760+0000 mon.a (mon.0) 1320 : cluster 0 osdmap e132: 8 total, 8 up, 8 in 2024-03-04T15:55:22.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:21 smithi117 bash[28843]: audit 2024-03-04T15:55:20.180901+0000 mon.a (mon.0) 1321 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi117-64133-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi117-64133-10"}]: dispatch 2024-03-04T15:55:22.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:21 smithi117 bash[28843]: audit 2024-03-04T15:55:21.112220+0000 mon.a (mon.0) 1322 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:21 smithi151 bash[23756]: cluster 2024-03-04T15:55:19.696764+0000 mgr.y (mgr.24428) 230 : cluster 0 pgmap v202: 516 pgs: 192 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-04T15:55:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:21 smithi151 bash[23756]: cluster 2024-03-04T15:55:20.158901+0000 mon.a (mon.0) 1316 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:55:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:21 smithi151 bash[23756]: audit 2024-03-04T15:55:20.168470+0000 mon.a (mon.0) 1317 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-11", "mode": "writeback"}]': finished 2024-03-04T15:55:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:21 smithi151 bash[23756]: audit 2024-03-04T15:55:20.168671+0000 mon.a (mon.0) 1318 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi117-63923-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:22.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:21 smithi151 bash[23756]: audit 2024-03-04T15:55:20.168836+0000 mon.a (mon.0) 1319 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi117-64133-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:55:22.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:21 smithi151 bash[23756]: cluster 2024-03-04T15:55:20.178760+0000 mon.a (mon.0) 1320 : cluster 0 osdmap e132: 8 total, 8 up, 8 in 2024-03-04T15:55:22.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:21 smithi151 bash[23756]: audit 2024-03-04T15:55:20.180901+0000 mon.a (mon.0) 1321 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi117-64133-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi117-64133-10"}]: dispatch 2024-03-04T15:55:22.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:21 smithi151 bash[23756]: audit 2024-03-04T15:55:21.112220+0000 mon.a (mon.0) 1322 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:24.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:23 smithi117 bash[21480]: cluster 2024-03-04T15:55:21.698014+0000 mgr.y (mgr.24428) 231 : cluster 0 pgmap v205: 452 pgs: 96 unknown, 356 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 3 op/s 2024-03-04T15:55:24.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:23 smithi117 bash[21480]: cluster 2024-03-04T15:55:21.988931+0000 mon.a (mon.0) 1323 : cluster 0 osdmap e133: 8 total, 8 up, 8 in 2024-03-04T15:55:24.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:23 smithi117 bash[21480]: audit 2024-03-04T15:55:22.005500+0000 mon.a (mon.0) 1324 : audit 1 from='client.? 172.21.15.117:0/1283412757' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi117-63915-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:24.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:23 smithi117 bash[21480]: audit 2024-03-04T15:55:22.860703+0000 mon.a (mon.0) 1325 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:24.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:23 smithi117 bash[21480]: audit 2024-03-04T15:55:22.862004+0000 mon.a (mon.0) 1326 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi117-64133-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi117-64133-10"}]': finished 2024-03-04T15:55:24.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:23 smithi117 bash[21480]: audit 2024-03-04T15:55:22.862201+0000 mon.a (mon.0) 1327 : audit 1 from='client.? 172.21.15.117:0/1283412757' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi117-63915-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:24.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:23 smithi117 bash[21480]: cluster 2024-03-04T15:55:22.870086+0000 mon.a (mon.0) 1328 : cluster 0 osdmap e134: 8 total, 8 up, 8 in 2024-03-04T15:55:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:23 smithi117 bash[28843]: cluster 2024-03-04T15:55:21.698014+0000 mgr.y (mgr.24428) 231 : cluster 0 pgmap v205: 452 pgs: 96 unknown, 356 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 3 op/s 2024-03-04T15:55:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:23 smithi117 bash[28843]: cluster 2024-03-04T15:55:21.988931+0000 mon.a (mon.0) 1323 : cluster 0 osdmap e133: 8 total, 8 up, 8 in 2024-03-04T15:55:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:23 smithi117 bash[28843]: audit 2024-03-04T15:55:22.005500+0000 mon.a (mon.0) 1324 : audit 1 from='client.? 172.21.15.117:0/1283412757' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi117-63915-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:23 smithi117 bash[28843]: audit 2024-03-04T15:55:22.860703+0000 mon.a (mon.0) 1325 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:23 smithi117 bash[28843]: audit 2024-03-04T15:55:22.862004+0000 mon.a (mon.0) 1326 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi117-64133-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi117-64133-10"}]': finished 2024-03-04T15:55:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:23 smithi117 bash[28843]: audit 2024-03-04T15:55:22.862201+0000 mon.a (mon.0) 1327 : audit 1 from='client.? 172.21.15.117:0/1283412757' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi117-63915-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:23 smithi117 bash[28843]: cluster 2024-03-04T15:55:22.870086+0000 mon.a (mon.0) 1328 : cluster 0 osdmap e134: 8 total, 8 up, 8 in 2024-03-04T15:55:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:23 smithi151 bash[23756]: cluster 2024-03-04T15:55:21.698014+0000 mgr.y (mgr.24428) 231 : cluster 0 pgmap v205: 452 pgs: 96 unknown, 356 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.5 KiB/s rd, 3 op/s 2024-03-04T15:55:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:23 smithi151 bash[23756]: cluster 2024-03-04T15:55:21.988931+0000 mon.a (mon.0) 1323 : cluster 0 osdmap e133: 8 total, 8 up, 8 in 2024-03-04T15:55:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:23 smithi151 bash[23756]: audit 2024-03-04T15:55:22.005500+0000 mon.a (mon.0) 1324 : audit 1 from='client.? 172.21.15.117:0/1283412757' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi117-63915-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:23 smithi151 bash[23756]: audit 2024-03-04T15:55:22.860703+0000 mon.a (mon.0) 1325 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:23 smithi151 bash[23756]: audit 2024-03-04T15:55:22.862004+0000 mon.a (mon.0) 1326 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi117-64133-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi117-64133-10"}]': finished 2024-03-04T15:55:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:23 smithi151 bash[23756]: audit 2024-03-04T15:55:22.862201+0000 mon.a (mon.0) 1327 : audit 1 from='client.? 172.21.15.117:0/1283412757' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi117-63915-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:23 smithi151 bash[23756]: cluster 2024-03-04T15:55:22.870086+0000 mon.a (mon.0) 1328 : cluster 0 osdmap e134: 8 total, 8 up, 8 in 2024-03-04T15:55:25.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[28843]: audit 2024-03-04T15:55:22.983252+0000 mon.c (mon.2) 207 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:55:25.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[28843]: cluster 2024-03-04T15:55:23.371028+0000 mon.a (mon.0) 1329 : cluster 3 Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:25.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[28843]: cluster 2024-03-04T15:55:23.699111+0000 mgr.y (mgr.24428) 232 : cluster 0 pgmap v207: 460 pgs: 136 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.6 KiB/s rd, 226 KiB/s wr, 5 op/s 2024-03-04T15:55:25.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[28843]: audit 2024-03-04T15:55:23.787584+0000 mon.a (mon.0) 1330 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:55:25.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[28843]: audit 2024-03-04T15:55:23.864886+0000 mon.a (mon.0) 1331 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:25.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[28843]: audit 2024-03-04T15:55:23.866647+0000 mon.a (mon.0) 1332 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:55:25.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[28843]: cluster 2024-03-04T15:55:23.875312+0000 mon.a (mon.0) 1333 : cluster 0 osdmap e135: 8 total, 8 up, 8 in 2024-03-04T15:55:25.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[28843]: audit 2024-03-04T15:55:23.875976+0000 mon.c (mon.2) 208 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-11"}]: dispatch 2024-03-04T15:55:25.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[28843]: audit 2024-03-04T15:55:23.880563+0000 mon.a (mon.0) 1334 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-11"}]: dispatch 2024-03-04T15:55:25.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[28843]: audit 2024-03-04T15:55:23.888439+0000 mon.b (mon.1) 70 : audit 1 from='client.? 172.21.15.117:0/2941535752' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi117-63923-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:25.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[28843]: audit 2024-03-04T15:55:23.901982+0000 mon.a (mon.0) 1335 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi117-63923-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:25.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[21480]: audit 2024-03-04T15:55:22.983252+0000 mon.c (mon.2) 207 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:55:25.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[21480]: cluster 2024-03-04T15:55:23.371028+0000 mon.a (mon.0) 1329 : cluster 3 Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:25.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[21480]: cluster 2024-03-04T15:55:23.699111+0000 mgr.y (mgr.24428) 232 : cluster 0 pgmap v207: 460 pgs: 136 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.6 KiB/s rd, 226 KiB/s wr, 5 op/s 2024-03-04T15:55:25.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[21480]: audit 2024-03-04T15:55:23.787584+0000 mon.a (mon.0) 1330 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:55:25.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[21480]: audit 2024-03-04T15:55:23.864886+0000 mon.a (mon.0) 1331 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:25.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[21480]: audit 2024-03-04T15:55:23.866647+0000 mon.a (mon.0) 1332 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:55:25.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[21480]: cluster 2024-03-04T15:55:23.875312+0000 mon.a (mon.0) 1333 : cluster 0 osdmap e135: 8 total, 8 up, 8 in 2024-03-04T15:55:25.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[21480]: audit 2024-03-04T15:55:23.875976+0000 mon.c (mon.2) 208 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-11"}]: dispatch 2024-03-04T15:55:25.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[21480]: audit 2024-03-04T15:55:23.880563+0000 mon.a (mon.0) 1334 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-11"}]: dispatch 2024-03-04T15:55:25.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[21480]: audit 2024-03-04T15:55:23.888439+0000 mon.b (mon.1) 70 : audit 1 from='client.? 172.21.15.117:0/2941535752' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi117-63923-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:25.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:24 smithi117 bash[21480]: audit 2024-03-04T15:55:23.901982+0000 mon.a (mon.0) 1335 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi117-63923-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:25.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:24 smithi151 bash[23756]: audit 2024-03-04T15:55:22.983252+0000 mon.c (mon.2) 207 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:55:25.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:24 smithi151 bash[23756]: cluster 2024-03-04T15:55:23.371028+0000 mon.a (mon.0) 1329 : cluster 3 Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:25.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:24 smithi151 bash[23756]: cluster 2024-03-04T15:55:23.699111+0000 mgr.y (mgr.24428) 232 : cluster 0 pgmap v207: 460 pgs: 136 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.6 KiB/s rd, 226 KiB/s wr, 5 op/s 2024-03-04T15:55:25.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:24 smithi151 bash[23756]: audit 2024-03-04T15:55:23.787584+0000 mon.a (mon.0) 1330 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:55:25.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:24 smithi151 bash[23756]: audit 2024-03-04T15:55:23.864886+0000 mon.a (mon.0) 1331 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:25.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:24 smithi151 bash[23756]: audit 2024-03-04T15:55:23.866647+0000 mon.a (mon.0) 1332 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:55:25.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:24 smithi151 bash[23756]: cluster 2024-03-04T15:55:23.875312+0000 mon.a (mon.0) 1333 : cluster 0 osdmap e135: 8 total, 8 up, 8 in 2024-03-04T15:55:25.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:24 smithi151 bash[23756]: audit 2024-03-04T15:55:23.875976+0000 mon.c (mon.2) 208 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-11"}]: dispatch 2024-03-04T15:55:25.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:24 smithi151 bash[23756]: audit 2024-03-04T15:55:23.880563+0000 mon.a (mon.0) 1334 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-11"}]: dispatch 2024-03-04T15:55:25.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:24 smithi151 bash[23756]: audit 2024-03-04T15:55:23.888439+0000 mon.b (mon.1) 70 : audit 1 from='client.? 172.21.15.117:0/2941535752' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi117-63923-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:25.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:24 smithi151 bash[23756]: audit 2024-03-04T15:55:23.901982+0000 mon.a (mon.0) 1335 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi117-63923-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:25.905 INFO:tasks.workunit.client.0.smithi117.stdout: misc: Running main() from gmock_main.cc 2024-03-04T15:55:25.905 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [==========] Running 12 tests from 1 test suite. 2024-03-04T15:55:25.905 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [----------] Global test environment set-up. 2024-03-04T15:55:25.905 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [----------] 12 tests from NeoRadosMisc 2024-03-04T15:55:25.905 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ RUN ] NeoRadosMisc.Version 2024-03-04T15:55:25.905 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ OK ] NeoRadosMisc.Version (16445 ms) 2024-03-04T15:55:25.905 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ RUN ] NeoRadosMisc.WaitOSDMap 2024-03-04T15:55:25.906 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ OK ] NeoRadosMisc.WaitOSDMap (3866 ms) 2024-03-04T15:55:25.906 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ RUN ] NeoRadosMisc.LongName 2024-03-04T15:55:25.906 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ OK ] NeoRadosMisc.LongName (17470 ms) 2024-03-04T15:55:25.906 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ RUN ] NeoRadosMisc.LongLocator 2024-03-04T15:55:25.906 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ OK ] NeoRadosMisc.LongLocator (5160 ms) 2024-03-04T15:55:25.906 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ RUN ] NeoRadosMisc.LongNamespace 2024-03-04T15:55:25.906 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ OK ] NeoRadosMisc.LongNamespace (8406 ms) 2024-03-04T15:55:25.906 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ RUN ] NeoRadosMisc.LongAttrName 2024-03-04T15:55:25.906 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ OK ] NeoRadosMisc.LongAttrName (8622 ms) 2024-03-04T15:55:25.906 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ RUN ] NeoRadosMisc.Exec 2024-03-04T15:55:25.906 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ OK ] NeoRadosMisc.Exec (7461 ms) 2024-03-04T15:55:25.906 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ RUN ] NeoRadosMisc.Operate1 2024-03-04T15:55:25.906 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ OK ] NeoRadosMisc.Operate1 (5391 ms) 2024-03-04T15:55:25.906 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ RUN ] NeoRadosMisc.Operate2 2024-03-04T15:55:25.906 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ OK ] NeoRadosMisc.Operate2 (3687 ms) 2024-03-04T15:55:25.906 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ RUN ] NeoRadosMisc.BigObject 2024-03-04T15:55:25.907 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ OK ] NeoRadosMisc.BigObject (3411 ms) 2024-03-04T15:55:25.907 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ RUN ] NeoRadosMisc.BigAttr 2024-03-04T15:55:25.907 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ OK ] NeoRadosMisc.BigAttr (2699 ms) 2024-03-04T15:55:25.907 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ RUN ] NeoRadosMisc.WriteSame 2024-03-04T15:55:25.907 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ OK ] NeoRadosMisc.WriteSame (3017 ms) 2024-03-04T15:55:25.907 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [----------] 12 tests from NeoRadosMisc (85635 ms total) 2024-03-04T15:55:25.907 INFO:tasks.workunit.client.0.smithi117.stdout: misc: 2024-03-04T15:55:25.907 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [----------] Global test environment tear-down 2024-03-04T15:55:25.907 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [==========] 12 tests from 1 test suite ran. (85636 ms total) 2024-03-04T15:55:25.907 INFO:tasks.workunit.client.0.smithi117.stdout: misc: [ PASSED ] 12 tests. 2024-03-04T15:55:26.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:25 smithi117 bash[28843]: cluster 2024-03-04T15:55:24.868806+0000 mon.a (mon.0) 1336 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:55:26.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:25 smithi117 bash[28843]: audit 2024-03-04T15:55:24.869677+0000 mon.a (mon.0) 1337 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:26.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:25 smithi117 bash[28843]: audit 2024-03-04T15:55:24.871359+0000 mon.a (mon.0) 1338 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-11"}]': finished 2024-03-04T15:55:26.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:25 smithi117 bash[28843]: audit 2024-03-04T15:55:24.871533+0000 mon.a (mon.0) 1339 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi117-63923-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:26.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:25 smithi117 bash[28843]: cluster 2024-03-04T15:55:24.885051+0000 mon.a (mon.0) 1340 : cluster 0 osdmap e136: 8 total, 8 up, 8 in 2024-03-04T15:55:26.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:25 smithi117 bash[28843]: audit 2024-03-04T15:55:24.896341+0000 mon.c (mon.2) 209 : audit 1 from='client.? 172.21.15.117:0/1527770002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi117-63915-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:26.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:25 smithi117 bash[28843]: audit 2024-03-04T15:55:24.898260+0000 mon.a (mon.0) 1341 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi117-63915-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:26.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:25 smithi117 bash[21480]: cluster 2024-03-04T15:55:24.868806+0000 mon.a (mon.0) 1336 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:55:26.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:25 smithi117 bash[21480]: audit 2024-03-04T15:55:24.869677+0000 mon.a (mon.0) 1337 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:26.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:25 smithi117 bash[21480]: audit 2024-03-04T15:55:24.871359+0000 mon.a (mon.0) 1338 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-11"}]': finished 2024-03-04T15:55:26.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:25 smithi117 bash[21480]: audit 2024-03-04T15:55:24.871533+0000 mon.a (mon.0) 1339 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi117-63923-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:26.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:25 smithi117 bash[21480]: cluster 2024-03-04T15:55:24.885051+0000 mon.a (mon.0) 1340 : cluster 0 osdmap e136: 8 total, 8 up, 8 in 2024-03-04T15:55:26.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:25 smithi117 bash[21480]: audit 2024-03-04T15:55:24.896341+0000 mon.c (mon.2) 209 : audit 1 from='client.? 172.21.15.117:0/1527770002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi117-63915-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:26.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:25 smithi117 bash[21480]: audit 2024-03-04T15:55:24.898260+0000 mon.a (mon.0) 1341 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi117-63915-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:25 smithi151 bash[23756]: cluster 2024-03-04T15:55:24.868806+0000 mon.a (mon.0) 1336 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:55:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:25 smithi151 bash[23756]: audit 2024-03-04T15:55:24.869677+0000 mon.a (mon.0) 1337 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:25 smithi151 bash[23756]: audit 2024-03-04T15:55:24.871359+0000 mon.a (mon.0) 1338 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-11"}]': finished 2024-03-04T15:55:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:25 smithi151 bash[23756]: audit 2024-03-04T15:55:24.871533+0000 mon.a (mon.0) 1339 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi117-63923-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:25 smithi151 bash[23756]: cluster 2024-03-04T15:55:24.885051+0000 mon.a (mon.0) 1340 : cluster 0 osdmap e136: 8 total, 8 up, 8 in 2024-03-04T15:55:26.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:25 smithi151 bash[23756]: audit 2024-03-04T15:55:24.896341+0000 mon.c (mon.2) 209 : audit 1 from='client.? 172.21.15.117:0/1527770002' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi117-63915-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:26.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:25 smithi151 bash[23756]: audit 2024-03-04T15:55:24.898260+0000 mon.a (mon.0) 1341 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi117-63915-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:26.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:55:26 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:55:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:55:26.803 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:55:26 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:55:27.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:26 smithi117 bash[21480]: audit 2024-03-04T15:55:25.874314+0000 mon.a (mon.0) 1342 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:27.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:26 smithi117 bash[21480]: audit 2024-03-04T15:55:25.882790+0000 mon.a (mon.0) 1343 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi117-63915-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:27.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:26 smithi117 bash[21480]: cluster 2024-03-04T15:55:25.891520+0000 mon.a (mon.0) 1344 : cluster 0 osdmap e137: 8 total, 8 up, 8 in 2024-03-04T15:55:27.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:26 smithi117 bash[28843]: audit 2024-03-04T15:55:25.874314+0000 mon.a (mon.0) 1342 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:27.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:26 smithi117 bash[28843]: audit 2024-03-04T15:55:25.882790+0000 mon.a (mon.0) 1343 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi117-63915-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:27.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:26 smithi117 bash[28843]: cluster 2024-03-04T15:55:25.891520+0000 mon.a (mon.0) 1344 : cluster 0 osdmap e137: 8 total, 8 up, 8 in 2024-03-04T15:55:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:26 smithi151 bash[23756]: audit 2024-03-04T15:55:25.874314+0000 mon.a (mon.0) 1342 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:26 smithi151 bash[23756]: audit 2024-03-04T15:55:25.882790+0000 mon.a (mon.0) 1343 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi117-63915-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:26 smithi151 bash[23756]: cluster 2024-03-04T15:55:25.891520+0000 mon.a (mon.0) 1344 : cluster 0 osdmap e137: 8 total, 8 up, 8 in 2024-03-04T15:55:28.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:27 smithi151 bash[23756]: cluster 2024-03-04T15:55:25.700239+0000 mgr.y (mgr.24428) 233 : cluster 0 pgmap v210: 492 pgs: 168 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-04T15:55:28.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:27 smithi151 bash[23756]: audit 2024-03-04T15:55:26.530906+0000 mgr.y (mgr.24428) 234 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:55:28.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:27 smithi151 bash[23756]: audit 2024-03-04T15:55:26.876077+0000 mon.a (mon.0) 1345 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:28.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:27 smithi151 bash[23756]: cluster 2024-03-04T15:55:26.894726+0000 mon.a (mon.0) 1346 : cluster 0 osdmap e138: 8 total, 8 up, 8 in 2024-03-04T15:55:28.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:27 smithi151 bash[23756]: audit 2024-03-04T15:55:26.906302+0000 mon.c (mon.2) 210 : audit 1 from='client.? 172.21.15.117:0/2008781195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi117-63923-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:28.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:27 smithi151 bash[23756]: audit 2024-03-04T15:55:26.906683+0000 mon.c (mon.2) 211 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:28.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:27 smithi151 bash[23756]: audit 2024-03-04T15:55:26.907268+0000 mon.a (mon.0) 1347 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi117-63923-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:28.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:27 smithi151 bash[23756]: audit 2024-03-04T15:55:26.908161+0000 mon.a (mon.0) 1348 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:28.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:27 smithi151 bash[23756]: audit 2024-03-04T15:55:27.372694+0000 mon.a (mon.0) 1349 : audit 0 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-04T15:55:28.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:27 smithi151 bash[23756]: audit 2024-03-04T15:55:27.380171+0000 mon.a (mon.0) 1350 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi117-63923-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:28.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:27 smithi151 bash[23756]: audit 2024-03-04T15:55:27.380421+0000 mon.a (mon.0) 1351 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:28.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:27 smithi151 bash[23756]: cluster 2024-03-04T15:55:27.386561+0000 mon.a (mon.0) 1352 : cluster 0 osdmap e139: 8 total, 8 up, 8 in 2024-03-04T15:55:28.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:27 smithi151 bash[23756]: audit 2024-03-04T15:55:27.391098+0000 mon.c (mon.2) 212 : audit 1 from='client.? 172.21.15.117:0/2824855354' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi117-63915-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:28.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:27 smithi151 bash[23756]: audit 2024-03-04T15:55:27.391592+0000 mon.a (mon.0) 1353 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[21480]: cluster 2024-03-04T15:55:25.700239+0000 mgr.y (mgr.24428) 233 : cluster 0 pgmap v210: 492 pgs: 168 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-04T15:55:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[21480]: audit 2024-03-04T15:55:26.530906+0000 mgr.y (mgr.24428) 234 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:55:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[21480]: audit 2024-03-04T15:55:26.876077+0000 mon.a (mon.0) 1345 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[21480]: cluster 2024-03-04T15:55:26.894726+0000 mon.a (mon.0) 1346 : cluster 0 osdmap e138: 8 total, 8 up, 8 in 2024-03-04T15:55:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[21480]: audit 2024-03-04T15:55:26.906302+0000 mon.c (mon.2) 210 : audit 1 from='client.? 172.21.15.117:0/2008781195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi117-63923-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[21480]: audit 2024-03-04T15:55:26.906683+0000 mon.c (mon.2) 211 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[21480]: audit 2024-03-04T15:55:26.907268+0000 mon.a (mon.0) 1347 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi117-63923-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[21480]: audit 2024-03-04T15:55:26.908161+0000 mon.a (mon.0) 1348 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[21480]: audit 2024-03-04T15:55:27.372694+0000 mon.a (mon.0) 1349 : audit 0 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-04T15:55:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[21480]: audit 2024-03-04T15:55:27.380171+0000 mon.a (mon.0) 1350 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi117-63923-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[21480]: audit 2024-03-04T15:55:27.380421+0000 mon.a (mon.0) 1351 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[21480]: cluster 2024-03-04T15:55:27.386561+0000 mon.a (mon.0) 1352 : cluster 0 osdmap e139: 8 total, 8 up, 8 in 2024-03-04T15:55:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[21480]: audit 2024-03-04T15:55:27.391098+0000 mon.c (mon.2) 212 : audit 1 from='client.? 172.21.15.117:0/2824855354' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi117-63915-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[21480]: audit 2024-03-04T15:55:27.391592+0000 mon.a (mon.0) 1353 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:28.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[28843]: cluster 2024-03-04T15:55:25.700239+0000 mgr.y (mgr.24428) 233 : cluster 0 pgmap v210: 492 pgs: 168 unknown, 324 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-04T15:55:28.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[28843]: audit 2024-03-04T15:55:26.530906+0000 mgr.y (mgr.24428) 234 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:55:28.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[28843]: audit 2024-03-04T15:55:26.876077+0000 mon.a (mon.0) 1345 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:28.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[28843]: cluster 2024-03-04T15:55:26.894726+0000 mon.a (mon.0) 1346 : cluster 0 osdmap e138: 8 total, 8 up, 8 in 2024-03-04T15:55:28.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[28843]: audit 2024-03-04T15:55:26.906302+0000 mon.c (mon.2) 210 : audit 1 from='client.? 172.21.15.117:0/2008781195' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi117-63923-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:28.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[28843]: audit 2024-03-04T15:55:26.906683+0000 mon.c (mon.2) 211 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:28.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[28843]: audit 2024-03-04T15:55:26.907268+0000 mon.a (mon.0) 1347 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi117-63923-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:28.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[28843]: audit 2024-03-04T15:55:26.908161+0000 mon.a (mon.0) 1348 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:28.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[28843]: audit 2024-03-04T15:55:27.372694+0000 mon.a (mon.0) 1349 : audit 0 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-03-04T15:55:28.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[28843]: audit 2024-03-04T15:55:27.380171+0000 mon.a (mon.0) 1350 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi117-63923-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:28.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[28843]: audit 2024-03-04T15:55:27.380421+0000 mon.a (mon.0) 1351 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:28.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[28843]: cluster 2024-03-04T15:55:27.386561+0000 mon.a (mon.0) 1352 : cluster 0 osdmap e139: 8 total, 8 up, 8 in 2024-03-04T15:55:28.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[28843]: audit 2024-03-04T15:55:27.391098+0000 mon.c (mon.2) 212 : audit 1 from='client.? 172.21.15.117:0/2824855354' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi117-63915-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:28.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:27 smithi117 bash[28843]: audit 2024-03-04T15:55:27.391592+0000 mon.a (mon.0) 1353 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:28 smithi151 bash[23756]: cluster 2024-03-04T15:55:27.701311+0000 mgr.y (mgr.24428) 235 : cluster 0 pgmap v214: 460 pgs: 32 creating+peering, 96 unknown, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 5.2 KiB/s wr, 5 op/s 2024-03-04T15:55:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:28 smithi151 bash[23756]: audit 2024-03-04T15:55:27.856815+0000 mon.a (mon.0) 1354 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi117-63915-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:28 smithi151 bash[23756]: audit 2024-03-04T15:55:27.877367+0000 mon.a (mon.0) 1355 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:28 smithi151 bash[23756]: audit 2024-03-04T15:55:28.127291+0000 mon.c (mon.2) 213 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:55:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:28 smithi151 bash[23756]: audit 2024-03-04T15:55:28.384622+0000 mon.a (mon.0) 1356 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:28 smithi151 bash[23756]: audit 2024-03-04T15:55:28.384830+0000 mon.a (mon.0) 1357 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi117-63915-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:28 smithi151 bash[23756]: cluster 2024-03-04T15:55:28.393966+0000 mon.a (mon.0) 1358 : cluster 0 osdmap e140: 8 total, 8 up, 8 in 2024-03-04T15:55:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:28 smithi151 bash[23756]: audit 2024-03-04T15:55:28.399456+0000 mon.a (mon.0) 1359 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "app2"}]: dispatch 2024-03-04T15:55:29.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:28 smithi151 bash[23756]: audit 2024-03-04T15:55:28.404380+0000 mon.a (mon.0) 1360 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:29.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:28 smithi117 bash[21480]: cluster 2024-03-04T15:55:27.701311+0000 mgr.y (mgr.24428) 235 : cluster 0 pgmap v214: 460 pgs: 32 creating+peering, 96 unknown, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 5.2 KiB/s wr, 5 op/s 2024-03-04T15:55:29.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:28 smithi117 bash[21480]: audit 2024-03-04T15:55:27.856815+0000 mon.a (mon.0) 1354 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi117-63915-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:29.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:28 smithi117 bash[21480]: audit 2024-03-04T15:55:27.877367+0000 mon.a (mon.0) 1355 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:29.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:28 smithi117 bash[21480]: audit 2024-03-04T15:55:28.127291+0000 mon.c (mon.2) 213 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:55:29.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:28 smithi117 bash[21480]: audit 2024-03-04T15:55:28.384622+0000 mon.a (mon.0) 1356 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:29.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:28 smithi117 bash[21480]: audit 2024-03-04T15:55:28.384830+0000 mon.a (mon.0) 1357 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi117-63915-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:29.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:28 smithi117 bash[21480]: cluster 2024-03-04T15:55:28.393966+0000 mon.a (mon.0) 1358 : cluster 0 osdmap e140: 8 total, 8 up, 8 in 2024-03-04T15:55:29.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:28 smithi117 bash[21480]: audit 2024-03-04T15:55:28.399456+0000 mon.a (mon.0) 1359 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "app2"}]: dispatch 2024-03-04T15:55:29.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:28 smithi117 bash[21480]: audit 2024-03-04T15:55:28.404380+0000 mon.a (mon.0) 1360 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:29.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:28 smithi117 bash[28843]: cluster 2024-03-04T15:55:27.701311+0000 mgr.y (mgr.24428) 235 : cluster 0 pgmap v214: 460 pgs: 32 creating+peering, 96 unknown, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 0 B/s rd, 5.2 KiB/s wr, 5 op/s 2024-03-04T15:55:29.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:28 smithi117 bash[28843]: audit 2024-03-04T15:55:27.856815+0000 mon.a (mon.0) 1354 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi117-63915-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:29.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:28 smithi117 bash[28843]: audit 2024-03-04T15:55:27.877367+0000 mon.a (mon.0) 1355 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:29.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:28 smithi117 bash[28843]: audit 2024-03-04T15:55:28.127291+0000 mon.c (mon.2) 213 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:55:29.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:28 smithi117 bash[28843]: audit 2024-03-04T15:55:28.384622+0000 mon.a (mon.0) 1356 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:29.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:28 smithi117 bash[28843]: audit 2024-03-04T15:55:28.384830+0000 mon.a (mon.0) 1357 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi117-63915-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:29.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:28 smithi117 bash[28843]: cluster 2024-03-04T15:55:28.393966+0000 mon.a (mon.0) 1358 : cluster 0 osdmap e140: 8 total, 8 up, 8 in 2024-03-04T15:55:29.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:28 smithi117 bash[28843]: audit 2024-03-04T15:55:28.399456+0000 mon.a (mon.0) 1359 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "app2"}]: dispatch 2024-03-04T15:55:29.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:28 smithi117 bash[28843]: audit 2024-03-04T15:55:28.404380+0000 mon.a (mon.0) 1360 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:30.415 INFO:tasks.workunit.client.0.smithi117.stdout: io: Running main() from gmock_main.cc 2024-03-04T15:55:30.416 INFO:tasks.workunit.client.0.smithi117.stdout: io: [==========] Running 14 tests from 1 test suite. 2024-03-04T15:55:30.416 INFO:tasks.workunit.client.0.smithi117.stdout: io: [----------] Global test environment set-up. 2024-03-04T15:55:30.416 INFO:tasks.workunit.client.0.smithi117.stdout: io: [----------] 14 tests from NeoRadosIo 2024-03-04T15:55:30.416 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ RUN ] NeoRadosIo.Limits 2024-03-04T15:55:30.416 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ OK ] NeoRadosIo.Limits (18061 ms) 2024-03-04T15:55:30.416 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ RUN ] NeoRadosIo.SimpleWrite 2024-03-04T15:55:30.416 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ OK ] NeoRadosIo.SimpleWrite (13294 ms) 2024-03-04T15:55:30.416 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ RUN ] NeoRadosIo.ReadOp 2024-03-04T15:55:30.416 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ OK ] NeoRadosIo.ReadOp (6506 ms) 2024-03-04T15:55:30.416 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ RUN ] NeoRadosIo.SparseRead 2024-03-04T15:55:30.416 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ OK ] NeoRadosIo.SparseRead (6255 ms) 2024-03-04T15:55:30.416 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ RUN ] NeoRadosIo.RoundTrip 2024-03-04T15:55:30.416 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ OK ] NeoRadosIo.RoundTrip (7315 ms) 2024-03-04T15:55:30.417 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ RUN ] NeoRadosIo.ReadIntoBuufferlist 2024-03-04T15:55:30.417 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ OK ] NeoRadosIo.ReadIntoBuufferlist (8569 ms) 2024-03-04T15:55:30.417 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ RUN ] NeoRadosIo.OverlappingWriteRoundTrip 2024-03-04T15:55:30.417 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ OK ] NeoRadosIo.OverlappingWriteRoundTrip (5615 ms) 2024-03-04T15:55:30.417 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ RUN ] NeoRadosIo.WriteFullRoundTrip 2024-03-04T15:55:30.417 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ OK ] NeoRadosIo.WriteFullRoundTrip (5010 ms) 2024-03-04T15:55:30.417 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ RUN ] NeoRadosIo.AppendRoundTrip 2024-03-04T15:55:30.417 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ OK ] NeoRadosIo.AppendRoundTrip (3272 ms) 2024-03-04T15:55:30.417 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ RUN ] NeoRadosIo.Trunc 2024-03-04T15:55:30.417 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ OK ] NeoRadosIo.Trunc (2669 ms) 2024-03-04T15:55:30.417 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ RUN ] NeoRadosIo.Remove 2024-03-04T15:55:30.418 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ OK ] NeoRadosIo.Remove (3405 ms) 2024-03-04T15:55:30.418 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ RUN ] NeoRadosIo.XattrsRoundTrip 2024-03-04T15:55:30.418 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ OK ] NeoRadosIo.XattrsRoundTrip (3697 ms) 2024-03-04T15:55:30.418 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ RUN ] NeoRadosIo.RmXattr 2024-03-04T15:55:30.418 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ OK ] NeoRadosIo.RmXattr (3027 ms) 2024-03-04T15:55:30.418 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ RUN ] NeoRadosIo.GetXattrs 2024-03-04T15:55:30.418 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ OK ] NeoRadosIo.GetXattrs (3504 ms) 2024-03-04T15:55:30.418 INFO:tasks.workunit.client.0.smithi117.stdout: io: [----------] 14 tests from NeoRadosIo (90199 ms total) 2024-03-04T15:55:30.418 INFO:tasks.workunit.client.0.smithi117.stdout: io: 2024-03-04T15:55:30.418 INFO:tasks.workunit.client.0.smithi117.stdout: io: [----------] Global test environment tear-down 2024-03-04T15:55:30.418 INFO:tasks.workunit.client.0.smithi117.stdout: io: [==========] 14 tests from 1 test suite ran. (90199 ms total) 2024-03-04T15:55:30.418 INFO:tasks.workunit.client.0.smithi117.stdout: io: [ PASSED ] 14 tests. 2024-03-04T15:55:30.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:30 smithi117 bash[21480]: cluster 2024-03-04T15:55:28.900758+0000 mon.a (mon.0) 1361 : cluster 3 Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:30.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:30 smithi117 bash[21480]: audit 2024-03-04T15:55:28.901981+0000 mon.a (mon.0) 1362 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:30.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:30 smithi117 bash[21480]: audit 2024-03-04T15:55:29.390575+0000 mon.a (mon.0) 1363 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:30.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:30 smithi117 bash[21480]: cluster 2024-03-04T15:55:29.411597+0000 mon.a (mon.0) 1364 : cluster 0 osdmap e141: 8 total, 8 up, 8 in 2024-03-04T15:55:30.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:30 smithi117 bash[21480]: audit 2024-03-04T15:55:29.416337+0000 mon.a (mon.0) 1365 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi117-63991-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-03-04T15:55:30.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:30 smithi117 bash[21480]: audit 2024-03-04T15:55:29.416860+0000 mon.a (mon.0) 1366 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi117-63991-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-04T15:55:30.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:30 smithi117 bash[21480]: audit 2024-03-04T15:55:29.433420+0000 mon.a (mon.0) 1367 : audit 1 from='client.? 172.21.15.117:0/3553256248' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi117-63923-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:30 smithi117 bash[28843]: cluster 2024-03-04T15:55:28.900758+0000 mon.a (mon.0) 1361 : cluster 3 Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:30 smithi117 bash[28843]: audit 2024-03-04T15:55:28.901981+0000 mon.a (mon.0) 1362 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:30 smithi117 bash[28843]: audit 2024-03-04T15:55:29.390575+0000 mon.a (mon.0) 1363 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:30 smithi117 bash[28843]: cluster 2024-03-04T15:55:29.411597+0000 mon.a (mon.0) 1364 : cluster 0 osdmap e141: 8 total, 8 up, 8 in 2024-03-04T15:55:30.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:30 smithi117 bash[28843]: audit 2024-03-04T15:55:29.416337+0000 mon.a (mon.0) 1365 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi117-63991-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-03-04T15:55:30.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:30 smithi117 bash[28843]: audit 2024-03-04T15:55:29.416860+0000 mon.a (mon.0) 1366 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi117-63991-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-04T15:55:30.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:30 smithi117 bash[28843]: audit 2024-03-04T15:55:29.433420+0000 mon.a (mon.0) 1367 : audit 1 from='client.? 172.21.15.117:0/3553256248' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi117-63923-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:30 smithi151 bash[23756]: cluster 2024-03-04T15:55:28.900758+0000 mon.a (mon.0) 1361 : cluster 3 Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:30 smithi151 bash[23756]: audit 2024-03-04T15:55:28.901981+0000 mon.a (mon.0) 1362 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:30 smithi151 bash[23756]: audit 2024-03-04T15:55:29.390575+0000 mon.a (mon.0) 1363 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi117-63991-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:30 smithi151 bash[23756]: cluster 2024-03-04T15:55:29.411597+0000 mon.a (mon.0) 1364 : cluster 0 osdmap e141: 8 total, 8 up, 8 in 2024-03-04T15:55:30.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:30 smithi151 bash[23756]: audit 2024-03-04T15:55:29.416337+0000 mon.a (mon.0) 1365 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi117-63991-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-03-04T15:55:30.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:30 smithi151 bash[23756]: audit 2024-03-04T15:55:29.416860+0000 mon.a (mon.0) 1366 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi117-63991-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-03-04T15:55:30.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:30 smithi151 bash[23756]: audit 2024-03-04T15:55:29.433420+0000 mon.a (mon.0) 1367 : audit 1 from='client.? 172.21.15.117:0/3553256248' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi117-63923-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:31.439 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: Running main() from gmock_main.cc 2024-03-04T15:55:31.440 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [==========] Running 14 tests from 1 test suite. 2024-03-04T15:55:31.440 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [----------] Global test environment set-up. 2024-03-04T15:55:31.440 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps 2024-03-04T15:55:31.440 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ RUN ] NeoRadosReadOps.SetOpFlags 2024-03-04T15:55:31.440 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ OK ] NeoRadosReadOps.SetOpFlags (17995 ms) 2024-03-04T15:55:31.440 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertExists 2024-03-04T15:55:31.440 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertExists (14750 ms) 2024-03-04T15:55:31.440 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ RUN ] NeoRadosReadOps.AssertVersion 2024-03-04T15:55:31.440 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ OK ] NeoRadosReadOps.AssertVersion (7818 ms) 2024-03-04T15:55:31.440 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpXattr 2024-03-04T15:55:31.440 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpXattr (4590 ms) 2024-03-04T15:55:31.440 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ RUN ] NeoRadosReadOps.Read 2024-03-04T15:55:31.440 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ OK ] NeoRadosReadOps.Read (6192 ms) 2024-03-04T15:55:31.440 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ RUN ] NeoRadosReadOps.Checksum 2024-03-04T15:55:31.441 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ OK ] NeoRadosReadOps.Checksum (8683 ms) 2024-03-04T15:55:31.441 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ RUN ] NeoRadosReadOps.RWOrderedRead 2024-03-04T15:55:31.441 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ OK ] NeoRadosReadOps.RWOrderedRead (5501 ms) 2024-03-04T15:55:31.441 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ RUN ] NeoRadosReadOps.ShortRead 2024-03-04T15:55:31.441 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ OK ] NeoRadosReadOps.ShortRead (5030 ms) 2024-03-04T15:55:31.441 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ RUN ] NeoRadosReadOps.Exec 2024-03-04T15:55:31.441 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ OK ] NeoRadosReadOps.Exec (4286 ms) 2024-03-04T15:55:31.441 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ RUN ] NeoRadosReadOps.Stat 2024-03-04T15:55:31.441 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ OK ] NeoRadosReadOps.Stat (3044 ms) 2024-03-04T15:55:31.441 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ RUN ] NeoRadosReadOps.Omap 2024-03-04T15:55:31.441 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ OK ] NeoRadosReadOps.Omap (3827 ms) 2024-03-04T15:55:31.441 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ RUN ] NeoRadosReadOps.OmapNuls 2024-03-04T15:55:31.441 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ OK ] NeoRadosReadOps.OmapNuls (2904 ms) 2024-03-04T15:55:31.441 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ RUN ] NeoRadosReadOps.GetXattrs 2024-03-04T15:55:31.441 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ OK ] NeoRadosReadOps.GetXattrs (3514 ms) 2024-03-04T15:55:31.441 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ RUN ] NeoRadosReadOps.CmpExt 2024-03-04T15:55:31.441 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ OK ] NeoRadosReadOps.CmpExt (3022 ms) 2024-03-04T15:55:31.441 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [----------] 14 tests from NeoRadosReadOps (91156 ms total) 2024-03-04T15:55:31.442 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: 2024-03-04T15:55:31.442 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [----------] Global test environment tear-down 2024-03-04T15:55:31.442 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [==========] 14 tests from 1 test suite ran. (91156 ms total) 2024-03-04T15:55:31.442 INFO:tasks.workunit.client.0.smithi117.stdout: read_operations: [ PASSED ] 14 tests. 2024-03-04T15:55:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:31 smithi151 bash[23756]: cluster 2024-03-04T15:55:29.702378+0000 mgr.y (mgr.24428) 236 : cluster 0 pgmap v217: 428 pgs: 128 unknown, 300 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-03-04T15:55:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:31 smithi151 bash[23756]: audit 2024-03-04T15:55:30.319793+0000 mon.a (mon.0) 1368 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:31 smithi151 bash[23756]: audit 2024-03-04T15:55:30.395477+0000 mon.a (mon.0) 1369 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi117-63991-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-04T15:55:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:31 smithi151 bash[23756]: audit 2024-03-04T15:55:30.395653+0000 mon.a (mon.0) 1370 : audit 1 from='client.? 172.21.15.117:0/3553256248' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi117-63923-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:31 smithi151 bash[23756]: cluster 2024-03-04T15:55:30.401448+0000 mon.a (mon.0) 1371 : cluster 0 osdmap e142: 8 total, 8 up, 8 in 2024-03-04T15:55:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:31 smithi151 bash[23756]: audit 2024-03-04T15:55:30.405849+0000 mon.b (mon.1) 71 : audit 1 from='client.? 172.21.15.117:0/2867130944' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi117-63915-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:31 smithi151 bash[23756]: audit 2024-03-04T15:55:30.413403+0000 mon.a (mon.0) 1372 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi117-63915-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:31.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:31 smithi151 bash[23756]: audit 2024-03-04T15:55:30.413618+0000 mon.a (mon.0) 1373 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi117-63991-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-04T15:55:31.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:31 smithi117 bash[28843]: cluster 2024-03-04T15:55:29.702378+0000 mgr.y (mgr.24428) 236 : cluster 0 pgmap v217: 428 pgs: 128 unknown, 300 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-03-04T15:55:31.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:31 smithi117 bash[28843]: audit 2024-03-04T15:55:30.319793+0000 mon.a (mon.0) 1368 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:31 smithi117 bash[28843]: audit 2024-03-04T15:55:30.395477+0000 mon.a (mon.0) 1369 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi117-63991-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-04T15:55:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:31 smithi117 bash[28843]: audit 2024-03-04T15:55:30.395653+0000 mon.a (mon.0) 1370 : audit 1 from='client.? 172.21.15.117:0/3553256248' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi117-63923-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:31 smithi117 bash[28843]: cluster 2024-03-04T15:55:30.401448+0000 mon.a (mon.0) 1371 : cluster 0 osdmap e142: 8 total, 8 up, 8 in 2024-03-04T15:55:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:31 smithi117 bash[28843]: audit 2024-03-04T15:55:30.405849+0000 mon.b (mon.1) 71 : audit 1 from='client.? 172.21.15.117:0/2867130944' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi117-63915-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:31 smithi117 bash[28843]: audit 2024-03-04T15:55:30.413403+0000 mon.a (mon.0) 1372 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi117-63915-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:31 smithi117 bash[28843]: audit 2024-03-04T15:55:30.413618+0000 mon.a (mon.0) 1373 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi117-63991-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-04T15:55:31.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:31 smithi117 bash[21480]: cluster 2024-03-04T15:55:29.702378+0000 mgr.y (mgr.24428) 236 : cluster 0 pgmap v217: 428 pgs: 128 unknown, 300 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.6 KiB/s wr, 2 op/s 2024-03-04T15:55:31.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:31 smithi117 bash[21480]: audit 2024-03-04T15:55:30.319793+0000 mon.a (mon.0) 1368 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:31.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:31 smithi117 bash[21480]: audit 2024-03-04T15:55:30.395477+0000 mon.a (mon.0) 1369 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi117-63991-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-03-04T15:55:31.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:31 smithi117 bash[21480]: audit 2024-03-04T15:55:30.395653+0000 mon.a (mon.0) 1370 : audit 1 from='client.? 172.21.15.117:0/3553256248' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi117-63923-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:31.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:31 smithi117 bash[21480]: cluster 2024-03-04T15:55:30.401448+0000 mon.a (mon.0) 1371 : cluster 0 osdmap e142: 8 total, 8 up, 8 in 2024-03-04T15:55:31.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:31 smithi117 bash[21480]: audit 2024-03-04T15:55:30.405849+0000 mon.b (mon.1) 71 : audit 1 from='client.? 172.21.15.117:0/2867130944' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi117-63915-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:31.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:31 smithi117 bash[21480]: audit 2024-03-04T15:55:30.413403+0000 mon.a (mon.0) 1372 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi117-63915-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:31.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:31 smithi117 bash[21480]: audit 2024-03-04T15:55:30.413618+0000 mon.a (mon.0) 1373 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi117-63991-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-03-04T15:55:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:32 smithi151 bash[23756]: audit 2024-03-04T15:55:31.321753+0000 mon.a (mon.0) 1374 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:32 smithi151 bash[23756]: audit 2024-03-04T15:55:31.401074+0000 mon.a (mon.0) 1375 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi117-63915-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:32 smithi151 bash[23756]: audit 2024-03-04T15:55:31.401377+0000 mon.a (mon.0) 1376 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi117-63991-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-04T15:55:32.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:32 smithi151 bash[23756]: cluster 2024-03-04T15:55:31.413275+0000 mon.a (mon.0) 1377 : cluster 0 osdmap e143: 8 total, 8 up, 8 in 2024-03-04T15:55:32.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:32 smithi151 bash[23756]: audit 2024-03-04T15:55:31.419224+0000 mon.a (mon.0) 1378 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi117-63991-1","app":"app1","key":"key1"}]: dispatch 2024-03-04T15:55:32.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:32 smithi151 bash[23756]: audit 2024-03-04T15:55:31.662919+0000 mon.c (mon.2) 214 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:55:32.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:32 smithi151 bash[23756]: cluster 2024-03-04T15:55:31.703001+0000 mgr.y (mgr.24428) 237 : cluster 0 pgmap v220: 396 pgs: 64 creating+peering, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 98 KiB/s wr, 99 op/s 2024-03-04T15:55:32.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:32 smithi151 bash[23756]: audit 2024-03-04T15:55:31.919144+0000 mon.a (mon.0) 1379 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:55:32.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:32 smithi117 bash[28843]: audit 2024-03-04T15:55:31.321753+0000 mon.a (mon.0) 1374 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:32.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:32 smithi117 bash[28843]: audit 2024-03-04T15:55:31.401074+0000 mon.a (mon.0) 1375 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi117-63915-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:32.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:32 smithi117 bash[28843]: audit 2024-03-04T15:55:31.401377+0000 mon.a (mon.0) 1376 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi117-63991-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-04T15:55:32.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:32 smithi117 bash[28843]: cluster 2024-03-04T15:55:31.413275+0000 mon.a (mon.0) 1377 : cluster 0 osdmap e143: 8 total, 8 up, 8 in 2024-03-04T15:55:32.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:32 smithi117 bash[28843]: audit 2024-03-04T15:55:31.419224+0000 mon.a (mon.0) 1378 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi117-63991-1","app":"app1","key":"key1"}]: dispatch 2024-03-04T15:55:32.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:32 smithi117 bash[28843]: audit 2024-03-04T15:55:31.662919+0000 mon.c (mon.2) 214 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:55:32.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:32 smithi117 bash[28843]: cluster 2024-03-04T15:55:31.703001+0000 mgr.y (mgr.24428) 237 : cluster 0 pgmap v220: 396 pgs: 64 creating+peering, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 98 KiB/s wr, 99 op/s 2024-03-04T15:55:32.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:32 smithi117 bash[28843]: audit 2024-03-04T15:55:31.919144+0000 mon.a (mon.0) 1379 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:55:32.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:32 smithi117 bash[21480]: audit 2024-03-04T15:55:31.321753+0000 mon.a (mon.0) 1374 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:32.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:32 smithi117 bash[21480]: audit 2024-03-04T15:55:31.401074+0000 mon.a (mon.0) 1375 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi117-63915-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:32.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:32 smithi117 bash[21480]: audit 2024-03-04T15:55:31.401377+0000 mon.a (mon.0) 1376 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi117-63991-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-03-04T15:55:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:32 smithi117 bash[21480]: cluster 2024-03-04T15:55:31.413275+0000 mon.a (mon.0) 1377 : cluster 0 osdmap e143: 8 total, 8 up, 8 in 2024-03-04T15:55:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:32 smithi117 bash[21480]: audit 2024-03-04T15:55:31.419224+0000 mon.a (mon.0) 1378 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi117-63991-1","app":"app1","key":"key1"}]: dispatch 2024-03-04T15:55:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:32 smithi117 bash[21480]: audit 2024-03-04T15:55:31.662919+0000 mon.c (mon.2) 214 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:55:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:32 smithi117 bash[21480]: cluster 2024-03-04T15:55:31.703001+0000 mgr.y (mgr.24428) 237 : cluster 0 pgmap v220: 396 pgs: 64 creating+peering, 332 active+clean; 217 MiB data, 1.1 GiB used, 714 GiB / 715 GiB avail; 2.0 KiB/s rd, 98 KiB/s wr, 99 op/s 2024-03-04T15:55:32.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:32 smithi117 bash[21480]: audit 2024-03-04T15:55:31.919144+0000 mon.a (mon.0) 1379 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:55:33.406 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-03-04T15:55:33.406 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-03-04T15:55:33.406 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-03-04T15:55:33.406 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-03-04T15:55:33.406 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-03-04T15:55:33.406 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-03-04T15:55:33.406 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: 2024-03-04T15:55:33.406 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-03-04T15:55:33.406 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: seed 63991 2024-03-04T15:55:33.406 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-03-04T15:55:33.406 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (443 ms) 2024-03-04T15:55:33.407 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-03-04T15:55:33.407 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (19 ms) 2024-03-04T15:55:33.407 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-03-04T15:55:33.407 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (118 ms) 2024-03-04T15:55:33.407 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-03-04T15:55:33.407 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (6 ms) 2024-03-04T15:55:33.407 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-03-04T15:55:33.407 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (6 ms) 2024-03-04T15:55:33.407 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-03-04T15:55:33.407 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (1 ms) 2024-03-04T15:55:33.407 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-03-04T15:55:33.407 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (2 ms) 2024-03-04T15:55:33.407 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-03-04T15:55:33.407 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (4 ms) 2024-03-04T15:55:33.407 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-03-04T15:55:33.408 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (2 ms) 2024-03-04T15:55:33.408 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-03-04T15:55:33.408 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (6 ms) 2024-03-04T15:55:33.408 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-03-04T15:55:33.408 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (2 ms) 2024-03-04T15:55:33.408 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-03-04T15:55:33.408 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (3 ms) 2024-03-04T15:55:33.408 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-03-04T15:55:33.408 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (5 ms) 2024-03-04T15:55:33.408 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-03-04T15:55:33.408 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: osd_max_attr_size = 0 2024-03-04T15:55:33.408 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-03-04T15:55:33.408 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (2134 ms) 2024-03-04T15:55:33.408 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-03-04T15:55:33.408 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (489 ms) 2024-03-04T15:55:33.409 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-03-04T15:55:33.409 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-03-04T15:55:33.409 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: done waiting, doing copies 2024-03-04T15:55:33.409 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: waiting for final deep scrubs... 2024-03-04T15:55:33.409 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: done waiting 2024-03-04T15:55:33.409 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (65107 ms) 2024-03-04T15:55:33.409 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-03-04T15:55:33.409 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (9 ms) 2024-03-04T15:55:33.409 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-03-04T15:55:33.409 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (4 ms) 2024-03-04T15:55:33.409 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-03-04T15:55:33.409 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (5338 ms) 2024-03-04T15:55:33.409 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-03-04T15:55:33.409 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-03-04T15:55:33.409 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-03-04T15:55:33.410 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-03-04T15:55:33.410 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-03-04T15:55:33.410 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-03-04T15:55:33.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:33 smithi151 bash[23756]: audit 2024-03-04T15:55:32.323949+0000 mon.a (mon.0) 1380 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:33.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:33 smithi151 bash[23756]: audit 2024-03-04T15:55:32.377318+0000 mon.a (mon.0) 1381 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi117-63991-1","app":"app1","key":"key1"}]': finished 2024-03-04T15:55:33.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:33 smithi151 bash[23756]: audit 2024-03-04T15:55:32.377523+0000 mon.a (mon.0) 1382 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-13", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:55:33.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:33 smithi151 bash[23756]: audit 2024-03-04T15:55:32.386317+0000 mon.c (mon.2) 215 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-13"}]: dispatch 2024-03-04T15:55:33.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:33 smithi151 bash[23756]: cluster 2024-03-04T15:55:32.386546+0000 mon.a (mon.0) 1383 : cluster 0 osdmap e144: 8 total, 8 up, 8 in 2024-03-04T15:55:33.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:33 smithi151 bash[23756]: audit 2024-03-04T15:55:32.409109+0000 mon.a (mon.0) 1384 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-13"}]: dispatch 2024-03-04T15:55:33.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:33 smithi151 bash[23756]: audit 2024-03-04T15:55:32.410414+0000 mon.a (mon.0) 1385 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi117-64133-10"}]: dispatch 2024-03-04T15:55:33.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:33 smithi151 bash[23756]: audit 2024-03-04T15:55:32.410987+0000 mon.a (mon.0) 1386 : audit 1 from='client.? 172.21.15.117:0/1006594653' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi117-63923-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:33.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:33 smithi117 bash[21480]: audit 2024-03-04T15:55:32.323949+0000 mon.a (mon.0) 1380 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:33.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:33 smithi117 bash[21480]: audit 2024-03-04T15:55:32.377318+0000 mon.a (mon.0) 1381 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi117-63991-1","app":"app1","key":"key1"}]': finished 2024-03-04T15:55:33.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:33 smithi117 bash[21480]: audit 2024-03-04T15:55:32.377523+0000 mon.a (mon.0) 1382 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-13", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:55:33.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:33 smithi117 bash[21480]: audit 2024-03-04T15:55:32.386317+0000 mon.c (mon.2) 215 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-13"}]: dispatch 2024-03-04T15:55:33.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:33 smithi117 bash[21480]: cluster 2024-03-04T15:55:32.386546+0000 mon.a (mon.0) 1383 : cluster 0 osdmap e144: 8 total, 8 up, 8 in 2024-03-04T15:55:33.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:33 smithi117 bash[21480]: audit 2024-03-04T15:55:32.409109+0000 mon.a (mon.0) 1384 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-13"}]: dispatch 2024-03-04T15:55:33.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:33 smithi117 bash[21480]: audit 2024-03-04T15:55:32.410414+0000 mon.a (mon.0) 1385 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi117-64133-10"}]: dispatch 2024-03-04T15:55:33.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:33 smithi117 bash[21480]: audit 2024-03-04T15:55:32.410987+0000 mon.a (mon.0) 1386 : audit 1 from='client.? 172.21.15.117:0/1006594653' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi117-63923-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:33.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:33 smithi117 bash[28843]: audit 2024-03-04T15:55:32.323949+0000 mon.a (mon.0) 1380 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:33.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:33 smithi117 bash[28843]: audit 2024-03-04T15:55:32.377318+0000 mon.a (mon.0) 1381 : audit 1 from='client.? 172.21.15.117:0/2318822253' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi117-63991-1","app":"app1","key":"key1"}]': finished 2024-03-04T15:55:33.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:33 smithi117 bash[28843]: audit 2024-03-04T15:55:32.377523+0000 mon.a (mon.0) 1382 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-13", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:55:33.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:33 smithi117 bash[28843]: audit 2024-03-04T15:55:32.386317+0000 mon.c (mon.2) 215 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-13"}]: dispatch 2024-03-04T15:55:33.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:33 smithi117 bash[28843]: cluster 2024-03-04T15:55:32.386546+0000 mon.a (mon.0) 1383 : cluster 0 osdmap e144: 8 total, 8 up, 8 in 2024-03-04T15:55:33.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:33 smithi117 bash[28843]: audit 2024-03-04T15:55:32.409109+0000 mon.a (mon.0) 1384 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-13"}]: dispatch 2024-03-04T15:55:33.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:33 smithi117 bash[28843]: audit 2024-03-04T15:55:32.410414+0000 mon.a (mon.0) 1385 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi117-64133-10"}]: dispatch 2024-03-04T15:55:33.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:33 smithi117 bash[28843]: audit 2024-03-04T15:55:32.410987+0000 mon.a (mon.0) 1386 : audit 1 from='client.? 172.21.15.117:0/1006594653' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi117-63923-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:34.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:34 smithi151 bash[23756]: audit 2024-03-04T15:55:33.326281+0000 mon.a (mon.0) 1387 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:34.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:34 smithi151 bash[23756]: audit 2024-03-04T15:55:33.381548+0000 mon.a (mon.0) 1388 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-13"}]': finished 2024-03-04T15:55:34.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:34 smithi151 bash[23756]: audit 2024-03-04T15:55:33.381759+0000 mon.a (mon.0) 1389 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi117-64133-10"}]': finished 2024-03-04T15:55:34.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:34 smithi151 bash[23756]: audit 2024-03-04T15:55:33.381933+0000 mon.a (mon.0) 1390 : audit 1 from='client.? 172.21.15.117:0/1006594653' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi117-63923-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:34.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:34 smithi151 bash[23756]: cluster 2024-03-04T15:55:33.390291+0000 mon.a (mon.0) 1391 : cluster 0 osdmap e145: 8 total, 8 up, 8 in 2024-03-04T15:55:34.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:34 smithi151 bash[23756]: audit 2024-03-04T15:55:33.395799+0000 mon.c (mon.2) 216 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-13", "mode": "writeback"}]: dispatch 2024-03-04T15:55:34.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:34 smithi151 bash[23756]: audit 2024-03-04T15:55:33.401048+0000 mon.a (mon.0) 1392 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi117-64133-10"}]: dispatch 2024-03-04T15:55:34.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:34 smithi151 bash[23756]: audit 2024-03-04T15:55:33.403381+0000 mon.a (mon.0) 1393 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-13", "mode": "writeback"}]: dispatch 2024-03-04T15:55:34.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:34 smithi151 bash[23756]: audit 2024-03-04T15:55:33.411568+0000 mon.a (mon.0) 1394 : audit 1 from='client.? 172.21.15.117:0/602529058' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi117-63915-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:34.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:34 smithi151 bash[23756]: audit 2024-03-04T15:55:33.447336+0000 mon.a (mon.0) 1395 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-63991-24"}]: dispatch 2024-03-04T15:55:34.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:34 smithi151 bash[23756]: audit 2024-03-04T15:55:33.453739+0000 mon.a (mon.0) 1396 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-63991-24"}]: dispatch 2024-03-04T15:55:34.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:34 smithi151 bash[23756]: audit 2024-03-04T15:55:33.454606+0000 mon.a (mon.0) 1397 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-63991-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:34.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:34 smithi151 bash[23756]: cluster 2024-03-04T15:55:33.703929+0000 mgr.y (mgr.24428) 238 : cluster 0 pgmap v223: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 590 KiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-04T15:55:34.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[21480]: audit 2024-03-04T15:55:33.326281+0000 mon.a (mon.0) 1387 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:34.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[21480]: audit 2024-03-04T15:55:33.381548+0000 mon.a (mon.0) 1388 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-13"}]': finished 2024-03-04T15:55:34.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[21480]: audit 2024-03-04T15:55:33.381759+0000 mon.a (mon.0) 1389 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi117-64133-10"}]': finished 2024-03-04T15:55:34.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[21480]: audit 2024-03-04T15:55:33.381933+0000 mon.a (mon.0) 1390 : audit 1 from='client.? 172.21.15.117:0/1006594653' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi117-63923-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:34.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[21480]: cluster 2024-03-04T15:55:33.390291+0000 mon.a (mon.0) 1391 : cluster 0 osdmap e145: 8 total, 8 up, 8 in 2024-03-04T15:55:34.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[21480]: audit 2024-03-04T15:55:33.395799+0000 mon.c (mon.2) 216 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-13", "mode": "writeback"}]: dispatch 2024-03-04T15:55:34.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[21480]: audit 2024-03-04T15:55:33.401048+0000 mon.a (mon.0) 1392 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi117-64133-10"}]: dispatch 2024-03-04T15:55:34.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[21480]: audit 2024-03-04T15:55:33.403381+0000 mon.a (mon.0) 1393 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-13", "mode": "writeback"}]: dispatch 2024-03-04T15:55:34.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[21480]: audit 2024-03-04T15:55:33.411568+0000 mon.a (mon.0) 1394 : audit 1 from='client.? 172.21.15.117:0/602529058' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi117-63915-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:34.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[21480]: audit 2024-03-04T15:55:33.447336+0000 mon.a (mon.0) 1395 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-63991-24"}]: dispatch 2024-03-04T15:55:34.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[21480]: audit 2024-03-04T15:55:33.453739+0000 mon.a (mon.0) 1396 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-63991-24"}]: dispatch 2024-03-04T15:55:34.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[21480]: audit 2024-03-04T15:55:33.454606+0000 mon.a (mon.0) 1397 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-63991-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:34.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[21480]: cluster 2024-03-04T15:55:33.703929+0000 mgr.y (mgr.24428) 238 : cluster 0 pgmap v223: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 590 KiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-04T15:55:34.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[28843]: audit 2024-03-04T15:55:33.326281+0000 mon.a (mon.0) 1387 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:34.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[28843]: audit 2024-03-04T15:55:33.381548+0000 mon.a (mon.0) 1388 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-13"}]': finished 2024-03-04T15:55:34.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[28843]: audit 2024-03-04T15:55:33.381759+0000 mon.a (mon.0) 1389 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi117-64133-10"}]': finished 2024-03-04T15:55:34.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[28843]: audit 2024-03-04T15:55:33.381933+0000 mon.a (mon.0) 1390 : audit 1 from='client.? 172.21.15.117:0/1006594653' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi117-63923-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:34.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[28843]: cluster 2024-03-04T15:55:33.390291+0000 mon.a (mon.0) 1391 : cluster 0 osdmap e145: 8 total, 8 up, 8 in 2024-03-04T15:55:34.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[28843]: audit 2024-03-04T15:55:33.395799+0000 mon.c (mon.2) 216 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-13", "mode": "writeback"}]: dispatch 2024-03-04T15:55:34.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[28843]: audit 2024-03-04T15:55:33.401048+0000 mon.a (mon.0) 1392 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi117-64133-10"}]: dispatch 2024-03-04T15:55:34.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[28843]: audit 2024-03-04T15:55:33.403381+0000 mon.a (mon.0) 1393 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-13", "mode": "writeback"}]: dispatch 2024-03-04T15:55:34.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[28843]: audit 2024-03-04T15:55:33.411568+0000 mon.a (mon.0) 1394 : audit 1 from='client.? 172.21.15.117:0/602529058' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi117-63915-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:34.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[28843]: audit 2024-03-04T15:55:33.447336+0000 mon.a (mon.0) 1395 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-63991-24"}]: dispatch 2024-03-04T15:55:34.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[28843]: audit 2024-03-04T15:55:33.453739+0000 mon.a (mon.0) 1396 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-63991-24"}]: dispatch 2024-03-04T15:55:34.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[28843]: audit 2024-03-04T15:55:33.454606+0000 mon.a (mon.0) 1397 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-63991-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:34.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:34 smithi117 bash[28843]: cluster 2024-03-04T15:55:33.703929+0000 mgr.y (mgr.24428) 238 : cluster 0 pgmap v223: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 590 KiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-04T15:55:35.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:35 smithi151 bash[23756]: audit 2024-03-04T15:55:34.328525+0000 mon.a (mon.0) 1398 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:35.862 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:35 smithi151 bash[23756]: cluster 2024-03-04T15:55:34.384027+0000 mon.a (mon.0) 1399 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:55:35.862 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:35 smithi151 bash[23756]: audit 2024-03-04T15:55:34.394360+0000 mon.a (mon.0) 1400 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi117-64133-10"}]': finished 2024-03-04T15:55:35.862 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:35 smithi151 bash[23756]: audit 2024-03-04T15:55:34.394539+0000 mon.a (mon.0) 1401 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-13", "mode": "writeback"}]': finished 2024-03-04T15:55:35.862 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:35 smithi151 bash[23756]: audit 2024-03-04T15:55:34.394716+0000 mon.a (mon.0) 1402 : audit 1 from='client.? 172.21.15.117:0/602529058' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi117-63915-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:35.862 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:35 smithi151 bash[23756]: audit 2024-03-04T15:55:34.394895+0000 mon.a (mon.0) 1403 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-63991-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:55:35.862 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:35 smithi151 bash[23756]: cluster 2024-03-04T15:55:34.404309+0000 mon.a (mon.0) 1404 : cluster 0 osdmap e146: 8 total, 8 up, 8 in 2024-03-04T15:55:35.862 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:35 smithi151 bash[23756]: audit 2024-03-04T15:55:34.406835+0000 mon.a (mon.0) 1405 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-63991-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-63991-24"}]: dispatch 2024-03-04T15:55:35.862 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:35 smithi151 bash[23756]: audit 2024-03-04T15:55:34.458930+0000 mon.c (mon.2) 217 : audit 1 from='client.? 172.21.15.117:0/4029576540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:35.863 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:35 smithi151 bash[23756]: audit 2024-03-04T15:55:34.459685+0000 mon.a (mon.0) 1406 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:35.863 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:35 smithi151 bash[23756]: audit 2024-03-04T15:55:34.462046+0000 mon.c (mon.2) 218 : audit 1 from='client.? 172.21.15.117:0/4029576540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:35.863 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:35 smithi151 bash[23756]: audit 2024-03-04T15:55:34.907826+0000 mon.a (mon.0) 1407 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:35.863 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:35 smithi151 bash[23756]: audit 2024-03-04T15:55:34.909121+0000 mon.c (mon.2) 219 : audit 1 from='client.? 172.21.15.117:0/4029576540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:35.863 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:35 smithi151 bash[23756]: audit 2024-03-04T15:55:34.910378+0000 mon.a (mon.0) 1408 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:35.864 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[21480]: audit 2024-03-04T15:55:34.328525+0000 mon.a (mon.0) 1398 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:35.864 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[21480]: cluster 2024-03-04T15:55:34.384027+0000 mon.a (mon.0) 1399 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:55:35.864 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[21480]: audit 2024-03-04T15:55:34.394360+0000 mon.a (mon.0) 1400 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi117-64133-10"}]': finished 2024-03-04T15:55:35.864 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[21480]: audit 2024-03-04T15:55:34.394539+0000 mon.a (mon.0) 1401 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-13", "mode": "writeback"}]': finished 2024-03-04T15:55:35.864 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[21480]: audit 2024-03-04T15:55:34.394716+0000 mon.a (mon.0) 1402 : audit 1 from='client.? 172.21.15.117:0/602529058' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi117-63915-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:35.864 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[21480]: audit 2024-03-04T15:55:34.394895+0000 mon.a (mon.0) 1403 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-63991-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:55:35.864 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[21480]: cluster 2024-03-04T15:55:34.404309+0000 mon.a (mon.0) 1404 : cluster 0 osdmap e146: 8 total, 8 up, 8 in 2024-03-04T15:55:35.864 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[21480]: audit 2024-03-04T15:55:34.406835+0000 mon.a (mon.0) 1405 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-63991-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-63991-24"}]: dispatch 2024-03-04T15:55:35.864 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[21480]: audit 2024-03-04T15:55:34.458930+0000 mon.c (mon.2) 217 : audit 1 from='client.? 172.21.15.117:0/4029576540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:35.864 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[21480]: audit 2024-03-04T15:55:34.459685+0000 mon.a (mon.0) 1406 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:35.864 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[21480]: audit 2024-03-04T15:55:34.462046+0000 mon.c (mon.2) 218 : audit 1 from='client.? 172.21.15.117:0/4029576540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:35.864 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[21480]: audit 2024-03-04T15:55:34.907826+0000 mon.a (mon.0) 1407 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:35.864 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[21480]: audit 2024-03-04T15:55:34.909121+0000 mon.c (mon.2) 219 : audit 1 from='client.? 172.21.15.117:0/4029576540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:35.864 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[21480]: audit 2024-03-04T15:55:34.910378+0000 mon.a (mon.0) 1408 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:35.865 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[28843]: audit 2024-03-04T15:55:34.328525+0000 mon.a (mon.0) 1398 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:35.865 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[28843]: cluster 2024-03-04T15:55:34.384027+0000 mon.a (mon.0) 1399 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:55:35.865 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[28843]: audit 2024-03-04T15:55:34.394360+0000 mon.a (mon.0) 1400 : audit 1 from='client.? 172.21.15.117:0/1935312024' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi117-64133-10"}]': finished 2024-03-04T15:55:35.865 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[28843]: audit 2024-03-04T15:55:34.394539+0000 mon.a (mon.0) 1401 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-13", "mode": "writeback"}]': finished 2024-03-04T15:55:35.865 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[28843]: audit 2024-03-04T15:55:34.394716+0000 mon.a (mon.0) 1402 : audit 1 from='client.? 172.21.15.117:0/602529058' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi117-63915-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:35.865 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[28843]: audit 2024-03-04T15:55:34.394895+0000 mon.a (mon.0) 1403 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-63991-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:55:35.865 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[28843]: cluster 2024-03-04T15:55:34.404309+0000 mon.a (mon.0) 1404 : cluster 0 osdmap e146: 8 total, 8 up, 8 in 2024-03-04T15:55:35.865 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[28843]: audit 2024-03-04T15:55:34.406835+0000 mon.a (mon.0) 1405 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-63991-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-63991-24"}]: dispatch 2024-03-04T15:55:35.865 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[28843]: audit 2024-03-04T15:55:34.458930+0000 mon.c (mon.2) 217 : audit 1 from='client.? 172.21.15.117:0/4029576540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:35.865 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[28843]: audit 2024-03-04T15:55:34.459685+0000 mon.a (mon.0) 1406 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:35.865 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[28843]: audit 2024-03-04T15:55:34.462046+0000 mon.c (mon.2) 218 : audit 1 from='client.? 172.21.15.117:0/4029576540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:35.865 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[28843]: audit 2024-03-04T15:55:34.907826+0000 mon.a (mon.0) 1407 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:35.865 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[28843]: audit 2024-03-04T15:55:34.909121+0000 mon.c (mon.2) 219 : audit 1 from='client.? 172.21.15.117:0/4029576540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:35.866 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:35 smithi117 bash[28843]: audit 2024-03-04T15:55:34.910378+0000 mon.a (mon.0) 1408 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:36.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:55:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:55:36.932 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:55:36 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:55:36.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:36 smithi151 bash[23756]: audit 2024-03-04T15:55:35.330112+0000 mon.a (mon.0) 1409 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:36.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:36 smithi151 bash[23756]: audit 2024-03-04T15:55:35.398660+0000 mon.a (mon.0) 1410 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:55:36.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:36 smithi151 bash[23756]: audit 2024-03-04T15:55:35.410175+0000 mon.c (mon.2) 220 : audit 1 from='client.? 172.21.15.117:0/4029576540' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi117-64133-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:36.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:36 smithi151 bash[23756]: audit 2024-03-04T15:55:35.560951+0000 mon.c (mon.2) 221 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.0"}]: dispatch 2024-03-04T15:55:36.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:36 smithi151 bash[23756]: audit 2024-03-04T15:55:35.561641+0000 mon.c (mon.2) 222 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.1"}]: dispatch 2024-03-04T15:55:36.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:36 smithi151 bash[23756]: audit 2024-03-04T15:55:35.562191+0000 mon.c (mon.2) 223 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.2"}]: dispatch 2024-03-04T15:55:36.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:36 smithi151 bash[23756]: audit 2024-03-04T15:55:35.562740+0000 mon.c (mon.2) 224 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.3"}]: dispatch 2024-03-04T15:55:36.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:36 smithi151 bash[23756]: audit 2024-03-04T15:55:35.563287+0000 mon.c (mon.2) 225 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.4"}]: dispatch 2024-03-04T15:55:36.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:36 smithi151 bash[23756]: audit 2024-03-04T15:55:35.563808+0000 mon.c (mon.2) 226 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.5"}]: dispatch 2024-03-04T15:55:36.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:36 smithi151 bash[23756]: audit 2024-03-04T15:55:35.564347+0000 mon.c (mon.2) 227 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.6"}]: dispatch 2024-03-04T15:55:36.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:36 smithi151 bash[23756]: audit 2024-03-04T15:55:35.564950+0000 mon.c (mon.2) 228 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.7"}]: dispatch 2024-03-04T15:55:36.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:36 smithi151 bash[23756]: audit 2024-03-04T15:55:35.565537+0000 mon.c (mon.2) 229 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.8"}]: dispatch 2024-03-04T15:55:36.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:36 smithi151 bash[23756]: audit 2024-03-04T15:55:35.566229+0000 mon.c (mon.2) 230 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.9"}]: dispatch 2024-03-04T15:55:36.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:36 smithi151 bash[23756]: audit 2024-03-04T15:55:35.810788+0000 mon.c (mon.2) 231 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:55:36.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:36 smithi151 bash[23756]: cluster 2024-03-04T15:55:35.888290+0000 mon.a (mon.0) 1411 : cluster 0 osdmap e147: 8 total, 8 up, 8 in 2024-03-04T15:55:36.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:36 smithi151 bash[23756]: audit 2024-03-04T15:55:35.889711+0000 mon.c (mon.2) 232 : audit 1 from='client.? 172.21.15.117:0/1521003020' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi117-63923-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:36.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:36 smithi151 bash[23756]: audit 2024-03-04T15:55:35.891053+0000 mon.a (mon.0) 1412 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi117-64133-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:36.934 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:36 smithi151 bash[23756]: audit 2024-03-04T15:55:35.894162+0000 mon.a (mon.0) 1413 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi117-63923-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:37.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[28843]: audit 2024-03-04T15:55:35.330112+0000 mon.a (mon.0) 1409 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:37.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[28843]: audit 2024-03-04T15:55:35.398660+0000 mon.a (mon.0) 1410 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:55:37.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[28843]: audit 2024-03-04T15:55:35.410175+0000 mon.c (mon.2) 220 : audit 1 from='client.? 172.21.15.117:0/4029576540' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi117-64133-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:37.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[28843]: audit 2024-03-04T15:55:35.560951+0000 mon.c (mon.2) 221 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.0"}]: dispatch 2024-03-04T15:55:37.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[28843]: audit 2024-03-04T15:55:35.561641+0000 mon.c (mon.2) 222 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.1"}]: dispatch 2024-03-04T15:55:37.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[28843]: audit 2024-03-04T15:55:35.562191+0000 mon.c (mon.2) 223 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.2"}]: dispatch 2024-03-04T15:55:37.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[28843]: audit 2024-03-04T15:55:35.562740+0000 mon.c (mon.2) 224 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.3"}]: dispatch 2024-03-04T15:55:37.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[28843]: audit 2024-03-04T15:55:35.563287+0000 mon.c (mon.2) 225 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.4"}]: dispatch 2024-03-04T15:55:37.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[28843]: audit 2024-03-04T15:55:35.563808+0000 mon.c (mon.2) 226 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.5"}]: dispatch 2024-03-04T15:55:37.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[28843]: audit 2024-03-04T15:55:35.564347+0000 mon.c (mon.2) 227 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.6"}]: dispatch 2024-03-04T15:55:37.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[28843]: audit 2024-03-04T15:55:35.564950+0000 mon.c (mon.2) 228 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.7"}]: dispatch 2024-03-04T15:55:37.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[28843]: audit 2024-03-04T15:55:35.565537+0000 mon.c (mon.2) 229 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.8"}]: dispatch 2024-03-04T15:55:37.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[28843]: audit 2024-03-04T15:55:35.566229+0000 mon.c (mon.2) 230 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.9"}]: dispatch 2024-03-04T15:55:37.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[28843]: audit 2024-03-04T15:55:35.810788+0000 mon.c (mon.2) 231 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:55:37.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[28843]: cluster 2024-03-04T15:55:35.888290+0000 mon.a (mon.0) 1411 : cluster 0 osdmap e147: 8 total, 8 up, 8 in 2024-03-04T15:55:37.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[28843]: audit 2024-03-04T15:55:35.889711+0000 mon.c (mon.2) 232 : audit 1 from='client.? 172.21.15.117:0/1521003020' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi117-63923-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:37.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[21480]: audit 2024-03-04T15:55:35.330112+0000 mon.a (mon.0) 1409 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:37.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[21480]: audit 2024-03-04T15:55:35.398660+0000 mon.a (mon.0) 1410 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:55:37.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[21480]: audit 2024-03-04T15:55:35.410175+0000 mon.c (mon.2) 220 : audit 1 from='client.? 172.21.15.117:0/4029576540' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi117-64133-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:37.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[21480]: audit 2024-03-04T15:55:35.560951+0000 mon.c (mon.2) 221 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.0"}]: dispatch 2024-03-04T15:55:37.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[21480]: audit 2024-03-04T15:55:35.561641+0000 mon.c (mon.2) 222 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.1"}]: dispatch 2024-03-04T15:55:37.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[21480]: audit 2024-03-04T15:55:35.562191+0000 mon.c (mon.2) 223 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.2"}]: dispatch 2024-03-04T15:55:37.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[21480]: audit 2024-03-04T15:55:35.562740+0000 mon.c (mon.2) 224 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.3"}]: dispatch 2024-03-04T15:55:37.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[21480]: audit 2024-03-04T15:55:35.563287+0000 mon.c (mon.2) 225 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.4"}]: dispatch 2024-03-04T15:55:37.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[21480]: audit 2024-03-04T15:55:35.563808+0000 mon.c (mon.2) 226 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.5"}]: dispatch 2024-03-04T15:55:37.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[21480]: audit 2024-03-04T15:55:35.564347+0000 mon.c (mon.2) 227 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.6"}]: dispatch 2024-03-04T15:55:37.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[21480]: audit 2024-03-04T15:55:35.564950+0000 mon.c (mon.2) 228 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.7"}]: dispatch 2024-03-04T15:55:37.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[21480]: audit 2024-03-04T15:55:35.565537+0000 mon.c (mon.2) 229 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.8"}]: dispatch 2024-03-04T15:55:37.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[21480]: audit 2024-03-04T15:55:35.566229+0000 mon.c (mon.2) 230 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "171.9"}]: dispatch 2024-03-04T15:55:37.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[21480]: audit 2024-03-04T15:55:35.810788+0000 mon.c (mon.2) 231 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:55:37.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[21480]: cluster 2024-03-04T15:55:35.888290+0000 mon.a (mon.0) 1411 : cluster 0 osdmap e147: 8 total, 8 up, 8 in 2024-03-04T15:55:37.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[21480]: audit 2024-03-04T15:55:35.889711+0000 mon.c (mon.2) 232 : audit 1 from='client.? 172.21.15.117:0/1521003020' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi117-63923-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:37.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[21480]: audit 2024-03-04T15:55:35.891053+0000 mon.a (mon.0) 1412 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi117-64133-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:37.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[21480]: audit 2024-03-04T15:55:35.894162+0000 mon.a (mon.0) 1413 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi117-63923-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:37.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[28843]: audit 2024-03-04T15:55:35.891053+0000 mon.a (mon.0) 1412 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi117-64133-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:37.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:36 smithi117 bash[28843]: audit 2024-03-04T15:55:35.894162+0000 mon.a (mon.0) 1413 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi117-63923-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:38.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:35.561137+0000 mgr.y (mgr.24428) 239 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.0"}]: dispatch 2024-03-04T15:55:38.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:35.561776+0000 mgr.y (mgr.24428) 240 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.1"}]: dispatch 2024-03-04T15:55:38.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:35.562313+0000 mgr.y (mgr.24428) 241 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.2"}]: dispatch 2024-03-04T15:55:38.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:35.562881+0000 mgr.y (mgr.24428) 242 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.3"}]: dispatch 2024-03-04T15:55:38.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:35.563412+0000 mgr.y (mgr.24428) 243 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.4"}]: dispatch 2024-03-04T15:55:38.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:35.563939+0000 mgr.y (mgr.24428) 244 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.5"}]: dispatch 2024-03-04T15:55:38.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:35.564470+0000 mgr.y (mgr.24428) 245 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.6"}]: dispatch 2024-03-04T15:55:38.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:35.565081+0000 mgr.y (mgr.24428) 246 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.7"}]: dispatch 2024-03-04T15:55:38.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:35.565753+0000 mgr.y (mgr.24428) 247 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.8"}]: dispatch 2024-03-04T15:55:38.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:35.566363+0000 mgr.y (mgr.24428) 248 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.9"}]: dispatch 2024-03-04T15:55:38.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:35.610041+0000 osd.4 (osd.4) 7 : cluster 0 171.4 scrub starts 2024-03-04T15:55:38.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:35.615713+0000 osd.4 (osd.4) 8 : cluster 0 171.4 scrub ok 2024-03-04T15:55:38.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:35.695413+0000 osd.6 (osd.6) 7 : cluster 0 171.0 scrub starts 2024-03-04T15:55:38.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:35.699582+0000 osd.6 (osd.6) 8 : cluster 0 171.0 scrub ok 2024-03-04T15:55:38.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:35.704842+0000 mgr.y (mgr.24428) 249 : cluster 0 pgmap v226: 324 pgs: 32 unknown, 292 active+clean; 590 KiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-04T15:55:38.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:35.715471+0000 osd.3 (osd.3) 7 : cluster 0 171.6 scrub starts 2024-03-04T15:55:38.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:35.722753+0000 osd.3 (osd.3) 8 : cluster 0 171.6 scrub ok 2024-03-04T15:55:38.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:35.724837+0000 osd.0 (osd.0) 7 : cluster 0 171.9 scrub starts 2024-03-04T15:55:38.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:35.728057+0000 osd.0 (osd.0) 8 : cluster 0 171.9 scrub ok 2024-03-04T15:55:38.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:36.492497+0000 mon.c (mon.2) 233 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:55:38.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:36.542258+0000 mgr.y (mgr.24428) 250 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:55:38.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:36.567969+0000 osd.5 (osd.5) 11 : cluster 0 171.2 deep-scrub starts 2024-03-04T15:55:38.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:36.582884+0000 osd.5 (osd.5) 12 : cluster 0 171.2 deep-scrub ok 2024-03-04T15:55:38.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:36.610756+0000 osd.4 (osd.4) 9 : cluster 0 171.5 scrub starts 2024-03-04T15:55:38.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:35.561137+0000 mgr.y (mgr.24428) 239 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.0"}]: dispatch 2024-03-04T15:55:38.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:35.561776+0000 mgr.y (mgr.24428) 240 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.1"}]: dispatch 2024-03-04T15:55:38.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:35.562313+0000 mgr.y (mgr.24428) 241 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.2"}]: dispatch 2024-03-04T15:55:38.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:35.562881+0000 mgr.y (mgr.24428) 242 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.3"}]: dispatch 2024-03-04T15:55:38.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:35.563412+0000 mgr.y (mgr.24428) 243 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.4"}]: dispatch 2024-03-04T15:55:38.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:35.563939+0000 mgr.y (mgr.24428) 244 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.5"}]: dispatch 2024-03-04T15:55:38.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:35.564470+0000 mgr.y (mgr.24428) 245 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.6"}]: dispatch 2024-03-04T15:55:38.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:35.565081+0000 mgr.y (mgr.24428) 246 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.7"}]: dispatch 2024-03-04T15:55:38.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:35.565753+0000 mgr.y (mgr.24428) 247 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.8"}]: dispatch 2024-03-04T15:55:38.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:35.566363+0000 mgr.y (mgr.24428) 248 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.9"}]: dispatch 2024-03-04T15:55:38.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:35.610041+0000 osd.4 (osd.4) 7 : cluster 0 171.4 scrub starts 2024-03-04T15:55:38.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:35.615713+0000 osd.4 (osd.4) 8 : cluster 0 171.4 scrub ok 2024-03-04T15:55:38.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:35.695413+0000 osd.6 (osd.6) 7 : cluster 0 171.0 scrub starts 2024-03-04T15:55:38.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:35.699582+0000 osd.6 (osd.6) 8 : cluster 0 171.0 scrub ok 2024-03-04T15:55:38.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:35.704842+0000 mgr.y (mgr.24428) 249 : cluster 0 pgmap v226: 324 pgs: 32 unknown, 292 active+clean; 590 KiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-04T15:55:38.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:35.715471+0000 osd.3 (osd.3) 7 : cluster 0 171.6 scrub starts 2024-03-04T15:55:38.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:35.722753+0000 osd.3 (osd.3) 8 : cluster 0 171.6 scrub ok 2024-03-04T15:55:38.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:35.724837+0000 osd.0 (osd.0) 7 : cluster 0 171.9 scrub starts 2024-03-04T15:55:38.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:35.728057+0000 osd.0 (osd.0) 8 : cluster 0 171.9 scrub ok 2024-03-04T15:55:38.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:36.492497+0000 mon.c (mon.2) 233 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:55:38.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:36.542258+0000 mgr.y (mgr.24428) 250 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:55:38.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:36.567969+0000 osd.5 (osd.5) 11 : cluster 0 171.2 deep-scrub starts 2024-03-04T15:55:38.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:36.582884+0000 osd.5 (osd.5) 12 : cluster 0 171.2 deep-scrub ok 2024-03-04T15:55:38.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:36.610756+0000 osd.4 (osd.4) 9 : cluster 0 171.5 scrub starts 2024-03-04T15:55:38.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:36.617965+0000 osd.4 (osd.4) 10 : cluster 0 171.5 scrub ok 2024-03-04T15:55:38.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:36.653238+0000 osd.6 (osd.6) 9 : cluster 0 171.1 scrub starts 2024-03-04T15:55:38.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:36.658234+0000 osd.6 (osd.6) 10 : cluster 0 171.1 scrub ok 2024-03-04T15:55:38.074 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:36.748478+0000 osd.0 (osd.0) 9 : cluster 0 171.8 deep-scrub starts 2024-03-04T15:55:38.074 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:36.750740+0000 osd.0 (osd.0) 10 : cluster 0 171.8 deep-scrub ok 2024-03-04T15:55:38.074 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:36.784417+0000 mon.a (mon.0) 1414 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:38.074 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:36.786670+0000 mon.a (mon.0) 1415 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:38.074 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:36.800998+0000 mon.c (mon.2) 234 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:55:38.074 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:36.803695+0000 mon.a (mon.0) 1416 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-63991-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-63991-24"}]': finished 2024-03-04T15:55:38.074 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:36.803944+0000 mon.a (mon.0) 1417 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi117-63923-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:38.074 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:36.831749+0000 mgr.y (mgr.24428) 251 : cluster 0 pgmap v228: 396 pgs: 72 unknown, 32 creating+peering, 292 active+clean; 594 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 29 KiB/s wr, 118 op/s 2024-03-04T15:55:38.074 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:37.265620+0000 mon.a (mon.0) 1418 : cluster 0 osdmap e148: 8 total, 8 up, 8 in 2024-03-04T15:55:38.074 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:37.267183+0000 mon.c (mon.2) 235 : audit 1 from='client.? 172.21.15.117:0/128315521' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi117-63915-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:38.074 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:37.585386+0000 osd.5 (osd.5) 13 : cluster 0 171.3 scrub starts 2024-03-04T15:55:38.074 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:37.591485+0000 osd.5 (osd.5) 14 : cluster 0 171.3 scrub ok 2024-03-04T15:55:38.074 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:37.688938+0000 mon.a (mon.0) 1419 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:55:38.074 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:37.694693+0000 mon.a (mon.0) 1420 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi117-64133-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]': finished 2024-03-04T15:55:38.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: cluster 2024-03-04T15:55:37.705654+0000 mon.a (mon.0) 1421 : cluster 0 osdmap e149: 8 total, 8 up, 8 in 2024-03-04T15:55:38.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:37.717094+0000 mon.a (mon.0) 1422 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63991-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:38.075 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[28843]: audit 2024-03-04T15:55:37.718360+0000 mon.a (mon.0) 1423 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi117-63915-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:38.075 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:36.617965+0000 osd.4 (osd.4) 10 : cluster 0 171.5 scrub ok 2024-03-04T15:55:38.075 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:36.653238+0000 osd.6 (osd.6) 9 : cluster 0 171.1 scrub starts 2024-03-04T15:55:38.075 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:36.658234+0000 osd.6 (osd.6) 10 : cluster 0 171.1 scrub ok 2024-03-04T15:55:38.075 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:36.748478+0000 osd.0 (osd.0) 9 : cluster 0 171.8 deep-scrub starts 2024-03-04T15:55:38.075 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:36.750740+0000 osd.0 (osd.0) 10 : cluster 0 171.8 deep-scrub ok 2024-03-04T15:55:38.075 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:36.784417+0000 mon.a (mon.0) 1414 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:38.075 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:36.786670+0000 mon.a (mon.0) 1415 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:38.075 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:36.800998+0000 mon.c (mon.2) 234 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:55:38.075 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:36.803695+0000 mon.a (mon.0) 1416 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-63991-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-63991-24"}]': finished 2024-03-04T15:55:38.076 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:36.803944+0000 mon.a (mon.0) 1417 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi117-63923-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:38.076 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:36.831749+0000 mgr.y (mgr.24428) 251 : cluster 0 pgmap v228: 396 pgs: 72 unknown, 32 creating+peering, 292 active+clean; 594 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 29 KiB/s wr, 118 op/s 2024-03-04T15:55:38.076 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:37.265620+0000 mon.a (mon.0) 1418 : cluster 0 osdmap e148: 8 total, 8 up, 8 in 2024-03-04T15:55:38.076 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:37.267183+0000 mon.c (mon.2) 235 : audit 1 from='client.? 172.21.15.117:0/128315521' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi117-63915-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:38.076 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:37.585386+0000 osd.5 (osd.5) 13 : cluster 0 171.3 scrub starts 2024-03-04T15:55:38.076 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:37.591485+0000 osd.5 (osd.5) 14 : cluster 0 171.3 scrub ok 2024-03-04T15:55:38.076 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:37.688938+0000 mon.a (mon.0) 1419 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:55:38.076 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:37.694693+0000 mon.a (mon.0) 1420 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi117-64133-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]': finished 2024-03-04T15:55:38.076 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: cluster 2024-03-04T15:55:37.705654+0000 mon.a (mon.0) 1421 : cluster 0 osdmap e149: 8 total, 8 up, 8 in 2024-03-04T15:55:38.076 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:37.717094+0000 mon.a (mon.0) 1422 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63991-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:38.076 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:37 smithi117 bash[21480]: audit 2024-03-04T15:55:37.718360+0000 mon.a (mon.0) 1423 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi117-63915-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:38.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:35.561137+0000 mgr.y (mgr.24428) 239 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.0"}]: dispatch 2024-03-04T15:55:38.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:35.561776+0000 mgr.y (mgr.24428) 240 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.1"}]: dispatch 2024-03-04T15:55:38.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:35.562313+0000 mgr.y (mgr.24428) 241 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.2"}]: dispatch 2024-03-04T15:55:38.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:35.562881+0000 mgr.y (mgr.24428) 242 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.3"}]: dispatch 2024-03-04T15:55:38.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:35.563412+0000 mgr.y (mgr.24428) 243 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.4"}]: dispatch 2024-03-04T15:55:38.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:35.563939+0000 mgr.y (mgr.24428) 244 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.5"}]: dispatch 2024-03-04T15:55:38.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:35.564470+0000 mgr.y (mgr.24428) 245 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.6"}]: dispatch 2024-03-04T15:55:38.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:35.565081+0000 mgr.y (mgr.24428) 246 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.7"}]: dispatch 2024-03-04T15:55:38.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:35.565753+0000 mgr.y (mgr.24428) 247 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.8"}]: dispatch 2024-03-04T15:55:38.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:35.566363+0000 mgr.y (mgr.24428) 248 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "171.9"}]: dispatch 2024-03-04T15:55:38.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:35.610041+0000 osd.4 (osd.4) 7 : cluster 0 171.4 scrub starts 2024-03-04T15:55:38.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:35.615713+0000 osd.4 (osd.4) 8 : cluster 0 171.4 scrub ok 2024-03-04T15:55:38.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:35.695413+0000 osd.6 (osd.6) 7 : cluster 0 171.0 scrub starts 2024-03-04T15:55:38.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:35.699582+0000 osd.6 (osd.6) 8 : cluster 0 171.0 scrub ok 2024-03-04T15:55:38.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:35.704842+0000 mgr.y (mgr.24428) 249 : cluster 0 pgmap v226: 324 pgs: 32 unknown, 292 active+clean; 590 KiB data, 1.1 GiB used, 714 GiB / 715 GiB avail 2024-03-04T15:55:38.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:35.715471+0000 osd.3 (osd.3) 7 : cluster 0 171.6 scrub starts 2024-03-04T15:55:38.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:35.722753+0000 osd.3 (osd.3) 8 : cluster 0 171.6 scrub ok 2024-03-04T15:55:38.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:35.724837+0000 osd.0 (osd.0) 7 : cluster 0 171.9 scrub starts 2024-03-04T15:55:38.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:35.728057+0000 osd.0 (osd.0) 8 : cluster 0 171.9 scrub ok 2024-03-04T15:55:38.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:36.492497+0000 mon.c (mon.2) 233 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:55:38.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:36.542258+0000 mgr.y (mgr.24428) 250 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:55:38.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:36.567969+0000 osd.5 (osd.5) 11 : cluster 0 171.2 deep-scrub starts 2024-03-04T15:55:38.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:36.582884+0000 osd.5 (osd.5) 12 : cluster 0 171.2 deep-scrub ok 2024-03-04T15:55:38.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:36.610756+0000 osd.4 (osd.4) 9 : cluster 0 171.5 scrub starts 2024-03-04T15:55:38.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:36.617965+0000 osd.4 (osd.4) 10 : cluster 0 171.5 scrub ok 2024-03-04T15:55:38.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:36.653238+0000 osd.6 (osd.6) 9 : cluster 0 171.1 scrub starts 2024-03-04T15:55:38.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:36.658234+0000 osd.6 (osd.6) 10 : cluster 0 171.1 scrub ok 2024-03-04T15:55:38.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:36.748478+0000 osd.0 (osd.0) 9 : cluster 0 171.8 deep-scrub starts 2024-03-04T15:55:38.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:36.750740+0000 osd.0 (osd.0) 10 : cluster 0 171.8 deep-scrub ok 2024-03-04T15:55:38.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:36.784417+0000 mon.a (mon.0) 1414 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:38.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:36.786670+0000 mon.a (mon.0) 1415 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:38.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:36.800998+0000 mon.c (mon.2) 234 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:55:38.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:36.803695+0000 mon.a (mon.0) 1416 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-63991-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-63991-24"}]': finished 2024-03-04T15:55:38.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:36.803944+0000 mon.a (mon.0) 1417 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi117-63923-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:38.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:36.831749+0000 mgr.y (mgr.24428) 251 : cluster 0 pgmap v228: 396 pgs: 72 unknown, 32 creating+peering, 292 active+clean; 594 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 29 KiB/s wr, 118 op/s 2024-03-04T15:55:38.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:37.265620+0000 mon.a (mon.0) 1418 : cluster 0 osdmap e148: 8 total, 8 up, 8 in 2024-03-04T15:55:38.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:37.267183+0000 mon.c (mon.2) 235 : audit 1 from='client.? 172.21.15.117:0/128315521' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi117-63915-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:38.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:37.585386+0000 osd.5 (osd.5) 13 : cluster 0 171.3 scrub starts 2024-03-04T15:55:38.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:37.591485+0000 osd.5 (osd.5) 14 : cluster 0 171.3 scrub ok 2024-03-04T15:55:38.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:37.688938+0000 mon.a (mon.0) 1419 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:55:38.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:37.694693+0000 mon.a (mon.0) 1420 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi117-64133-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]': finished 2024-03-04T15:55:38.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: cluster 2024-03-04T15:55:37.705654+0000 mon.a (mon.0) 1421 : cluster 0 osdmap e149: 8 total, 8 up, 8 in 2024-03-04T15:55:38.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:37.717094+0000 mon.a (mon.0) 1422 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63991-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:38.186 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:37 smithi151 bash[23756]: audit 2024-03-04T15:55:37.718360+0000 mon.a (mon.0) 1423 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi117-63915-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:39.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:39 smithi117 bash[21480]: cluster 2024-03-04T15:55:37.604456+0000 osd.6 (osd.6) 11 : cluster 0 171.7 scrub starts 2024-03-04T15:55:39.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:39 smithi117 bash[21480]: cluster 2024-03-04T15:55:37.609681+0000 osd.6 (osd.6) 12 : cluster 0 171.7 scrub ok 2024-03-04T15:55:39.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:39 smithi117 bash[21480]: audit 2024-03-04T15:55:37.787936+0000 mon.a (mon.0) 1424 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:39.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:39 smithi117 bash[21480]: cluster 2024-03-04T15:55:38.686469+0000 mon.a (mon.0) 1425 : cluster 1 Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-03-04T15:55:39.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:39 smithi117 bash[21480]: audit 2024-03-04T15:55:38.700557+0000 mon.a (mon.0) 1426 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63991-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:39.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:39 smithi117 bash[21480]: audit 2024-03-04T15:55:38.700718+0000 mon.a (mon.0) 1427 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi117-63915-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:39.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:39 smithi117 bash[21480]: cluster 2024-03-04T15:55:38.709061+0000 mon.a (mon.0) 1428 : cluster 0 osdmap e150: 8 total, 8 up, 8 in 2024-03-04T15:55:39.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:39 smithi117 bash[21480]: audit 2024-03-04T15:55:38.716371+0000 mon.a (mon.0) 1429 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63991-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:39.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:39 smithi117 bash[28843]: cluster 2024-03-04T15:55:37.604456+0000 osd.6 (osd.6) 11 : cluster 0 171.7 scrub starts 2024-03-04T15:55:39.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:39 smithi117 bash[28843]: cluster 2024-03-04T15:55:37.609681+0000 osd.6 (osd.6) 12 : cluster 0 171.7 scrub ok 2024-03-04T15:55:39.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:39 smithi117 bash[28843]: audit 2024-03-04T15:55:37.787936+0000 mon.a (mon.0) 1424 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:39.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:39 smithi117 bash[28843]: cluster 2024-03-04T15:55:38.686469+0000 mon.a (mon.0) 1425 : cluster 1 Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-03-04T15:55:39.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:39 smithi117 bash[28843]: audit 2024-03-04T15:55:38.700557+0000 mon.a (mon.0) 1426 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63991-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:39.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:39 smithi117 bash[28843]: audit 2024-03-04T15:55:38.700718+0000 mon.a (mon.0) 1427 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi117-63915-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:39.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:39 smithi117 bash[28843]: cluster 2024-03-04T15:55:38.709061+0000 mon.a (mon.0) 1428 : cluster 0 osdmap e150: 8 total, 8 up, 8 in 2024-03-04T15:55:39.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:39 smithi117 bash[28843]: audit 2024-03-04T15:55:38.716371+0000 mon.a (mon.0) 1429 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63991-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:39.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:39 smithi151 bash[23756]: cluster 2024-03-04T15:55:37.604456+0000 osd.6 (osd.6) 11 : cluster 0 171.7 scrub starts 2024-03-04T15:55:39.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:39 smithi151 bash[23756]: cluster 2024-03-04T15:55:37.609681+0000 osd.6 (osd.6) 12 : cluster 0 171.7 scrub ok 2024-03-04T15:55:39.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:39 smithi151 bash[23756]: audit 2024-03-04T15:55:37.787936+0000 mon.a (mon.0) 1424 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:39 smithi151 bash[23756]: cluster 2024-03-04T15:55:38.686469+0000 mon.a (mon.0) 1425 : cluster 1 Health check cleared: CEPHADM_STRAY_DAEMON (was: 1 stray daemon(s) not managed by cephadm) 2024-03-04T15:55:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:39 smithi151 bash[23756]: audit 2024-03-04T15:55:38.700557+0000 mon.a (mon.0) 1426 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63991-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:39 smithi151 bash[23756]: audit 2024-03-04T15:55:38.700718+0000 mon.a (mon.0) 1427 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi117-63915-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:39 smithi151 bash[23756]: cluster 2024-03-04T15:55:38.709061+0000 mon.a (mon.0) 1428 : cluster 0 osdmap e150: 8 total, 8 up, 8 in 2024-03-04T15:55:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:39 smithi151 bash[23756]: audit 2024-03-04T15:55:38.716371+0000 mon.a (mon.0) 1429 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63991-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:40.935 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:40 smithi151 bash[23756]: cluster 2024-03-04T15:55:38.832358+0000 mgr.y (mgr.24428) 252 : cluster 0 pgmap v231: 404 pgs: 112 unknown, 292 active+clean; 594 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 29 KiB/s wr, 118 op/s 2024-03-04T15:55:40.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:40 smithi151 bash[23756]: audit 2024-03-04T15:55:39.207608+0000 mon.a (mon.0) 1430 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:40.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:40 smithi151 bash[23756]: audit 2024-03-04T15:55:39.706273+0000 mon.a (mon.0) 1431 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63991-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:40.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:40 smithi151 bash[23756]: cluster 2024-03-04T15:55:39.714412+0000 mon.a (mon.0) 1432 : cluster 0 osdmap e151: 8 total, 8 up, 8 in 2024-03-04T15:55:40.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:40 smithi151 bash[23756]: audit 2024-03-04T15:55:39.717400+0000 mon.b (mon.1) 72 : audit 1 from='client.? 172.21.15.117:0/2199135005' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi117-63923-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:40.936 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:40 smithi151 bash[23756]: audit 2024-03-04T15:55:39.721441+0000 mon.a (mon.0) 1433 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi117-63923-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:41.080 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:40 smithi117 bash[28843]: cluster 2024-03-04T15:55:38.832358+0000 mgr.y (mgr.24428) 252 : cluster 0 pgmap v231: 404 pgs: 112 unknown, 292 active+clean; 594 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 29 KiB/s wr, 118 op/s 2024-03-04T15:55:41.080 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:40 smithi117 bash[28843]: audit 2024-03-04T15:55:39.207608+0000 mon.a (mon.0) 1430 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:41.081 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:40 smithi117 bash[28843]: audit 2024-03-04T15:55:39.706273+0000 mon.a (mon.0) 1431 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63991-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:41.081 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:40 smithi117 bash[28843]: cluster 2024-03-04T15:55:39.714412+0000 mon.a (mon.0) 1432 : cluster 0 osdmap e151: 8 total, 8 up, 8 in 2024-03-04T15:55:41.081 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:40 smithi117 bash[28843]: audit 2024-03-04T15:55:39.717400+0000 mon.b (mon.1) 72 : audit 1 from='client.? 172.21.15.117:0/2199135005' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi117-63923-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:41.081 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:40 smithi117 bash[28843]: audit 2024-03-04T15:55:39.721441+0000 mon.a (mon.0) 1433 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi117-63923-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:41.081 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:40 smithi117 bash[21480]: cluster 2024-03-04T15:55:38.832358+0000 mgr.y (mgr.24428) 252 : cluster 0 pgmap v231: 404 pgs: 112 unknown, 292 active+clean; 594 KiB data, 583 MiB used, 715 GiB / 715 GiB avail; 66 KiB/s rd, 29 KiB/s wr, 118 op/s 2024-03-04T15:55:41.081 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:40 smithi117 bash[21480]: audit 2024-03-04T15:55:39.207608+0000 mon.a (mon.0) 1430 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:41.081 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:40 smithi117 bash[21480]: audit 2024-03-04T15:55:39.706273+0000 mon.a (mon.0) 1431 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63991-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:41.081 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:40 smithi117 bash[21480]: cluster 2024-03-04T15:55:39.714412+0000 mon.a (mon.0) 1432 : cluster 0 osdmap e151: 8 total, 8 up, 8 in 2024-03-04T15:55:41.081 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:40 smithi117 bash[21480]: audit 2024-03-04T15:55:39.717400+0000 mon.b (mon.1) 72 : audit 1 from='client.? 172.21.15.117:0/2199135005' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi117-63923-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:41.081 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:40 smithi117 bash[21480]: audit 2024-03-04T15:55:39.721441+0000 mon.a (mon.0) 1433 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi117-63923-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:41.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:41 smithi151 bash[23756]: audit 2024-03-04T15:55:40.612651+0000 mon.a (mon.0) 1434 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:41.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:41 smithi151 bash[23756]: audit 2024-03-04T15:55:40.710373+0000 mon.a (mon.0) 1435 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi117-63923-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:41.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:41 smithi151 bash[23756]: cluster 2024-03-04T15:55:40.719005+0000 mon.a (mon.0) 1436 : cluster 0 osdmap e152: 8 total, 8 up, 8 in 2024-03-04T15:55:41.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:41 smithi151 bash[23756]: audit 2024-03-04T15:55:40.724238+0000 mon.c (mon.2) 236 : audit 1 from='client.? 172.21.15.117:0/3978140023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi117-63915-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:41.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:41 smithi151 bash[23756]: audit 2024-03-04T15:55:40.724971+0000 mon.a (mon.0) 1437 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi117-63915-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:41.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:41 smithi151 bash[23756]: cluster 2024-03-04T15:55:40.832920+0000 mgr.y (mgr.24428) 253 : cluster 0 pgmap v234: 404 pgs: 112 unknown, 292 active+clean; 594 KiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:55:42.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:41 smithi117 bash[21480]: audit 2024-03-04T15:55:40.612651+0000 mon.a (mon.0) 1434 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:42.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:41 smithi117 bash[21480]: audit 2024-03-04T15:55:40.710373+0000 mon.a (mon.0) 1435 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi117-63923-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:42.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:41 smithi117 bash[21480]: cluster 2024-03-04T15:55:40.719005+0000 mon.a (mon.0) 1436 : cluster 0 osdmap e152: 8 total, 8 up, 8 in 2024-03-04T15:55:42.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:41 smithi117 bash[21480]: audit 2024-03-04T15:55:40.724238+0000 mon.c (mon.2) 236 : audit 1 from='client.? 172.21.15.117:0/3978140023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi117-63915-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:42.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:41 smithi117 bash[21480]: audit 2024-03-04T15:55:40.724971+0000 mon.a (mon.0) 1437 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi117-63915-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:42.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:41 smithi117 bash[21480]: cluster 2024-03-04T15:55:40.832920+0000 mgr.y (mgr.24428) 253 : cluster 0 pgmap v234: 404 pgs: 112 unknown, 292 active+clean; 594 KiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:55:42.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:41 smithi117 bash[28843]: audit 2024-03-04T15:55:40.612651+0000 mon.a (mon.0) 1434 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:42.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:41 smithi117 bash[28843]: audit 2024-03-04T15:55:40.710373+0000 mon.a (mon.0) 1435 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi117-63923-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:42.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:41 smithi117 bash[28843]: cluster 2024-03-04T15:55:40.719005+0000 mon.a (mon.0) 1436 : cluster 0 osdmap e152: 8 total, 8 up, 8 in 2024-03-04T15:55:42.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:41 smithi117 bash[28843]: audit 2024-03-04T15:55:40.724238+0000 mon.c (mon.2) 236 : audit 1 from='client.? 172.21.15.117:0/3978140023' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi117-63915-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:42.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:41 smithi117 bash[28843]: audit 2024-03-04T15:55:40.724971+0000 mon.a (mon.0) 1437 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi117-63915-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:42.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:41 smithi117 bash[28843]: cluster 2024-03-04T15:55:40.832920+0000 mgr.y (mgr.24428) 253 : cluster 0 pgmap v234: 404 pgs: 112 unknown, 292 active+clean; 594 KiB data, 583 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:55:42.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:42 smithi151 bash[23756]: audit 2024-03-04T15:55:41.614264+0000 mon.a (mon.0) 1438 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:42.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:42 smithi151 bash[23756]: audit 2024-03-04T15:55:41.714974+0000 mon.a (mon.0) 1439 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi117-63915-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:42.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:42 smithi151 bash[23756]: cluster 2024-03-04T15:55:41.724525+0000 mon.a (mon.0) 1440 : cluster 0 osdmap e153: 8 total, 8 up, 8 in 2024-03-04T15:55:42.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:42 smithi151 bash[23756]: audit 2024-03-04T15:55:41.728889+0000 mon.a (mon.0) 1441 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-63991-24"}]: dispatch 2024-03-04T15:55:43.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:42 smithi117 bash[21480]: audit 2024-03-04T15:55:41.614264+0000 mon.a (mon.0) 1438 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:43.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:42 smithi117 bash[21480]: audit 2024-03-04T15:55:41.714974+0000 mon.a (mon.0) 1439 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi117-63915-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:43.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:42 smithi117 bash[21480]: cluster 2024-03-04T15:55:41.724525+0000 mon.a (mon.0) 1440 : cluster 0 osdmap e153: 8 total, 8 up, 8 in 2024-03-04T15:55:43.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:42 smithi117 bash[21480]: audit 2024-03-04T15:55:41.728889+0000 mon.a (mon.0) 1441 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-63991-24"}]: dispatch 2024-03-04T15:55:43.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:42 smithi117 bash[28843]: audit 2024-03-04T15:55:41.614264+0000 mon.a (mon.0) 1438 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:43.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:42 smithi117 bash[28843]: audit 2024-03-04T15:55:41.714974+0000 mon.a (mon.0) 1439 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi117-63915-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:43.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:42 smithi117 bash[28843]: cluster 2024-03-04T15:55:41.724525+0000 mon.a (mon.0) 1440 : cluster 0 osdmap e153: 8 total, 8 up, 8 in 2024-03-04T15:55:43.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:42 smithi117 bash[28843]: audit 2024-03-04T15:55:41.728889+0000 mon.a (mon.0) 1441 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-63991-24"}]: dispatch 2024-03-04T15:55:43.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:43 smithi151 bash[23756]: audit 2024-03-04T15:55:42.656349+0000 mon.a (mon.0) 1442 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:43.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:43 smithi151 bash[23756]: cluster 2024-03-04T15:55:42.685968+0000 mon.a (mon.0) 1443 : cluster 3 Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:43.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:43 smithi151 bash[23756]: audit 2024-03-04T15:55:42.719920+0000 mon.a (mon.0) 1444 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-63991-24"}]': finished 2024-03-04T15:55:43.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:43 smithi151 bash[23756]: cluster 2024-03-04T15:55:42.728016+0000 mon.a (mon.0) 1445 : cluster 0 osdmap e154: 8 total, 8 up, 8 in 2024-03-04T15:55:43.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:43 smithi151 bash[23756]: audit 2024-03-04T15:55:42.728993+0000 mon.a (mon.0) 1446 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-63991-24"}]: dispatch 2024-03-04T15:55:43.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:43 smithi151 bash[23756]: audit 2024-03-04T15:55:42.732056+0000 mon.c (mon.2) 237 : audit 1 from='client.? 172.21.15.117:0/1929430498' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64156-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:43.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:43 smithi151 bash[23756]: audit 2024-03-04T15:55:42.741018+0000 mon.a (mon.0) 1447 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64156-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:43.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:43 smithi151 bash[23756]: audit 2024-03-04T15:55:42.742714+0000 mon.c (mon.2) 238 : audit 1 from='client.? 172.21.15.117:0/2937549320' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi117-63923-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:43.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:43 smithi151 bash[23756]: audit 2024-03-04T15:55:42.743837+0000 mon.a (mon.0) 1448 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi117-63923-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:43.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:43 smithi151 bash[23756]: cluster 2024-03-04T15:55:42.833415+0000 mgr.y (mgr.24428) 254 : cluster 0 pgmap v237: 364 pgs: 64 unknown, 32 creating+peering, 268 active+clean; 608 KiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:55:43.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:43 smithi151 bash[23756]: audit 2024-03-04T15:55:43.212101+0000 mon.a (mon.0) 1449 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:55:43.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:43 smithi151 bash[23756]: audit 2024-03-04T15:55:43.214032+0000 mon.c (mon.2) 239 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:55:44.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[21480]: audit 2024-03-04T15:55:42.656349+0000 mon.a (mon.0) 1442 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:44.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[21480]: cluster 2024-03-04T15:55:42.685968+0000 mon.a (mon.0) 1443 : cluster 3 Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:44.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[21480]: audit 2024-03-04T15:55:42.719920+0000 mon.a (mon.0) 1444 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-63991-24"}]': finished 2024-03-04T15:55:44.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[21480]: cluster 2024-03-04T15:55:42.728016+0000 mon.a (mon.0) 1445 : cluster 0 osdmap e154: 8 total, 8 up, 8 in 2024-03-04T15:55:44.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[21480]: audit 2024-03-04T15:55:42.728993+0000 mon.a (mon.0) 1446 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-63991-24"}]: dispatch 2024-03-04T15:55:44.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[21480]: audit 2024-03-04T15:55:42.732056+0000 mon.c (mon.2) 237 : audit 1 from='client.? 172.21.15.117:0/1929430498' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64156-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:44.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[21480]: audit 2024-03-04T15:55:42.741018+0000 mon.a (mon.0) 1447 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64156-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:44.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[21480]: audit 2024-03-04T15:55:42.742714+0000 mon.c (mon.2) 238 : audit 1 from='client.? 172.21.15.117:0/2937549320' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi117-63923-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:44.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[21480]: audit 2024-03-04T15:55:42.743837+0000 mon.a (mon.0) 1448 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi117-63923-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:44.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[21480]: cluster 2024-03-04T15:55:42.833415+0000 mgr.y (mgr.24428) 254 : cluster 0 pgmap v237: 364 pgs: 64 unknown, 32 creating+peering, 268 active+clean; 608 KiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:55:44.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[21480]: audit 2024-03-04T15:55:43.212101+0000 mon.a (mon.0) 1449 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:55:44.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[21480]: audit 2024-03-04T15:55:43.214032+0000 mon.c (mon.2) 239 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:55:44.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[28843]: audit 2024-03-04T15:55:42.656349+0000 mon.a (mon.0) 1442 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:44.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[28843]: cluster 2024-03-04T15:55:42.685968+0000 mon.a (mon.0) 1443 : cluster 3 Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:44.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[28843]: audit 2024-03-04T15:55:42.719920+0000 mon.a (mon.0) 1444 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-63991-24"}]': finished 2024-03-04T15:55:44.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[28843]: cluster 2024-03-04T15:55:42.728016+0000 mon.a (mon.0) 1445 : cluster 0 osdmap e154: 8 total, 8 up, 8 in 2024-03-04T15:55:44.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[28843]: audit 2024-03-04T15:55:42.728993+0000 mon.a (mon.0) 1446 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-63991-24"}]: dispatch 2024-03-04T15:55:44.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[28843]: audit 2024-03-04T15:55:42.732056+0000 mon.c (mon.2) 237 : audit 1 from='client.? 172.21.15.117:0/1929430498' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64156-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:44.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[28843]: audit 2024-03-04T15:55:42.741018+0000 mon.a (mon.0) 1447 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64156-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:44.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[28843]: audit 2024-03-04T15:55:42.742714+0000 mon.c (mon.2) 238 : audit 1 from='client.? 172.21.15.117:0/2937549320' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi117-63923-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:44.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[28843]: audit 2024-03-04T15:55:42.743837+0000 mon.a (mon.0) 1448 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi117-63923-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:44.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[28843]: cluster 2024-03-04T15:55:42.833415+0000 mgr.y (mgr.24428) 254 : cluster 0 pgmap v237: 364 pgs: 64 unknown, 32 creating+peering, 268 active+clean; 608 KiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:55:44.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[28843]: audit 2024-03-04T15:55:43.212101+0000 mon.a (mon.0) 1449 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:55:44.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:43 smithi117 bash[28843]: audit 2024-03-04T15:55:43.214032+0000 mon.c (mon.2) 239 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:55:44.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:44 smithi151 bash[23756]: audit 2024-03-04T15:55:43.658181+0000 mon.a (mon.0) 1450 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:44.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:44 smithi151 bash[23756]: audit 2024-03-04T15:55:43.733259+0000 mon.a (mon.0) 1451 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-63991-24"}]': finished 2024-03-04T15:55:44.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:44 smithi151 bash[23756]: audit 2024-03-04T15:55:43.733393+0000 mon.a (mon.0) 1452 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64156-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:44.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:44 smithi151 bash[23756]: audit 2024-03-04T15:55:43.733629+0000 mon.a (mon.0) 1453 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi117-63923-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:44.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:44 smithi151 bash[23756]: cluster 2024-03-04T15:55:43.741384+0000 mon.a (mon.0) 1454 : cluster 0 osdmap e155: 8 total, 8 up, 8 in 2024-03-04T15:55:44.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:44 smithi151 bash[23756]: audit 2024-03-04T15:55:43.759180+0000 mon.a (mon.0) 1455 : audit 1 from='client.? 172.21.15.117:0/729776569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi117-63915-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:44.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:44 smithi151 bash[23756]: audit 2024-03-04T15:55:44.660188+0000 mon.a (mon.0) 1456 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:45.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:44 smithi117 bash[21480]: audit 2024-03-04T15:55:43.658181+0000 mon.a (mon.0) 1450 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:45.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:44 smithi117 bash[21480]: audit 2024-03-04T15:55:43.733259+0000 mon.a (mon.0) 1451 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-63991-24"}]': finished 2024-03-04T15:55:45.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:44 smithi117 bash[21480]: audit 2024-03-04T15:55:43.733393+0000 mon.a (mon.0) 1452 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64156-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:45.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:44 smithi117 bash[21480]: audit 2024-03-04T15:55:43.733629+0000 mon.a (mon.0) 1453 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi117-63923-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:45.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:44 smithi117 bash[21480]: cluster 2024-03-04T15:55:43.741384+0000 mon.a (mon.0) 1454 : cluster 0 osdmap e155: 8 total, 8 up, 8 in 2024-03-04T15:55:45.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:44 smithi117 bash[21480]: audit 2024-03-04T15:55:43.759180+0000 mon.a (mon.0) 1455 : audit 1 from='client.? 172.21.15.117:0/729776569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi117-63915-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:45.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:44 smithi117 bash[21480]: audit 2024-03-04T15:55:44.660188+0000 mon.a (mon.0) 1456 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:45.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:44 smithi117 bash[28843]: audit 2024-03-04T15:55:43.658181+0000 mon.a (mon.0) 1450 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:45.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:44 smithi117 bash[28843]: audit 2024-03-04T15:55:43.733259+0000 mon.a (mon.0) 1451 : audit 1 from='client.? 172.21.15.117:0/480252417' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-63991-24"}]': finished 2024-03-04T15:55:45.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:44 smithi117 bash[28843]: audit 2024-03-04T15:55:43.733393+0000 mon.a (mon.0) 1452 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64156-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:45.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:44 smithi117 bash[28843]: audit 2024-03-04T15:55:43.733629+0000 mon.a (mon.0) 1453 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi117-63923-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:45.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:44 smithi117 bash[28843]: cluster 2024-03-04T15:55:43.741384+0000 mon.a (mon.0) 1454 : cluster 0 osdmap e155: 8 total, 8 up, 8 in 2024-03-04T15:55:45.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:44 smithi117 bash[28843]: audit 2024-03-04T15:55:43.759180+0000 mon.a (mon.0) 1455 : audit 1 from='client.? 172.21.15.117:0/729776569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi117-63915-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:45.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:44 smithi117 bash[28843]: audit 2024-03-04T15:55:44.660188+0000 mon.a (mon.0) 1456 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:46.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:45 smithi117 bash[21480]: audit 2024-03-04T15:55:44.737601+0000 mon.a (mon.0) 1457 : audit 1 from='client.? 172.21.15.117:0/729776569' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi117-63915-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:46.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:45 smithi117 bash[21480]: cluster 2024-03-04T15:55:44.748153+0000 mon.a (mon.0) 1458 : cluster 0 osdmap e156: 8 total, 8 up, 8 in 2024-03-04T15:55:46.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:45 smithi117 bash[21480]: audit 2024-03-04T15:55:44.753518+0000 mon.c (mon.2) 240 : audit 1 from='client.? 172.21.15.117:0/2109362381' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi117-63991-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:46.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:45 smithi117 bash[21480]: audit 2024-03-04T15:55:44.790383+0000 mon.a (mon.0) 1459 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi117-63991-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:46.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:45 smithi117 bash[21480]: cluster 2024-03-04T15:55:44.833899+0000 mgr.y (mgr.24428) 255 : cluster 0 pgmap v240: 396 pgs: 96 unknown, 32 creating+peering, 268 active+clean; 608 KiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:55:46.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:45 smithi117 bash[21480]: audit 2024-03-04T15:55:45.662090+0000 mon.a (mon.0) 1460 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:46.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:45 smithi117 bash[28843]: audit 2024-03-04T15:55:44.737601+0000 mon.a (mon.0) 1457 : audit 1 from='client.? 172.21.15.117:0/729776569' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi117-63915-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:46.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:45 smithi117 bash[28843]: cluster 2024-03-04T15:55:44.748153+0000 mon.a (mon.0) 1458 : cluster 0 osdmap e156: 8 total, 8 up, 8 in 2024-03-04T15:55:46.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:45 smithi117 bash[28843]: audit 2024-03-04T15:55:44.753518+0000 mon.c (mon.2) 240 : audit 1 from='client.? 172.21.15.117:0/2109362381' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi117-63991-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:46.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:45 smithi117 bash[28843]: audit 2024-03-04T15:55:44.790383+0000 mon.a (mon.0) 1459 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi117-63991-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:46.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:45 smithi117 bash[28843]: cluster 2024-03-04T15:55:44.833899+0000 mgr.y (mgr.24428) 255 : cluster 0 pgmap v240: 396 pgs: 96 unknown, 32 creating+peering, 268 active+clean; 608 KiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:55:46.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:45 smithi117 bash[28843]: audit 2024-03-04T15:55:45.662090+0000 mon.a (mon.0) 1460 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:46.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:45 smithi151 bash[23756]: audit 2024-03-04T15:55:44.737601+0000 mon.a (mon.0) 1457 : audit 1 from='client.? 172.21.15.117:0/729776569' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi117-63915-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:46.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:45 smithi151 bash[23756]: cluster 2024-03-04T15:55:44.748153+0000 mon.a (mon.0) 1458 : cluster 0 osdmap e156: 8 total, 8 up, 8 in 2024-03-04T15:55:46.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:45 smithi151 bash[23756]: audit 2024-03-04T15:55:44.753518+0000 mon.c (mon.2) 240 : audit 1 from='client.? 172.21.15.117:0/2109362381' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi117-63991-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:46.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:45 smithi151 bash[23756]: audit 2024-03-04T15:55:44.790383+0000 mon.a (mon.0) 1459 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi117-63991-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:46.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:45 smithi151 bash[23756]: cluster 2024-03-04T15:55:44.833899+0000 mgr.y (mgr.24428) 255 : cluster 0 pgmap v240: 396 pgs: 96 unknown, 32 creating+peering, 268 active+clean; 608 KiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:55:46.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:45 smithi151 bash[23756]: audit 2024-03-04T15:55:45.662090+0000 mon.a (mon.0) 1460 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:46.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:55:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:55:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:55:46.811 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:55:46 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:55:47.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:46 smithi117 bash[28843]: audit 2024-03-04T15:55:45.752624+0000 mon.a (mon.0) 1461 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi117-63991-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:47.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:46 smithi117 bash[28843]: cluster 2024-03-04T15:55:45.764373+0000 mon.a (mon.0) 1462 : cluster 0 osdmap e157: 8 total, 8 up, 8 in 2024-03-04T15:55:47.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:46 smithi117 bash[28843]: audit 2024-03-04T15:55:45.783417+0000 mon.a (mon.0) 1463 : audit 1 from='client.? 172.21.15.117:0/3557241919' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi117-63923-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:47.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:46 smithi117 bash[28843]: audit 2024-03-04T15:55:46.663946+0000 mon.a (mon.0) 1464 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:47.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:46 smithi117 bash[21480]: audit 2024-03-04T15:55:45.752624+0000 mon.a (mon.0) 1461 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi117-63991-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:47.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:46 smithi117 bash[21480]: cluster 2024-03-04T15:55:45.764373+0000 mon.a (mon.0) 1462 : cluster 0 osdmap e157: 8 total, 8 up, 8 in 2024-03-04T15:55:47.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:46 smithi117 bash[21480]: audit 2024-03-04T15:55:45.783417+0000 mon.a (mon.0) 1463 : audit 1 from='client.? 172.21.15.117:0/3557241919' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi117-63923-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:47.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:46 smithi117 bash[21480]: audit 2024-03-04T15:55:46.663946+0000 mon.a (mon.0) 1464 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:47.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:46 smithi151 bash[23756]: audit 2024-03-04T15:55:45.752624+0000 mon.a (mon.0) 1461 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi117-63991-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:47.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:46 smithi151 bash[23756]: cluster 2024-03-04T15:55:45.764373+0000 mon.a (mon.0) 1462 : cluster 0 osdmap e157: 8 total, 8 up, 8 in 2024-03-04T15:55:47.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:46 smithi151 bash[23756]: audit 2024-03-04T15:55:45.783417+0000 mon.a (mon.0) 1463 : audit 1 from='client.? 172.21.15.117:0/3557241919' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi117-63923-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:47.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:46 smithi151 bash[23756]: audit 2024-03-04T15:55:46.663946+0000 mon.a (mon.0) 1464 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:48.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:46.553665+0000 mgr.y (mgr.24428) 256 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:55:48.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:46.757759+0000 mon.a (mon.0) 1465 : audit 1 from='client.? 172.21.15.117:0/3557241919' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi117-63923-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:48.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: cluster 2024-03-04T15:55:46.770874+0000 mon.a (mon.0) 1466 : cluster 0 osdmap e158: 8 total, 8 up, 8 in 2024-03-04T15:55:48.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:46.774490+0000 mon.c (mon.2) 241 : audit 1 from='client.? 172.21.15.117:0/4029576540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:48.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:46.775177+0000 mon.c (mon.2) 242 : audit 1 from='client.? 172.21.15.117:0/998918543' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi117-63915-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:48.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:46.792144+0000 mon.a (mon.0) 1467 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:48.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:46.793103+0000 mon.a (mon.0) 1468 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi117-63915-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:48.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: cluster 2024-03-04T15:55:46.836301+0000 mgr.y (mgr.24428) 257 : cluster 0 pgmap v243: 356 pgs: 32 creating+peering, 32 unknown, 292 active+clean; 592 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-03-04T15:55:48.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:46.874784+0000 mon.c (mon.2) 243 : audit 1 from='client.? 172.21.15.117:0/3950150976' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:55:48.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:47.285463+0000 mon.a (mon.0) 1469 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:55:48.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:47.286643+0000 mon.c (mon.2) 244 : audit 1 from='client.? 172.21.15.117:0/3950150976' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:55:48.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:47.287268+0000 mon.a (mon.0) 1470 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:55:48.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:47.288135+0000 mon.c (mon.2) 245 : audit 1 from='client.? 172.21.15.117:0/3950150976' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:48.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:47.288745+0000 mon.a (mon.0) 1471 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:48.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:47.665683+0000 mon.a (mon.0) 1472 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:48.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:46.553665+0000 mgr.y (mgr.24428) 256 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:55:48.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:46.757759+0000 mon.a (mon.0) 1465 : audit 1 from='client.? 172.21.15.117:0/3557241919' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi117-63923-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:48.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: cluster 2024-03-04T15:55:46.770874+0000 mon.a (mon.0) 1466 : cluster 0 osdmap e158: 8 total, 8 up, 8 in 2024-03-04T15:55:48.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:46.774490+0000 mon.c (mon.2) 241 : audit 1 from='client.? 172.21.15.117:0/4029576540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:48.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:46.775177+0000 mon.c (mon.2) 242 : audit 1 from='client.? 172.21.15.117:0/998918543' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi117-63915-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:48.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:46.792144+0000 mon.a (mon.0) 1467 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:48.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:46.793103+0000 mon.a (mon.0) 1468 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi117-63915-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:48.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: cluster 2024-03-04T15:55:46.836301+0000 mgr.y (mgr.24428) 257 : cluster 0 pgmap v243: 356 pgs: 32 creating+peering, 32 unknown, 292 active+clean; 592 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-03-04T15:55:48.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:46.874784+0000 mon.c (mon.2) 243 : audit 1 from='client.? 172.21.15.117:0/3950150976' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:55:48.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:47.285463+0000 mon.a (mon.0) 1469 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:55:48.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:47.286643+0000 mon.c (mon.2) 244 : audit 1 from='client.? 172.21.15.117:0/3950150976' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:55:48.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:47.287268+0000 mon.a (mon.0) 1470 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:55:48.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:47.288135+0000 mon.c (mon.2) 245 : audit 1 from='client.? 172.21.15.117:0/3950150976' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:48.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:47.288745+0000 mon.a (mon.0) 1471 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:48.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:47.665683+0000 mon.a (mon.0) 1472 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:48.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:47.761944+0000 mon.a (mon.0) 1473 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]': finished 2024-03-04T15:55:48.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:47.762124+0000 mon.a (mon.0) 1474 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi117-63915-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:48.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:47.762337+0000 mon.a (mon.0) 1475 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:55:48.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: cluster 2024-03-04T15:55:47.773198+0000 mon.a (mon.0) 1476 : cluster 0 osdmap e159: 8 total, 8 up, 8 in 2024-03-04T15:55:48.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:47.773933+0000 mon.c (mon.2) 246 : audit 1 from='client.? 172.21.15.117:0/3950150976' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi117-64156-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:55:48.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:47.774506+0000 mon.c (mon.2) 247 : audit 1 from='client.? 172.21.15.117:0/4029576540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:48.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:47.784953+0000 mon.a (mon.0) 1477 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi117-64156-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:55:48.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:47.785844+0000 mon.a (mon.0) 1478 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:48.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[28843]: audit 2024-03-04T15:55:47.787060+0000 mon.a (mon.0) 1479 : audit 1 from='client.? 172.21.15.117:0/709647812' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi117-63991-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:48.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:47.761944+0000 mon.a (mon.0) 1473 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]': finished 2024-03-04T15:55:48.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:47.762124+0000 mon.a (mon.0) 1474 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi117-63915-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:48.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:47.762337+0000 mon.a (mon.0) 1475 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:55:48.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: cluster 2024-03-04T15:55:47.773198+0000 mon.a (mon.0) 1476 : cluster 0 osdmap e159: 8 total, 8 up, 8 in 2024-03-04T15:55:48.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:47.773933+0000 mon.c (mon.2) 246 : audit 1 from='client.? 172.21.15.117:0/3950150976' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi117-64156-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:55:48.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:47.774506+0000 mon.c (mon.2) 247 : audit 1 from='client.? 172.21.15.117:0/4029576540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:48.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:47.784953+0000 mon.a (mon.0) 1477 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi117-64156-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:55:48.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:47.785844+0000 mon.a (mon.0) 1478 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:48.073 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:47 smithi117 bash[21480]: audit 2024-03-04T15:55:47.787060+0000 mon.a (mon.0) 1479 : audit 1 from='client.? 172.21.15.117:0/709647812' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi117-63991-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:48.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:46.553665+0000 mgr.y (mgr.24428) 256 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:55:48.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:46.757759+0000 mon.a (mon.0) 1465 : audit 1 from='client.? 172.21.15.117:0/3557241919' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi117-63923-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:48.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: cluster 2024-03-04T15:55:46.770874+0000 mon.a (mon.0) 1466 : cluster 0 osdmap e158: 8 total, 8 up, 8 in 2024-03-04T15:55:48.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:46.774490+0000 mon.c (mon.2) 241 : audit 1 from='client.? 172.21.15.117:0/4029576540' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:48.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:46.775177+0000 mon.c (mon.2) 242 : audit 1 from='client.? 172.21.15.117:0/998918543' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi117-63915-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:48.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:46.792144+0000 mon.a (mon.0) 1467 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:48.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:46.793103+0000 mon.a (mon.0) 1468 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi117-63915-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:48.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: cluster 2024-03-04T15:55:46.836301+0000 mgr.y (mgr.24428) 257 : cluster 0 pgmap v243: 356 pgs: 32 creating+peering, 32 unknown, 292 active+clean; 592 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-03-04T15:55:48.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:46.874784+0000 mon.c (mon.2) 243 : audit 1 from='client.? 172.21.15.117:0/3950150976' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:55:48.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:47.285463+0000 mon.a (mon.0) 1469 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:55:48.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:47.286643+0000 mon.c (mon.2) 244 : audit 1 from='client.? 172.21.15.117:0/3950150976' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:55:48.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:47.287268+0000 mon.a (mon.0) 1470 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:55:48.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:47.288135+0000 mon.c (mon.2) 245 : audit 1 from='client.? 172.21.15.117:0/3950150976' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:48.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:47.288745+0000 mon.a (mon.0) 1471 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:48.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:47.665683+0000 mon.a (mon.0) 1472 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:48.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:47.761944+0000 mon.a (mon.0) 1473 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]': finished 2024-03-04T15:55:48.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:47.762124+0000 mon.a (mon.0) 1474 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi117-63915-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:48.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:47.762337+0000 mon.a (mon.0) 1475 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:55:48.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: cluster 2024-03-04T15:55:47.773198+0000 mon.a (mon.0) 1476 : cluster 0 osdmap e159: 8 total, 8 up, 8 in 2024-03-04T15:55:48.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:47.773933+0000 mon.c (mon.2) 246 : audit 1 from='client.? 172.21.15.117:0/3950150976' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi117-64156-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:55:48.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:47.774506+0000 mon.c (mon.2) 247 : audit 1 from='client.? 172.21.15.117:0/4029576540' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:48.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:47.784953+0000 mon.a (mon.0) 1477 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi117-64156-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:55:48.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:47.785844+0000 mon.a (mon.0) 1478 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]: dispatch 2024-03-04T15:55:48.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:47 smithi151 bash[23756]: audit 2024-03-04T15:55:47.787060+0000 mon.a (mon.0) 1479 : audit 1 from='client.? 172.21.15.117:0/709647812' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi117-63991-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:48.799 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP api_snapshots: Running main() from gmock_main.cc 2024-03-04T15:55:48.799 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [==========] Running 13 tests from 4 test suites. 2024-03-04T15:55:48.799 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [----------] Global test environment set-up. 2024-03-04T15:55:48.799 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-03-04T15:55:48.799 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-03-04T15:55:48.800 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (14813 ms) 2024-03-04T15:55:48.800 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-03-04T15:55:48.800 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (6488 ms) 2024-03-04T15:55:48.800 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-03-04T15:55:48.800 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (5161 ms) 2024-03-04T15:55:48.800 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-03-04T15:55:48.800 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (6180 ms) 2024-03-04T15:55:48.800 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (32642 ms total) 2024-03-04T15:55:48.800 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: 2024-03-04T15:55:48.800 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged 2024-03-04T15:55:48.800 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-03-04T15:55:48.800 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (7939 ms) 2024-03-04T15:55:48.800 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-03-04T15:55:48.801 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (6037 ms) 2024-03-04T15:55:48.801 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.FutureSnapRollback 2024-03-04T15:55:48.801 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.FutureSnapRollback (7277 ms) 2024-03-04T15:55:48.801 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [----------] 3 tests from LibRadosSnapshotsSelfManaged (21253 ms total) 2024-03-04T15:55:48.801 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: 2024-03-04T15:55:48.801 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-03-04T15:55:48.801 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-03-04T15:55:48.801 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (3016 ms) 2024-03-04T15:55:48.801 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-03-04T15:55:48.801 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (1497 ms) 2024-03-04T15:55:48.801 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-03-04T15:55:48.801 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (2022 ms) 2024-03-04T15:55:48.801 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-03-04T15:55:48.802 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (2001 ms) 2024-03-04T15:55:48.802 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (8536 ms total) 2024-03-04T15:55:48.802 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: 2024-03-04T15:55:48.802 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-03-04T15:55:48.802 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-03-04T15:55:48.802 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4105 ms) 2024-03-04T15:55:48.802 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-03-04T15:55:48.802 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4015 ms) 2024-03-04T15:55:48.802 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8120 ms total) 2024-03-04T15:55:48.802 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: 2024-03-04T15:55:48.802 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [----------] Global test environment tear-down 2024-03-04T15:55:48.802 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [==========] 13 tests from 4 test suites ran. (108805 ms total) 2024-03-04T15:55:48.802 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots: [ PASSED ] 13 tests. 2024-03-04T15:55:49.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:49 smithi117 bash[21480]: cluster 2024-03-04T15:55:47.811326+0000 mon.a (mon.0) 1480 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:49.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:49 smithi117 bash[21480]: audit 2024-03-04T15:55:48.667440+0000 mon.a (mon.0) 1481 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:49.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:49 smithi117 bash[21480]: audit 2024-03-04T15:55:48.767458+0000 mon.a (mon.0) 1482 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]': finished 2024-03-04T15:55:49.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:49 smithi117 bash[21480]: audit 2024-03-04T15:55:48.767701+0000 mon.a (mon.0) 1483 : audit 1 from='client.? 172.21.15.117:0/709647812' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi117-63991-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:49.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:49 smithi117 bash[28843]: cluster 2024-03-04T15:55:47.811326+0000 mon.a (mon.0) 1480 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:49.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:49 smithi117 bash[28843]: audit 2024-03-04T15:55:48.667440+0000 mon.a (mon.0) 1481 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:49.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:49 smithi117 bash[28843]: audit 2024-03-04T15:55:48.767458+0000 mon.a (mon.0) 1482 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]': finished 2024-03-04T15:55:49.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:49 smithi117 bash[28843]: audit 2024-03-04T15:55:48.767701+0000 mon.a (mon.0) 1483 : audit 1 from='client.? 172.21.15.117:0/709647812' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi117-63991-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:49 smithi151 bash[23756]: cluster 2024-03-04T15:55:47.811326+0000 mon.a (mon.0) 1480 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:49 smithi151 bash[23756]: audit 2024-03-04T15:55:48.667440+0000 mon.a (mon.0) 1481 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:49 smithi151 bash[23756]: audit 2024-03-04T15:55:48.767458+0000 mon.a (mon.0) 1482 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi117-64133-15"}]': finished 2024-03-04T15:55:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:49 smithi151 bash[23756]: audit 2024-03-04T15:55:48.767701+0000 mon.a (mon.0) 1483 : audit 1 from='client.? 172.21.15.117:0/709647812' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi117-63991-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:50.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:50 smithi117 bash[28843]: cluster 2024-03-04T15:55:48.837227+0000 mgr.y (mgr.24428) 258 : cluster 0 pgmap v246: 356 pgs: 64 unknown, 292 active+clean; 592 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-03-04T15:55:50.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:50 smithi117 bash[28843]: cluster 2024-03-04T15:55:49.235745+0000 mon.a (mon.0) 1484 : cluster 0 osdmap e160: 8 total, 8 up, 8 in 2024-03-04T15:55:50.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:50 smithi117 bash[28843]: audit 2024-03-04T15:55:49.237714+0000 mon.c (mon.2) 248 : audit 1 from='client.? 172.21.15.117:0/59429415' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi117-63923-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:50.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:50 smithi117 bash[28843]: audit 2024-03-04T15:55:49.256486+0000 mon.a (mon.0) 1485 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi117-63923-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:50.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:50 smithi117 bash[28843]: audit 2024-03-04T15:55:49.704119+0000 mon.a (mon.0) 1486 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:50.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:50 smithi117 bash[28843]: audit 2024-03-04T15:55:49.773741+0000 mon.a (mon.0) 1487 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi117-64156-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]': finished 2024-03-04T15:55:50.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:50 smithi117 bash[28843]: audit 2024-03-04T15:55:49.774034+0000 mon.a (mon.0) 1488 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi117-63923-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:50.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:50 smithi117 bash[28843]: cluster 2024-03-04T15:55:49.788669+0000 mon.a (mon.0) 1489 : cluster 0 osdmap e161: 8 total, 8 up, 8 in 2024-03-04T15:55:50.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:50 smithi117 bash[28843]: audit 2024-03-04T15:55:49.804144+0000 mon.c (mon.2) 249 : audit 1 from='client.? 172.21.15.117:0/212458582' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi117-63915-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:50.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:50 smithi117 bash[28843]: audit 2024-03-04T15:55:49.813075+0000 mon.a (mon.0) 1490 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi117-63915-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:50.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:50 smithi117 bash[21480]: cluster 2024-03-04T15:55:48.837227+0000 mgr.y (mgr.24428) 258 : cluster 0 pgmap v246: 356 pgs: 64 unknown, 292 active+clean; 592 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-03-04T15:55:50.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:50 smithi117 bash[21480]: cluster 2024-03-04T15:55:49.235745+0000 mon.a (mon.0) 1484 : cluster 0 osdmap e160: 8 total, 8 up, 8 in 2024-03-04T15:55:50.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:50 smithi117 bash[21480]: audit 2024-03-04T15:55:49.237714+0000 mon.c (mon.2) 248 : audit 1 from='client.? 172.21.15.117:0/59429415' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi117-63923-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:50.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:50 smithi117 bash[21480]: audit 2024-03-04T15:55:49.256486+0000 mon.a (mon.0) 1485 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi117-63923-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:50.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:50 smithi117 bash[21480]: audit 2024-03-04T15:55:49.704119+0000 mon.a (mon.0) 1486 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:50.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:50 smithi117 bash[21480]: audit 2024-03-04T15:55:49.773741+0000 mon.a (mon.0) 1487 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi117-64156-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]': finished 2024-03-04T15:55:50.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:50 smithi117 bash[21480]: audit 2024-03-04T15:55:49.774034+0000 mon.a (mon.0) 1488 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi117-63923-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:50.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:50 smithi117 bash[21480]: cluster 2024-03-04T15:55:49.788669+0000 mon.a (mon.0) 1489 : cluster 0 osdmap e161: 8 total, 8 up, 8 in 2024-03-04T15:55:50.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:50 smithi117 bash[21480]: audit 2024-03-04T15:55:49.804144+0000 mon.c (mon.2) 249 : audit 1 from='client.? 172.21.15.117:0/212458582' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi117-63915-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:50.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:50 smithi117 bash[21480]: audit 2024-03-04T15:55:49.813075+0000 mon.a (mon.0) 1490 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi117-63915-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:50.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:50 smithi151 bash[23756]: cluster 2024-03-04T15:55:48.837227+0000 mgr.y (mgr.24428) 258 : cluster 0 pgmap v246: 356 pgs: 64 unknown, 292 active+clean; 592 KiB data, 590 MiB used, 715 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-03-04T15:55:50.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:50 smithi151 bash[23756]: cluster 2024-03-04T15:55:49.235745+0000 mon.a (mon.0) 1484 : cluster 0 osdmap e160: 8 total, 8 up, 8 in 2024-03-04T15:55:50.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:50 smithi151 bash[23756]: audit 2024-03-04T15:55:49.237714+0000 mon.c (mon.2) 248 : audit 1 from='client.? 172.21.15.117:0/59429415' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi117-63923-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:50.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:50 smithi151 bash[23756]: audit 2024-03-04T15:55:49.256486+0000 mon.a (mon.0) 1485 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi117-63923-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:50.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:50 smithi151 bash[23756]: audit 2024-03-04T15:55:49.704119+0000 mon.a (mon.0) 1486 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:50.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:50 smithi151 bash[23756]: audit 2024-03-04T15:55:49.773741+0000 mon.a (mon.0) 1487 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi117-64156-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]': finished 2024-03-04T15:55:50.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:50 smithi151 bash[23756]: audit 2024-03-04T15:55:49.774034+0000 mon.a (mon.0) 1488 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi117-63923-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:50.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:50 smithi151 bash[23756]: cluster 2024-03-04T15:55:49.788669+0000 mon.a (mon.0) 1489 : cluster 0 osdmap e161: 8 total, 8 up, 8 in 2024-03-04T15:55:50.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:50 smithi151 bash[23756]: audit 2024-03-04T15:55:49.804144+0000 mon.c (mon.2) 249 : audit 1 from='client.? 172.21.15.117:0/212458582' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi117-63915-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:50.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:50 smithi151 bash[23756]: audit 2024-03-04T15:55:49.813075+0000 mon.a (mon.0) 1490 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi117-63915-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:51.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:51 smithi117 bash[21480]: audit 2024-03-04T15:55:50.705744+0000 mon.a (mon.0) 1491 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:51.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:51 smithi117 bash[21480]: audit 2024-03-04T15:55:50.779615+0000 mon.a (mon.0) 1492 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi117-63915-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:51.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:51 smithi117 bash[21480]: cluster 2024-03-04T15:55:50.788568+0000 mon.a (mon.0) 1493 : cluster 0 osdmap e162: 8 total, 8 up, 8 in 2024-03-04T15:55:51.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:51 smithi117 bash[21480]: audit 2024-03-04T15:55:50.799325+0000 mon.c (mon.2) 250 : audit 1 from='client.? 172.21.15.117:0/1846448511' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi117-63991-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:51.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:51 smithi117 bash[21480]: audit 2024-03-04T15:55:50.809104+0000 mon.a (mon.0) 1494 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi117-63991-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:51.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:51 smithi117 bash[21480]: cluster 2024-03-04T15:55:50.839477+0000 mgr.y (mgr.24428) 259 : cluster 0 pgmap v249: 364 pgs: 72 unknown, 292 active+clean; 592 KiB data, 590 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:55:51.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:51 smithi117 bash[28843]: audit 2024-03-04T15:55:50.705744+0000 mon.a (mon.0) 1491 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:51.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:51 smithi117 bash[28843]: audit 2024-03-04T15:55:50.779615+0000 mon.a (mon.0) 1492 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi117-63915-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:51.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:51 smithi117 bash[28843]: cluster 2024-03-04T15:55:50.788568+0000 mon.a (mon.0) 1493 : cluster 0 osdmap e162: 8 total, 8 up, 8 in 2024-03-04T15:55:51.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:51 smithi117 bash[28843]: audit 2024-03-04T15:55:50.799325+0000 mon.c (mon.2) 250 : audit 1 from='client.? 172.21.15.117:0/1846448511' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi117-63991-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:51.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:51 smithi117 bash[28843]: audit 2024-03-04T15:55:50.809104+0000 mon.a (mon.0) 1494 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi117-63991-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:51.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:51 smithi117 bash[28843]: cluster 2024-03-04T15:55:50.839477+0000 mgr.y (mgr.24428) 259 : cluster 0 pgmap v249: 364 pgs: 72 unknown, 292 active+clean; 592 KiB data, 590 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:55:51.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:51 smithi151 bash[23756]: audit 2024-03-04T15:55:50.705744+0000 mon.a (mon.0) 1491 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:51.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:51 smithi151 bash[23756]: audit 2024-03-04T15:55:50.779615+0000 mon.a (mon.0) 1492 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Operate2Mtime_smithi117-63915-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:51.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:51 smithi151 bash[23756]: cluster 2024-03-04T15:55:50.788568+0000 mon.a (mon.0) 1493 : cluster 0 osdmap e162: 8 total, 8 up, 8 in 2024-03-04T15:55:51.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:51 smithi151 bash[23756]: audit 2024-03-04T15:55:50.799325+0000 mon.c (mon.2) 250 : audit 1 from='client.? 172.21.15.117:0/1846448511' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi117-63991-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:51.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:51 smithi151 bash[23756]: audit 2024-03-04T15:55:50.809104+0000 mon.a (mon.0) 1494 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi117-63991-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:51.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:51 smithi151 bash[23756]: cluster 2024-03-04T15:55:50.839477+0000 mgr.y (mgr.24428) 259 : cluster 0 pgmap v249: 364 pgs: 72 unknown, 292 active+clean; 592 KiB data, 590 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:55:52.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:52 smithi117 bash[21480]: audit 2024-03-04T15:55:51.707624+0000 mon.a (mon.0) 1495 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:52.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:52 smithi117 bash[21480]: audit 2024-03-04T15:55:51.790650+0000 mon.a (mon.0) 1496 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi117-63991-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:52.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:52 smithi117 bash[21480]: cluster 2024-03-04T15:55:51.797005+0000 mon.a (mon.0) 1497 : cluster 0 osdmap e163: 8 total, 8 up, 8 in 2024-03-04T15:55:52.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:52 smithi117 bash[21480]: audit 2024-03-04T15:55:51.802016+0000 mon.c (mon.2) 251 : audit 1 from='client.? 172.21.15.117:0/2967224829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi117-63923-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:52.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:52 smithi117 bash[21480]: audit 2024-03-04T15:55:51.804408+0000 mon.a (mon.0) 1498 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi117-63923-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:52.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:52 smithi117 bash[28843]: audit 2024-03-04T15:55:51.707624+0000 mon.a (mon.0) 1495 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:52.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:52 smithi117 bash[28843]: audit 2024-03-04T15:55:51.790650+0000 mon.a (mon.0) 1496 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi117-63991-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:52.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:52 smithi117 bash[28843]: cluster 2024-03-04T15:55:51.797005+0000 mon.a (mon.0) 1497 : cluster 0 osdmap e163: 8 total, 8 up, 8 in 2024-03-04T15:55:52.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:52 smithi117 bash[28843]: audit 2024-03-04T15:55:51.802016+0000 mon.c (mon.2) 251 : audit 1 from='client.? 172.21.15.117:0/2967224829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi117-63923-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:52.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:52 smithi117 bash[28843]: audit 2024-03-04T15:55:51.804408+0000 mon.a (mon.0) 1498 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi117-63923-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:52 smithi151 bash[23756]: audit 2024-03-04T15:55:51.707624+0000 mon.a (mon.0) 1495 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:52 smithi151 bash[23756]: audit 2024-03-04T15:55:51.790650+0000 mon.a (mon.0) 1496 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi117-63991-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:52 smithi151 bash[23756]: cluster 2024-03-04T15:55:51.797005+0000 mon.a (mon.0) 1497 : cluster 0 osdmap e163: 8 total, 8 up, 8 in 2024-03-04T15:55:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:52 smithi151 bash[23756]: audit 2024-03-04T15:55:51.802016+0000 mon.c (mon.2) 251 : audit 1 from='client.? 172.21.15.117:0/2967224829' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi117-63923-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:52 smithi151 bash[23756]: audit 2024-03-04T15:55:51.804408+0000 mon.a (mon.0) 1498 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi117-63923-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:54.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[28843]: audit 2024-03-04T15:55:52.694265+0000 mon.a (mon.0) 1499 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi117-63923-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:54.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[28843]: cluster 2024-03-04T15:55:52.704201+0000 mon.a (mon.0) 1500 : cluster 0 osdmap e164: 8 total, 8 up, 8 in 2024-03-04T15:55:54.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[28843]: audit 2024-03-04T15:55:52.708025+0000 mon.c (mon.2) 252 : audit 1 from='client.? 172.21.15.117:0/3876683642' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi117-63915-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:54.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[28843]: audit 2024-03-04T15:55:52.710617+0000 mon.a (mon.0) 1501 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi117-63915-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:54.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[28843]: audit 2024-03-04T15:55:52.711084+0000 mon.a (mon.0) 1502 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:54.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[28843]: audit 2024-03-04T15:55:52.750696+0000 mon.c (mon.2) 253 : audit 1 from='client.? 172.21.15.117:0/3111646600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:54.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[28843]: audit 2024-03-04T15:55:52.751395+0000 mon.a (mon.0) 1503 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:54.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[28843]: audit 2024-03-04T15:55:52.752330+0000 mon.c (mon.2) 254 : audit 1 from='client.? 172.21.15.117:0/3111646600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:54.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[28843]: audit 2024-03-04T15:55:52.752975+0000 mon.a (mon.0) 1504 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:54.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[28843]: audit 2024-03-04T15:55:52.753533+0000 mon.c (mon.2) 255 : audit 1 from='client.? 172.21.15.117:0/3111646600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:54.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[28843]: audit 2024-03-04T15:55:52.753858+0000 mon.a (mon.0) 1505 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:54.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[28843]: cluster 2024-03-04T15:55:52.840087+0000 mgr.y (mgr.24428) 260 : cluster 0 pgmap v252: 364 pgs: 64 unknown, 300 active+clean; 592 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:55:54.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[21480]: audit 2024-03-04T15:55:52.694265+0000 mon.a (mon.0) 1499 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi117-63923-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:54.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[21480]: cluster 2024-03-04T15:55:52.704201+0000 mon.a (mon.0) 1500 : cluster 0 osdmap e164: 8 total, 8 up, 8 in 2024-03-04T15:55:54.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[21480]: audit 2024-03-04T15:55:52.708025+0000 mon.c (mon.2) 252 : audit 1 from='client.? 172.21.15.117:0/3876683642' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi117-63915-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:54.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[21480]: audit 2024-03-04T15:55:52.710617+0000 mon.a (mon.0) 1501 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi117-63915-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:54.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[21480]: audit 2024-03-04T15:55:52.711084+0000 mon.a (mon.0) 1502 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:54.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[21480]: audit 2024-03-04T15:55:52.750696+0000 mon.c (mon.2) 253 : audit 1 from='client.? 172.21.15.117:0/3111646600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:54.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[21480]: audit 2024-03-04T15:55:52.751395+0000 mon.a (mon.0) 1503 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:54.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[21480]: audit 2024-03-04T15:55:52.752330+0000 mon.c (mon.2) 254 : audit 1 from='client.? 172.21.15.117:0/3111646600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:54.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[21480]: audit 2024-03-04T15:55:52.752975+0000 mon.a (mon.0) 1504 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:54.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[21480]: audit 2024-03-04T15:55:52.753533+0000 mon.c (mon.2) 255 : audit 1 from='client.? 172.21.15.117:0/3111646600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:54.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[21480]: audit 2024-03-04T15:55:52.753858+0000 mon.a (mon.0) 1505 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:54.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:53 smithi117 bash[21480]: cluster 2024-03-04T15:55:52.840087+0000 mgr.y (mgr.24428) 260 : cluster 0 pgmap v252: 364 pgs: 64 unknown, 300 active+clean; 592 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:55:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:53 smithi151 bash[23756]: audit 2024-03-04T15:55:52.694265+0000 mon.a (mon.0) 1499 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi117-63923-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:53 smithi151 bash[23756]: cluster 2024-03-04T15:55:52.704201+0000 mon.a (mon.0) 1500 : cluster 0 osdmap e164: 8 total, 8 up, 8 in 2024-03-04T15:55:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:53 smithi151 bash[23756]: audit 2024-03-04T15:55:52.708025+0000 mon.c (mon.2) 252 : audit 1 from='client.? 172.21.15.117:0/3876683642' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi117-63915-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:53 smithi151 bash[23756]: audit 2024-03-04T15:55:52.710617+0000 mon.a (mon.0) 1501 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi117-63915-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:54.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:53 smithi151 bash[23756]: audit 2024-03-04T15:55:52.711084+0000 mon.a (mon.0) 1502 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:54.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:53 smithi151 bash[23756]: audit 2024-03-04T15:55:52.750696+0000 mon.c (mon.2) 253 : audit 1 from='client.? 172.21.15.117:0/3111646600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:54.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:53 smithi151 bash[23756]: audit 2024-03-04T15:55:52.751395+0000 mon.a (mon.0) 1503 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:54.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:53 smithi151 bash[23756]: audit 2024-03-04T15:55:52.752330+0000 mon.c (mon.2) 254 : audit 1 from='client.? 172.21.15.117:0/3111646600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:54.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:53 smithi151 bash[23756]: audit 2024-03-04T15:55:52.752975+0000 mon.a (mon.0) 1504 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:54.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:53 smithi151 bash[23756]: audit 2024-03-04T15:55:52.753533+0000 mon.c (mon.2) 255 : audit 1 from='client.? 172.21.15.117:0/3111646600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:54.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:53 smithi151 bash[23756]: audit 2024-03-04T15:55:52.753858+0000 mon.a (mon.0) 1505 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:55:54.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:53 smithi151 bash[23756]: cluster 2024-03-04T15:55:52.840087+0000 mgr.y (mgr.24428) 260 : cluster 0 pgmap v252: 364 pgs: 64 unknown, 300 active+clean; 592 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:55:55.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:55 smithi151 bash[23756]: audit 2024-03-04T15:55:53.704624+0000 mon.a (mon.0) 1506 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi117-63915-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:55.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:55 smithi151 bash[23756]: audit 2024-03-04T15:55:53.704866+0000 mon.a (mon.0) 1507 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:55:55.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:55 smithi151 bash[23756]: cluster 2024-03-04T15:55:53.716440+0000 mon.a (mon.0) 1508 : cluster 0 osdmap e165: 8 total, 8 up, 8 in 2024-03-04T15:55:55.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:55 smithi151 bash[23756]: audit 2024-03-04T15:55:53.718706+0000 mon.a (mon.0) 1509 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:55.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:55 smithi151 bash[23756]: audit 2024-03-04T15:55:53.726645+0000 mon.c (mon.2) 256 : audit 1 from='client.? 172.21.15.117:0/3111646600' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi117-63991-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:55.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:55 smithi151 bash[23756]: audit 2024-03-04T15:55:53.730495+0000 mon.a (mon.0) 1510 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi117-63991-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:55.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:55 smithi117 bash[28843]: audit 2024-03-04T15:55:53.704624+0000 mon.a (mon.0) 1506 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi117-63915-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:55.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:55 smithi117 bash[28843]: audit 2024-03-04T15:55:53.704866+0000 mon.a (mon.0) 1507 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:55:55.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:55 smithi117 bash[28843]: cluster 2024-03-04T15:55:53.716440+0000 mon.a (mon.0) 1508 : cluster 0 osdmap e165: 8 total, 8 up, 8 in 2024-03-04T15:55:55.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:55 smithi117 bash[28843]: audit 2024-03-04T15:55:53.718706+0000 mon.a (mon.0) 1509 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:55.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:55 smithi117 bash[28843]: audit 2024-03-04T15:55:53.726645+0000 mon.c (mon.2) 256 : audit 1 from='client.? 172.21.15.117:0/3111646600' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi117-63991-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:55.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:55 smithi117 bash[28843]: audit 2024-03-04T15:55:53.730495+0000 mon.a (mon.0) 1510 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi117-63991-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:55.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:55 smithi117 bash[21480]: audit 2024-03-04T15:55:53.704624+0000 mon.a (mon.0) 1506 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi117-63915-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:55.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:55 smithi117 bash[21480]: audit 2024-03-04T15:55:53.704866+0000 mon.a (mon.0) 1507 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:55:55.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:55 smithi117 bash[21480]: cluster 2024-03-04T15:55:53.716440+0000 mon.a (mon.0) 1508 : cluster 0 osdmap e165: 8 total, 8 up, 8 in 2024-03-04T15:55:55.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:55 smithi117 bash[21480]: audit 2024-03-04T15:55:53.718706+0000 mon.a (mon.0) 1509 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:55.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:55 smithi117 bash[21480]: audit 2024-03-04T15:55:53.726645+0000 mon.c (mon.2) 256 : audit 1 from='client.? 172.21.15.117:0/3111646600' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi117-63991-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:55.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:55 smithi117 bash[21480]: audit 2024-03-04T15:55:53.730495+0000 mon.a (mon.0) 1510 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi117-63991-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:56.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:56 smithi151 bash[23756]: cluster 2024-03-04T15:55:54.840881+0000 mgr.y (mgr.24428) 261 : cluster 0 pgmap v255: 332 pgs: 32 unknown, 300 active+clean; 592 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:55:56.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:56 smithi151 bash[23756]: cluster 2024-03-04T15:55:55.143135+0000 mon.a (mon.0) 1511 : cluster 0 osdmap e166: 8 total, 8 up, 8 in 2024-03-04T15:55:56.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:56 smithi151 bash[23756]: audit 2024-03-04T15:55:55.154912+0000 mon.a (mon.0) 1512 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:56.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:56 smithi151 bash[23756]: audit 2024-03-04T15:55:55.157045+0000 mon.a (mon.0) 1513 : audit 1 from='client.? 172.21.15.117:0/3132477562' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi117-63923-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:56.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:56 smithi151 bash[23756]: audit 2024-03-04T15:55:55.714240+0000 mon.a (mon.0) 1514 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi117-63991-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi117-63991-36"}]': finished 2024-03-04T15:55:56.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:56 smithi151 bash[23756]: audit 2024-03-04T15:55:55.714531+0000 mon.a (mon.0) 1515 : audit 1 from='client.? 172.21.15.117:0/3132477562' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi117-63923-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:56.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:56 smithi151 bash[23756]: cluster 2024-03-04T15:55:55.726525+0000 mon.a (mon.0) 1516 : cluster 0 osdmap e167: 8 total, 8 up, 8 in 2024-03-04T15:55:56.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:56 smithi151 bash[23756]: audit 2024-03-04T15:55:55.731537+0000 mon.c (mon.2) 257 : audit 1 from='client.? 172.21.15.117:0/1800255503' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi117-63915-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:56.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:56 smithi151 bash[23756]: audit 2024-03-04T15:55:55.740580+0000 mon.a (mon.0) 1517 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi117-63915-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:56.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:55:56 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:55:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:55:56.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:56 smithi117 bash[21480]: cluster 2024-03-04T15:55:54.840881+0000 mgr.y (mgr.24428) 261 : cluster 0 pgmap v255: 332 pgs: 32 unknown, 300 active+clean; 592 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:55:56.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:56 smithi117 bash[21480]: cluster 2024-03-04T15:55:55.143135+0000 mon.a (mon.0) 1511 : cluster 0 osdmap e166: 8 total, 8 up, 8 in 2024-03-04T15:55:56.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:56 smithi117 bash[21480]: audit 2024-03-04T15:55:55.154912+0000 mon.a (mon.0) 1512 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:56.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:56 smithi117 bash[21480]: audit 2024-03-04T15:55:55.157045+0000 mon.a (mon.0) 1513 : audit 1 from='client.? 172.21.15.117:0/3132477562' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi117-63923-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:56.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:56 smithi117 bash[21480]: audit 2024-03-04T15:55:55.714240+0000 mon.a (mon.0) 1514 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi117-63991-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi117-63991-36"}]': finished 2024-03-04T15:55:56.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:56 smithi117 bash[21480]: audit 2024-03-04T15:55:55.714531+0000 mon.a (mon.0) 1515 : audit 1 from='client.? 172.21.15.117:0/3132477562' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi117-63923-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:56.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:56 smithi117 bash[21480]: cluster 2024-03-04T15:55:55.726525+0000 mon.a (mon.0) 1516 : cluster 0 osdmap e167: 8 total, 8 up, 8 in 2024-03-04T15:55:56.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:56 smithi117 bash[21480]: audit 2024-03-04T15:55:55.731537+0000 mon.c (mon.2) 257 : audit 1 from='client.? 172.21.15.117:0/1800255503' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi117-63915-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:56.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:56 smithi117 bash[21480]: audit 2024-03-04T15:55:55.740580+0000 mon.a (mon.0) 1517 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi117-63915-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:56.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:56 smithi117 bash[28843]: cluster 2024-03-04T15:55:54.840881+0000 mgr.y (mgr.24428) 261 : cluster 0 pgmap v255: 332 pgs: 32 unknown, 300 active+clean; 592 KiB data, 566 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:55:56.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:56 smithi117 bash[28843]: cluster 2024-03-04T15:55:55.143135+0000 mon.a (mon.0) 1511 : cluster 0 osdmap e166: 8 total, 8 up, 8 in 2024-03-04T15:55:56.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:56 smithi117 bash[28843]: audit 2024-03-04T15:55:55.154912+0000 mon.a (mon.0) 1512 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:56.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:56 smithi117 bash[28843]: audit 2024-03-04T15:55:55.157045+0000 mon.a (mon.0) 1513 : audit 1 from='client.? 172.21.15.117:0/3132477562' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi117-63923-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:56.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:56 smithi117 bash[28843]: audit 2024-03-04T15:55:55.714240+0000 mon.a (mon.0) 1514 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi117-63991-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi117-63991-36"}]': finished 2024-03-04T15:55:56.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:56 smithi117 bash[28843]: audit 2024-03-04T15:55:55.714531+0000 mon.a (mon.0) 1515 : audit 1 from='client.? 172.21.15.117:0/3132477562' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi117-63923-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:56.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:56 smithi117 bash[28843]: cluster 2024-03-04T15:55:55.726525+0000 mon.a (mon.0) 1516 : cluster 0 osdmap e167: 8 total, 8 up, 8 in 2024-03-04T15:55:56.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:56 smithi117 bash[28843]: audit 2024-03-04T15:55:55.731537+0000 mon.c (mon.2) 257 : audit 1 from='client.? 172.21.15.117:0/1800255503' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi117-63915-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:56.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:56 smithi117 bash[28843]: audit 2024-03-04T15:55:55.740580+0000 mon.a (mon.0) 1517 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi117-63915-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:56.732 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: Running main() from gmock_main.cc 2024-03-04T15:55:56.732 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [==========] Running 21 tests from 5 test suites. 2024-03-04T15:55:56.732 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-03-04T15:55:56.732 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-03-04T15:55:56.732 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: seed 64156 2024-03-04T15:55:56.732 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-03-04T15:55:56.732 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (5234 ms) 2024-03-04T15:55:56.732 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-03-04T15:55:56.732 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (6108 ms) 2024-03-04T15:55:56.732 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-03-04T15:55:56.732 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (6340 ms) 2024-03-04T15:55:56.732 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-03-04T15:55:56.733 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (5021 ms) 2024-03-04T15:55:56.733 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-03-04T15:55:56.733 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (7378 ms) 2024-03-04T15:55:56.733 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (30081 ms total) 2024-03-04T15:55:56.733 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: 2024-03-04T15:55:56.733 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP 2024-03-04T15:55:56.733 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-03-04T15:55:56.733 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (7643 ms) 2024-03-04T15:55:56.733 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-03-04T15:55:56.733 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (6535 ms) 2024-03-04T15:55:56.733 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-03-04T15:55:56.733 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (9282 ms) 2024-03-04T15:55:56.733 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-03-04T15:55:56.733 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (5087 ms) 2024-03-04T15:55:56.733 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-03-04T15:55:56.734 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2788 ms) 2024-03-04T15:55:56.734 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-03-04T15:55:56.734 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: ./src/test/librados/snapshots_cxx.cc:463: Skipped 2024-03-04T15:55:56.734 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: 2024-03-04T15:55:56.734 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback (0 ms) 2024-03-04T15:55:56.734 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-03-04T15:55:56.734 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: deleting snap 14 in pool LibRadosSnapshotsSelfManagedPP_smithi117-64156-7 2024-03-04T15:55:56.734 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: waiting for snaps to purge 2024-03-04T15:55:56.734 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17934 ms) 2024-03-04T15:55:56.734 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [----------] 7 tests from LibRadosSnapshotsSelfManagedPP (49269 ms total) 2024-03-04T15:55:56.734 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: 2024-03-04T15:55:56.734 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-03-04T15:55:56.734 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-03-04T15:55:56.734 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (17 ms) 2024-03-04T15:55:56.735 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-03-04T15:55:56.735 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5104 ms) 2024-03-04T15:55:56.735 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5121 ms total) 2024-03-04T15:55:56.735 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: 2024-03-04T15:55:56.735 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-03-04T15:55:56.735 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-03-04T15:55:56.735 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (2908 ms) 2024-03-04T15:55:56.735 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-03-04T15:55:56.735 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapRemovePP (2437 ms) 2024-03-04T15:55:56.735 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-03-04T15:55:56.735 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (1591 ms) 2024-03-04T15:55:56.932 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:55:56 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:55:57.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:57 smithi117 bash[21480]: audit 2024-03-04T15:55:56.157875+0000 mon.a (mon.0) 1518 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:57.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:57 smithi117 bash[21480]: audit 2024-03-04T15:55:56.717581+0000 mon.a (mon.0) 1519 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi117-63915-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:57.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:57 smithi117 bash[21480]: cluster 2024-03-04T15:55:56.726021+0000 mon.a (mon.0) 1520 : cluster 0 osdmap e168: 8 total, 8 up, 8 in 2024-03-04T15:55:57.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:57 smithi117 bash[28843]: audit 2024-03-04T15:55:56.157875+0000 mon.a (mon.0) 1518 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:57.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:57 smithi117 bash[28843]: audit 2024-03-04T15:55:56.717581+0000 mon.a (mon.0) 1519 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi117-63915-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:57.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:57 smithi117 bash[28843]: cluster 2024-03-04T15:55:56.726021+0000 mon.a (mon.0) 1520 : cluster 0 osdmap e168: 8 total, 8 up, 8 in 2024-03-04T15:55:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:57 smithi151 bash[23756]: audit 2024-03-04T15:55:56.157875+0000 mon.a (mon.0) 1518 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:57 smithi151 bash[23756]: audit 2024-03-04T15:55:56.717581+0000 mon.a (mon.0) 1519 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi117-63915-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:57 smithi151 bash[23756]: cluster 2024-03-04T15:55:56.726021+0000 mon.a (mon.0) 1520 : cluster 0 osdmap e168: 8 total, 8 up, 8 in 2024-03-04T15:55:58.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:58 smithi117 bash[21480]: audit 2024-03-04T15:55:56.562907+0000 mgr.y (mgr.24428) 262 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:55:58.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:58 smithi117 bash[21480]: cluster 2024-03-04T15:55:56.841458+0000 mgr.y (mgr.24428) 263 : cluster 0 pgmap v258: 340 pgs: 40 creating+peering, 300 active+clean; 592 KiB data, 551 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T15:55:58.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:58 smithi117 bash[21480]: audit 2024-03-04T15:55:57.192051+0000 mon.a (mon.0) 1521 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:58.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:58 smithi117 bash[21480]: cluster 2024-03-04T15:55:57.693128+0000 mon.a (mon.0) 1522 : cluster 3 Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:58.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:58 smithi117 bash[21480]: cluster 2024-03-04T15:55:57.728854+0000 mon.a (mon.0) 1523 : cluster 0 osdmap e169: 8 total, 8 up, 8 in 2024-03-04T15:55:58.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:58 smithi117 bash[21480]: audit 2024-03-04T15:55:57.733756+0000 mon.c (mon.2) 258 : audit 1 from='client.? 172.21.15.117:0/1012407569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi117-63923-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:58.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:58 smithi117 bash[21480]: audit 2024-03-04T15:55:57.734677+0000 mon.a (mon.0) 1524 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi117-63923-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:58.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:58 smithi117 bash[21480]: audit 2024-03-04T15:55:57.738092+0000 mon.c (mon.2) 259 : audit 1 from='client.? 172.21.15.117:0/3111646600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:58.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:58 smithi117 bash[21480]: audit 2024-03-04T15:55:57.738580+0000 mon.a (mon.0) 1525 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:58.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:58 smithi117 bash[28843]: audit 2024-03-04T15:55:56.562907+0000 mgr.y (mgr.24428) 262 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:55:58.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:58 smithi117 bash[28843]: cluster 2024-03-04T15:55:56.841458+0000 mgr.y (mgr.24428) 263 : cluster 0 pgmap v258: 340 pgs: 40 creating+peering, 300 active+clean; 592 KiB data, 551 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T15:55:58.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:58 smithi117 bash[28843]: audit 2024-03-04T15:55:57.192051+0000 mon.a (mon.0) 1521 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:58.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:58 smithi117 bash[28843]: cluster 2024-03-04T15:55:57.693128+0000 mon.a (mon.0) 1522 : cluster 3 Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:58.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:58 smithi117 bash[28843]: cluster 2024-03-04T15:55:57.728854+0000 mon.a (mon.0) 1523 : cluster 0 osdmap e169: 8 total, 8 up, 8 in 2024-03-04T15:55:58.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:58 smithi117 bash[28843]: audit 2024-03-04T15:55:57.733756+0000 mon.c (mon.2) 258 : audit 1 from='client.? 172.21.15.117:0/1012407569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi117-63923-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:58.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:58 smithi117 bash[28843]: audit 2024-03-04T15:55:57.734677+0000 mon.a (mon.0) 1524 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi117-63923-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:58.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:58 smithi117 bash[28843]: audit 2024-03-04T15:55:57.738092+0000 mon.c (mon.2) 259 : audit 1 from='client.? 172.21.15.117:0/3111646600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:58.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:58 smithi117 bash[28843]: audit 2024-03-04T15:55:57.738580+0000 mon.a (mon.0) 1525 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:58 smithi151 bash[23756]: audit 2024-03-04T15:55:56.562907+0000 mgr.y (mgr.24428) 262 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:55:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:58 smithi151 bash[23756]: cluster 2024-03-04T15:55:56.841458+0000 mgr.y (mgr.24428) 263 : cluster 0 pgmap v258: 340 pgs: 40 creating+peering, 300 active+clean; 592 KiB data, 551 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T15:55:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:58 smithi151 bash[23756]: audit 2024-03-04T15:55:57.192051+0000 mon.a (mon.0) 1521 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:58 smithi151 bash[23756]: cluster 2024-03-04T15:55:57.693128+0000 mon.a (mon.0) 1522 : cluster 3 Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:55:58.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:58 smithi151 bash[23756]: cluster 2024-03-04T15:55:57.728854+0000 mon.a (mon.0) 1523 : cluster 0 osdmap e169: 8 total, 8 up, 8 in 2024-03-04T15:55:58.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:58 smithi151 bash[23756]: audit 2024-03-04T15:55:57.733756+0000 mon.c (mon.2) 258 : audit 1 from='client.? 172.21.15.117:0/1012407569' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi117-63923-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:58.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:58 smithi151 bash[23756]: audit 2024-03-04T15:55:57.734677+0000 mon.a (mon.0) 1524 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi117-63923-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:58.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:58 smithi151 bash[23756]: audit 2024-03-04T15:55:57.738092+0000 mon.c (mon.2) 259 : audit 1 from='client.? 172.21.15.117:0/3111646600' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:58.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:58 smithi151 bash[23756]: audit 2024-03-04T15:55:57.738580+0000 mon.a (mon.0) 1525 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:59.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:59 smithi117 bash[21480]: audit 2024-03-04T15:55:58.206502+0000 mon.a (mon.0) 1526 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:59.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:59 smithi117 bash[21480]: audit 2024-03-04T15:55:58.228924+0000 mon.c (mon.2) 260 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:55:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:59 smithi117 bash[21480]: audit 2024-03-04T15:55:58.725184+0000 mon.a (mon.0) 1527 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi117-63923-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:59 smithi117 bash[21480]: audit 2024-03-04T15:55:58.725470+0000 mon.a (mon.0) 1528 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36"}]': finished 2024-03-04T15:55:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:59 smithi117 bash[21480]: cluster 2024-03-04T15:55:58.736303+0000 mon.a (mon.0) 1529 : cluster 0 osdmap e170: 8 total, 8 up, 8 in 2024-03-04T15:55:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:59 smithi117 bash[21480]: audit 2024-03-04T15:55:58.740541+0000 mon.c (mon.2) 261 : audit 1 from='client.? 172.21.15.117:0/3111646600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:59 smithi117 bash[21480]: audit 2024-03-04T15:55:58.741977+0000 mon.a (mon.0) 1530 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:59 smithi117 bash[21480]: audit 2024-03-04T15:55:58.742891+0000 mon.a (mon.0) 1531 : audit 1 from='client.? 172.21.15.117:0/1862332692' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi117-63915-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:55:59 smithi117 bash[21480]: audit 2024-03-04T15:55:59.208212+0000 mon.a (mon.0) 1532 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:59.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:59 smithi117 bash[28843]: audit 2024-03-04T15:55:58.206502+0000 mon.a (mon.0) 1526 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:59.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:59 smithi117 bash[28843]: audit 2024-03-04T15:55:58.228924+0000 mon.c (mon.2) 260 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:55:59.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:59 smithi117 bash[28843]: audit 2024-03-04T15:55:58.725184+0000 mon.a (mon.0) 1527 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi117-63923-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:59.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:59 smithi117 bash[28843]: audit 2024-03-04T15:55:58.725470+0000 mon.a (mon.0) 1528 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36"}]': finished 2024-03-04T15:55:59.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:59 smithi117 bash[28843]: cluster 2024-03-04T15:55:58.736303+0000 mon.a (mon.0) 1529 : cluster 0 osdmap e170: 8 total, 8 up, 8 in 2024-03-04T15:55:59.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:59 smithi117 bash[28843]: audit 2024-03-04T15:55:58.740541+0000 mon.c (mon.2) 261 : audit 1 from='client.? 172.21.15.117:0/3111646600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:59.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:59 smithi117 bash[28843]: audit 2024-03-04T15:55:58.741977+0000 mon.a (mon.0) 1530 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:59.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:59 smithi117 bash[28843]: audit 2024-03-04T15:55:58.742891+0000 mon.a (mon.0) 1531 : audit 1 from='client.? 172.21.15.117:0/1862332692' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi117-63915-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:59.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:55:59 smithi117 bash[28843]: audit 2024-03-04T15:55:59.208212+0000 mon.a (mon.0) 1532 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:59 smithi151 bash[23756]: audit 2024-03-04T15:55:58.206502+0000 mon.a (mon.0) 1526 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:59 smithi151 bash[23756]: audit 2024-03-04T15:55:58.228924+0000 mon.c (mon.2) 260 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:55:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:59 smithi151 bash[23756]: audit 2024-03-04T15:55:58.725184+0000 mon.a (mon.0) 1527 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi117-63923-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:55:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:59 smithi151 bash[23756]: audit 2024-03-04T15:55:58.725470+0000 mon.a (mon.0) 1528 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi117-63991-36"}]': finished 2024-03-04T15:55:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:59 smithi151 bash[23756]: cluster 2024-03-04T15:55:58.736303+0000 mon.a (mon.0) 1529 : cluster 0 osdmap e170: 8 total, 8 up, 8 in 2024-03-04T15:55:59.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:59 smithi151 bash[23756]: audit 2024-03-04T15:55:58.740541+0000 mon.c (mon.2) 261 : audit 1 from='client.? 172.21.15.117:0/3111646600' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:59.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:59 smithi151 bash[23756]: audit 2024-03-04T15:55:58.741977+0000 mon.a (mon.0) 1530 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi117-63991-36"}]: dispatch 2024-03-04T15:55:59.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:59 smithi151 bash[23756]: audit 2024-03-04T15:55:58.742891+0000 mon.a (mon.0) 1531 : audit 1 from='client.? 172.21.15.117:0/1862332692' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi117-63915-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:55:59.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:55:59 smithi151 bash[23756]: audit 2024-03-04T15:55:59.208212+0000 mon.a (mon.0) 1532 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:55:59.757 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_ snapshots: Running main() from gmock_main.cc 2024-03-04T15:55:59.757 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [==========] Running 11 tests from 2 test suites. 2024-03-04T15:55:59.757 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [----------] Global test environment set-up. 2024-03-04T15:55:59.757 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots 2024-03-04T15:55:59.757 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapList 2024-03-04T15:55:59.757 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapList (23562 ms) 2024-03-04T15:55:59.757 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapRemove 2024-03-04T15:55:59.758 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapRemove (9190 ms) 2024-03-04T15:55:59.758 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ RUN ] NeoRadosSnapshots.Rollback 2024-03-04T15:55:59.758 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ OK ] NeoRadosSnapshots.Rollback (10161 ms) 2024-03-04T15:55:59.758 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapGetName 2024-03-04T15:55:59.758 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapGetName (9762 ms) 2024-03-04T15:55:59.758 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ RUN ] NeoRadosSnapshots.SnapCreateRemove 2024-03-04T15:55:59.758 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ OK ] NeoRadosSnapshots.SnapCreateRemove (15806 ms) 2024-03-04T15:55:59.758 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [----------] 5 tests from NeoRadosSnapshots (68481 ms total) 2024-03-04T15:55:59.758 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: 2024-03-04T15:55:59.758 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps 2024-03-04T15:55:59.758 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Snap 2024-03-04T15:55:59.758 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Snap (6335 ms) 2024-03-04T15:55:59.758 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Rollback 2024-03-04T15:55:59.758 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Rollback (6886 ms) 2024-03-04T15:55:59.758 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.SnapOverlap 2024-03-04T15:55:59.759 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.SnapOverlap (7413 ms) 2024-03-04T15:55:59.759 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.Bug11677 2024-03-04T15:55:59.759 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.Bug11677 (6483 ms) 2024-03-04T15:55:59.759 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.OrderSnap 2024-03-04T15:55:59.759 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.OrderSnap (3829 ms) 2024-03-04T15:55:59.759 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ RUN ] NeoRadosSelfManagedSnaps.ReusePurgedSnap 2024-03-04T15:55:59.759 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: Deleting snap 3 in pool ReusePurgedSnapsmithi117-64758-11. 2024-03-04T15:55:59.759 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: Waiting for snaps to purge. 2024-03-04T15:55:59.759 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ OK ] NeoRadosSelfManagedSnaps.ReusePurgedSnap (20031 ms) 2024-03-04T15:55:59.759 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [----------] 6 tests from NeoRadosSelfManagedSnaps (50977 ms total) 2024-03-04T15:55:59.759 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: 2024-03-04T15:55:59.759 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [----------] Global test environment tear-down 2024-03-04T15:55:59.759 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [==========] 11 tests from 2 test suites ran. (119458 ms total) 2024-03-04T15:55:59.759 INFO:tasks.workunit.client.0.smithi117.stdout: snapshots: [ PASSED ] 11 tests. 2024-03-04T15:55:59.768 INFO:tasks.workunit.client.0.smithi117.stdout:(73698 ms total) 2024-03-04T15:55:59.768 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: 2024-03-04T15:55:59.768 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-03-04T15:55:59.768 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-03-04T15:55:59.768 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (932 ms) 2024-03-04T15:55:59.768 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (932 ms total) 2024-03-04T15:55:59.768 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: 2024-03-04T15:55:59.769 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-03-04T15:55:59.769 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-03-04T15:55:59.769 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (67 ms) 2024-03-04T15:55:59.769 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-03-04T15:55:59.769 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (9 ms) 2024-03-04T15:55:59.769 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (76 ms total) 2024-03-04T15:55:59.769 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: 2024-03-04T15:55:59.769 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-03-04T15:55:59.769 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-03-04T15:55:59.769 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (10 ms) 2024-03-04T15:55:59.769 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-03-04T15:55:59.769 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (5 ms) 2024-03-04T15:55:59.769 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (15 ms total) 2024-03-04T15:55:59.769 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: 2024-03-04T15:55:59.769 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-03-04T15:55:59.770 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-03-04T15:55:59.770 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (71 ms) 2024-03-04T15:55:59.770 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-03-04T15:55:59.770 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (8 ms) 2024-03-04T15:55:59.770 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (79 ms total) 2024-03-04T15:55:59.770 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: 2024-03-04T15:55:59.770 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-03-04T15:55:59.770 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-03-04T15:55:59.770 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1053 ms) 2024-03-04T15:55:59.770 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1053 ms total) 2024-03-04T15:55:59.770 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: 2024-03-04T15:55:59.770 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-03-04T15:55:59.770 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (119812 ms total) 2024-03-04T15:55:59.770 INFO:tasks.workunit.client.0.smithi117.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-03-04T15:56:00.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:00 smithi117 bash[28843]: cluster 2024-03-04T15:55:58.842349+0000 mgr.y (mgr.24428) 264 : cluster 0 pgmap v261: 364 pgs: 64 unknown, 300 active+clean; 592 KiB data, 551 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T15:56:00.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:00 smithi117 bash[28843]: audit 2024-03-04T15:55:59.735837+0000 mon.a (mon.0) 1533 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi117-63991-36"}]': finished 2024-03-04T15:56:00.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:00 smithi117 bash[28843]: audit 2024-03-04T15:55:59.736050+0000 mon.a (mon.0) 1534 : audit 1 from='client.? 172.21.15.117:0/1862332692' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi117-63915-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:00.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:00 smithi117 bash[28843]: cluster 2024-03-04T15:55:59.743963+0000 mon.a (mon.0) 1535 : cluster 0 osdmap e171: 8 total, 8 up, 8 in 2024-03-04T15:56:00.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:00 smithi117 bash[28843]: audit 2024-03-04T15:55:59.752182+0000 mon.c (mon.2) 262 : audit 1 from='client.? 172.21.15.117:0/3950150976' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:56:00.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:00 smithi117 bash[28843]: audit 2024-03-04T15:55:59.755821+0000 mon.a (mon.0) 1536 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:56:00.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:00 smithi117 bash[21480]: cluster 2024-03-04T15:55:58.842349+0000 mgr.y (mgr.24428) 264 : cluster 0 pgmap v261: 364 pgs: 64 unknown, 300 active+clean; 592 KiB data, 551 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T15:56:00.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:00 smithi117 bash[21480]: audit 2024-03-04T15:55:59.735837+0000 mon.a (mon.0) 1533 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi117-63991-36"}]': finished 2024-03-04T15:56:00.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:00 smithi117 bash[21480]: audit 2024-03-04T15:55:59.736050+0000 mon.a (mon.0) 1534 : audit 1 from='client.? 172.21.15.117:0/1862332692' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi117-63915-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:00.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:00 smithi117 bash[21480]: cluster 2024-03-04T15:55:59.743963+0000 mon.a (mon.0) 1535 : cluster 0 osdmap e171: 8 total, 8 up, 8 in 2024-03-04T15:56:00.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:00 smithi117 bash[21480]: audit 2024-03-04T15:55:59.752182+0000 mon.c (mon.2) 262 : audit 1 from='client.? 172.21.15.117:0/3950150976' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:56:00.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:00 smithi117 bash[21480]: audit 2024-03-04T15:55:59.755821+0000 mon.a (mon.0) 1536 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:56:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:00 smithi151 bash[23756]: cluster 2024-03-04T15:55:58.842349+0000 mgr.y (mgr.24428) 264 : cluster 0 pgmap v261: 364 pgs: 64 unknown, 300 active+clean; 592 KiB data, 551 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T15:56:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:00 smithi151 bash[23756]: audit 2024-03-04T15:55:59.735837+0000 mon.a (mon.0) 1533 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi117-63991-36"}]': finished 2024-03-04T15:56:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:00 smithi151 bash[23756]: audit 2024-03-04T15:55:59.736050+0000 mon.a (mon.0) 1534 : audit 1 from='client.? 172.21.15.117:0/1862332692' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi117-63915-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:00 smithi151 bash[23756]: cluster 2024-03-04T15:55:59.743963+0000 mon.a (mon.0) 1535 : cluster 0 osdmap e171: 8 total, 8 up, 8 in 2024-03-04T15:56:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:00 smithi151 bash[23756]: audit 2024-03-04T15:55:59.752182+0000 mon.c (mon.2) 262 : audit 1 from='client.? 172.21.15.117:0/3950150976' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:56:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:00 smithi151 bash[23756]: audit 2024-03-04T15:55:59.755821+0000 mon.a (mon.0) 1536 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:56:01.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:01 smithi117 bash[21480]: audit 2024-03-04T15:56:00.249491+0000 mon.a (mon.0) 1537 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:01.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:01 smithi117 bash[21480]: audit 2024-03-04T15:56:00.740382+0000 mon.a (mon.0) 1538 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]': finished 2024-03-04T15:56:01.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:01 smithi117 bash[21480]: audit 2024-03-04T15:56:00.758876+0000 mon.c (mon.2) 263 : audit 1 from='client.? 172.21.15.117:0/3950150976' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:56:01.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:01 smithi117 bash[21480]: cluster 2024-03-04T15:56:00.762522+0000 mon.a (mon.0) 1539 : cluster 0 osdmap e172: 8 total, 8 up, 8 in 2024-03-04T15:56:01.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:01 smithi117 bash[21480]: audit 2024-03-04T15:56:00.764071+0000 mon.c (mon.2) 264 : audit 1 from='client.? 172.21.15.117:0/550297396' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi117-63923-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:01.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:01 smithi117 bash[21480]: audit 2024-03-04T15:56:00.765827+0000 mon.a (mon.0) 1540 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:56:01.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:01 smithi117 bash[21480]: audit 2024-03-04T15:56:00.769383+0000 mon.a (mon.0) 1541 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi117-63923-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:01.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:01 smithi117 bash[28843]: audit 2024-03-04T15:56:00.249491+0000 mon.a (mon.0) 1537 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:01.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:01 smithi117 bash[28843]: audit 2024-03-04T15:56:00.740382+0000 mon.a (mon.0) 1538 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]': finished 2024-03-04T15:56:01.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:01 smithi117 bash[28843]: audit 2024-03-04T15:56:00.758876+0000 mon.c (mon.2) 263 : audit 1 from='client.? 172.21.15.117:0/3950150976' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:56:01.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:01 smithi117 bash[28843]: cluster 2024-03-04T15:56:00.762522+0000 mon.a (mon.0) 1539 : cluster 0 osdmap e172: 8 total, 8 up, 8 in 2024-03-04T15:56:01.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:01 smithi117 bash[28843]: audit 2024-03-04T15:56:00.764071+0000 mon.c (mon.2) 264 : audit 1 from='client.? 172.21.15.117:0/550297396' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi117-63923-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:01.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:01 smithi117 bash[28843]: audit 2024-03-04T15:56:00.765827+0000 mon.a (mon.0) 1540 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:56:01.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:01 smithi117 bash[28843]: audit 2024-03-04T15:56:00.769383+0000 mon.a (mon.0) 1541 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi117-63923-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:01 smithi151 bash[23756]: audit 2024-03-04T15:56:00.249491+0000 mon.a (mon.0) 1537 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:01 smithi151 bash[23756]: audit 2024-03-04T15:56:00.740382+0000 mon.a (mon.0) 1538 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi117-64156-16"}]': finished 2024-03-04T15:56:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:01 smithi151 bash[23756]: audit 2024-03-04T15:56:00.758876+0000 mon.c (mon.2) 263 : audit 1 from='client.? 172.21.15.117:0/3950150976' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:56:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:01 smithi151 bash[23756]: cluster 2024-03-04T15:56:00.762522+0000 mon.a (mon.0) 1539 : cluster 0 osdmap e172: 8 total, 8 up, 8 in 2024-03-04T15:56:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:01 smithi151 bash[23756]: audit 2024-03-04T15:56:00.764071+0000 mon.c (mon.2) 264 : audit 1 from='client.? 172.21.15.117:0/550297396' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi117-63923-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:01 smithi151 bash[23756]: audit 2024-03-04T15:56:00.765827+0000 mon.a (mon.0) 1540 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi117-64156-16"}]: dispatch 2024-03-04T15:56:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:01 smithi151 bash[23756]: audit 2024-03-04T15:56:00.769383+0000 mon.a (mon.0) 1541 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi117-63923-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:02.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[21480]: cluster 2024-03-04T15:56:00.842865+0000 mgr.y (mgr.24428) 265 : cluster 0 pgmap v264: 292 pgs: 32 unknown, 260 active+clean; 592 KiB data, 551 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:56:02.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[21480]: audit 2024-03-04T15:56:01.251340+0000 mon.a (mon.0) 1542 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:02.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[21480]: audit 2024-03-04T15:56:01.751746+0000 mon.a (mon.0) 1543 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi117-64156-16"}]': finished 2024-03-04T15:56:02.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[21480]: audit 2024-03-04T15:56:01.751958+0000 mon.a (mon.0) 1544 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi117-63923-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:02.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[21480]: cluster 2024-03-04T15:56:01.764211+0000 mon.a (mon.0) 1545 : cluster 0 osdmap e173: 8 total, 8 up, 8 in 2024-03-04T15:56:02.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[21480]: audit 2024-03-04T15:56:01.768054+0000 mon.c (mon.2) 265 : audit 1 from='client.? 172.21.15.117:0/3966603066' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi117-63915-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:02.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[21480]: audit 2024-03-04T15:56:01.774539+0000 mon.a (mon.0) 1546 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi117-63915-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:02.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[21480]: audit 2024-03-04T15:56:01.812855+0000 mon.a (mon.0) 1547 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]: dispatch 2024-03-04T15:56:02.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[21480]: audit 2024-03-04T15:56:01.815653+0000 mon.a (mon.0) 1548 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]: dispatch 2024-03-04T15:56:02.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[21480]: audit 2024-03-04T15:56:01.816048+0000 mon.a (mon.0) 1549 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:02.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[21480]: audit 2024-03-04T15:56:02.252616+0000 mon.a (mon.0) 1550 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:02.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[28843]: cluster 2024-03-04T15:56:00.842865+0000 mgr.y (mgr.24428) 265 : cluster 0 pgmap v264: 292 pgs: 32 unknown, 260 active+clean; 592 KiB data, 551 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:56:02.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[28843]: audit 2024-03-04T15:56:01.251340+0000 mon.a (mon.0) 1542 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:02.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[28843]: audit 2024-03-04T15:56:01.751746+0000 mon.a (mon.0) 1543 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi117-64156-16"}]': finished 2024-03-04T15:56:02.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[28843]: audit 2024-03-04T15:56:01.751958+0000 mon.a (mon.0) 1544 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi117-63923-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:02.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[28843]: cluster 2024-03-04T15:56:01.764211+0000 mon.a (mon.0) 1545 : cluster 0 osdmap e173: 8 total, 8 up, 8 in 2024-03-04T15:56:02.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[28843]: audit 2024-03-04T15:56:01.768054+0000 mon.c (mon.2) 265 : audit 1 from='client.? 172.21.15.117:0/3966603066' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi117-63915-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:02.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[28843]: audit 2024-03-04T15:56:01.774539+0000 mon.a (mon.0) 1546 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi117-63915-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:02.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[28843]: audit 2024-03-04T15:56:01.812855+0000 mon.a (mon.0) 1547 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]: dispatch 2024-03-04T15:56:02.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[28843]: audit 2024-03-04T15:56:01.815653+0000 mon.a (mon.0) 1548 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]: dispatch 2024-03-04T15:56:02.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[28843]: audit 2024-03-04T15:56:01.816048+0000 mon.a (mon.0) 1549 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:02.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:02 smithi117 bash[28843]: audit 2024-03-04T15:56:02.252616+0000 mon.a (mon.0) 1550 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:02 smithi151 bash[23756]: cluster 2024-03-04T15:56:00.842865+0000 mgr.y (mgr.24428) 265 : cluster 0 pgmap v264: 292 pgs: 32 unknown, 260 active+clean; 592 KiB data, 551 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:56:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:02 smithi151 bash[23756]: audit 2024-03-04T15:56:01.251340+0000 mon.a (mon.0) 1542 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:02 smithi151 bash[23756]: audit 2024-03-04T15:56:01.751746+0000 mon.a (mon.0) 1543 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi117-64156-16"}]': finished 2024-03-04T15:56:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:02 smithi151 bash[23756]: audit 2024-03-04T15:56:01.751958+0000 mon.a (mon.0) 1544 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi117-63923-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:02.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:02 smithi151 bash[23756]: cluster 2024-03-04T15:56:01.764211+0000 mon.a (mon.0) 1545 : cluster 0 osdmap e173: 8 total, 8 up, 8 in 2024-03-04T15:56:02.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:02 smithi151 bash[23756]: audit 2024-03-04T15:56:01.768054+0000 mon.c (mon.2) 265 : audit 1 from='client.? 172.21.15.117:0/3966603066' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi117-63915-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:02.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:02 smithi151 bash[23756]: audit 2024-03-04T15:56:01.774539+0000 mon.a (mon.0) 1546 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi117-63915-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:02.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:02 smithi151 bash[23756]: audit 2024-03-04T15:56:01.812855+0000 mon.a (mon.0) 1547 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]: dispatch 2024-03-04T15:56:02.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:02 smithi151 bash[23756]: audit 2024-03-04T15:56:01.815653+0000 mon.a (mon.0) 1548 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]: dispatch 2024-03-04T15:56:02.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:02 smithi151 bash[23756]: audit 2024-03-04T15:56:01.816048+0000 mon.a (mon.0) 1549 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:02.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:02 smithi151 bash[23756]: audit 2024-03-04T15:56:02.252616+0000 mon.a (mon.0) 1550 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:03.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:03 smithi117 bash[21480]: cluster 2024-03-04T15:56:02.694785+0000 mon.a (mon.0) 1551 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:03.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:03 smithi117 bash[21480]: audit 2024-03-04T15:56:02.755085+0000 mon.a (mon.0) 1552 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi117-63915-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:03.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:03 smithi117 bash[21480]: audit 2024-03-04T15:56:02.755280+0000 mon.a (mon.0) 1553 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:03.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:03 smithi117 bash[21480]: cluster 2024-03-04T15:56:02.761987+0000 mon.a (mon.0) 1554 : cluster 0 osdmap e174: 8 total, 8 up, 8 in 2024-03-04T15:56:03.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:03 smithi117 bash[21480]: audit 2024-03-04T15:56:02.763658+0000 mon.a (mon.0) 1555 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]: dispatch 2024-03-04T15:56:03.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:03 smithi117 bash[21480]: audit 2024-03-04T15:56:03.254198+0000 mon.a (mon.0) 1556 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:03.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:03 smithi117 bash[28843]: cluster 2024-03-04T15:56:02.694785+0000 mon.a (mon.0) 1551 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:03.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:03 smithi117 bash[28843]: audit 2024-03-04T15:56:02.755085+0000 mon.a (mon.0) 1552 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi117-63915-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:03.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:03 smithi117 bash[28843]: audit 2024-03-04T15:56:02.755280+0000 mon.a (mon.0) 1553 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:03.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:03 smithi117 bash[28843]: cluster 2024-03-04T15:56:02.761987+0000 mon.a (mon.0) 1554 : cluster 0 osdmap e174: 8 total, 8 up, 8 in 2024-03-04T15:56:03.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:03 smithi117 bash[28843]: audit 2024-03-04T15:56:02.763658+0000 mon.a (mon.0) 1555 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]: dispatch 2024-03-04T15:56:03.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:03 smithi117 bash[28843]: audit 2024-03-04T15:56:03.254198+0000 mon.a (mon.0) 1556 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:03 smithi151 bash[23756]: cluster 2024-03-04T15:56:02.694785+0000 mon.a (mon.0) 1551 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:03 smithi151 bash[23756]: audit 2024-03-04T15:56:02.755085+0000 mon.a (mon.0) 1552 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi117-63915-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:03 smithi151 bash[23756]: audit 2024-03-04T15:56:02.755280+0000 mon.a (mon.0) 1553 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:03 smithi151 bash[23756]: cluster 2024-03-04T15:56:02.761987+0000 mon.a (mon.0) 1554 : cluster 0 osdmap e174: 8 total, 8 up, 8 in 2024-03-04T15:56:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:03 smithi151 bash[23756]: audit 2024-03-04T15:56:02.763658+0000 mon.a (mon.0) 1555 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]: dispatch 2024-03-04T15:56:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:03 smithi151 bash[23756]: audit 2024-03-04T15:56:03.254198+0000 mon.a (mon.0) 1556 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:04.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:04 smithi151 bash[23756]: cluster 2024-03-04T15:56:02.843689+0000 mgr.y (mgr.24428) 266 : cluster 0 pgmap v267: 292 pgs: 32 unknown, 260 active+clean; 592 KiB data, 556 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T15:56:04.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:04 smithi151 bash[23756]: cluster 2024-03-04T15:56:04.228231+0000 mon.a (mon.0) 1557 : cluster 0 osdmap e175: 8 total, 8 up, 8 in 2024-03-04T15:56:04.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:04 smithi151 bash[23756]: audit 2024-03-04T15:56:04.230191+0000 mon.a (mon.0) 1558 : audit 1 from='client.? 172.21.15.117:0/3764263539' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi117-63923-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:05.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:04 smithi117 bash[28843]: cluster 2024-03-04T15:56:02.843689+0000 mgr.y (mgr.24428) 266 : cluster 0 pgmap v267: 292 pgs: 32 unknown, 260 active+clean; 592 KiB data, 556 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T15:56:05.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:04 smithi117 bash[28843]: cluster 2024-03-04T15:56:04.228231+0000 mon.a (mon.0) 1557 : cluster 0 osdmap e175: 8 total, 8 up, 8 in 2024-03-04T15:56:05.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:04 smithi117 bash[28843]: audit 2024-03-04T15:56:04.230191+0000 mon.a (mon.0) 1558 : audit 1 from='client.? 172.21.15.117:0/3764263539' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi117-63923-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:05.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:04 smithi117 bash[21480]: cluster 2024-03-04T15:56:02.843689+0000 mgr.y (mgr.24428) 266 : cluster 0 pgmap v267: 292 pgs: 32 unknown, 260 active+clean; 592 KiB data, 556 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T15:56:05.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:04 smithi117 bash[21480]: cluster 2024-03-04T15:56:04.228231+0000 mon.a (mon.0) 1557 : cluster 0 osdmap e175: 8 total, 8 up, 8 in 2024-03-04T15:56:05.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:04 smithi117 bash[21480]: audit 2024-03-04T15:56:04.230191+0000 mon.a (mon.0) 1558 : audit 1 from='client.? 172.21.15.117:0/3764263539' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi117-63923-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:05.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:05 smithi151 bash[23756]: audit 2024-03-04T15:56:04.656389+0000 mon.a (mon.0) 1559 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:05.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:05 smithi151 bash[23756]: audit 2024-03-04T15:56:04.771834+0000 mon.a (mon.0) 1560 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]': finished 2024-03-04T15:56:05.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:05 smithi151 bash[23756]: audit 2024-03-04T15:56:04.772035+0000 mon.a (mon.0) 1561 : audit 1 from='client.? 172.21.15.117:0/3764263539' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi117-63923-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:05.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:05 smithi151 bash[23756]: cluster 2024-03-04T15:56:04.777667+0000 mon.a (mon.0) 1562 : cluster 0 osdmap e176: 8 total, 8 up, 8 in 2024-03-04T15:56:05.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:05 smithi151 bash[23756]: audit 2024-03-04T15:56:04.779813+0000 mon.a (mon.0) 1563 : audit 1 from='client.? 172.21.15.117:0/1269143627' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi117-63915-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:05.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:05 smithi151 bash[23756]: cluster 2024-03-04T15:56:04.844338+0000 mgr.y (mgr.24428) 267 : cluster 0 pgmap v270: 332 pgs: 72 unknown, 260 active+clean; 592 KiB data, 556 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:56:06.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:05 smithi117 bash[21480]: audit 2024-03-04T15:56:04.656389+0000 mon.a (mon.0) 1559 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:06.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:05 smithi117 bash[21480]: audit 2024-03-04T15:56:04.771834+0000 mon.a (mon.0) 1560 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]': finished 2024-03-04T15:56:06.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:05 smithi117 bash[21480]: audit 2024-03-04T15:56:04.772035+0000 mon.a (mon.0) 1561 : audit 1 from='client.? 172.21.15.117:0/3764263539' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi117-63923-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:06.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:05 smithi117 bash[21480]: cluster 2024-03-04T15:56:04.777667+0000 mon.a (mon.0) 1562 : cluster 0 osdmap e176: 8 total, 8 up, 8 in 2024-03-04T15:56:06.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:05 smithi117 bash[21480]: audit 2024-03-04T15:56:04.779813+0000 mon.a (mon.0) 1563 : audit 1 from='client.? 172.21.15.117:0/1269143627' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi117-63915-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:06.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:05 smithi117 bash[21480]: cluster 2024-03-04T15:56:04.844338+0000 mgr.y (mgr.24428) 267 : cluster 0 pgmap v270: 332 pgs: 72 unknown, 260 active+clean; 592 KiB data, 556 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:56:06.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:05 smithi117 bash[28843]: audit 2024-03-04T15:56:04.656389+0000 mon.a (mon.0) 1559 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:06.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:05 smithi117 bash[28843]: audit 2024-03-04T15:56:04.771834+0000 mon.a (mon.0) 1560 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]': finished 2024-03-04T15:56:06.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:05 smithi117 bash[28843]: audit 2024-03-04T15:56:04.772035+0000 mon.a (mon.0) 1561 : audit 1 from='client.? 172.21.15.117:0/3764263539' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi117-63923-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:06.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:05 smithi117 bash[28843]: cluster 2024-03-04T15:56:04.777667+0000 mon.a (mon.0) 1562 : cluster 0 osdmap e176: 8 total, 8 up, 8 in 2024-03-04T15:56:06.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:05 smithi117 bash[28843]: audit 2024-03-04T15:56:04.779813+0000 mon.a (mon.0) 1563 : audit 1 from='client.? 172.21.15.117:0/1269143627' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi117-63915-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:06.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:05 smithi117 bash[28843]: cluster 2024-03-04T15:56:04.844338+0000 mgr.y (mgr.24428) 267 : cluster 0 pgmap v270: 332 pgs: 72 unknown, 260 active+clean; 592 KiB data, 556 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:56:06.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:56:06 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:56:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:56:06.804 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: Running main() from gmock_main.cc 2024-03-04T15:56:06.804 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [==========] Running 42 tests from 2 test suites. 2024-03-04T15:56:06.804 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [----------] Global test environment set-up. 2024-03-04T15:56:06.804 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [----------] 26 tests from LibRadosAio 2024-03-04T15:56:06.804 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-03-04T15:56:06.804 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.TooBig (16856 ms) 2024-03-04T15:56:06.804 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-03-04T15:56:06.804 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (14825 ms) 2024-03-04T15:56:06.804 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-03-04T15:56:06.804 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (6490 ms) 2024-03-04T15:56:06.804 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-03-04T15:56:06.805 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (5162 ms) 2024-03-04T15:56:06.805 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-03-04T15:56:06.805 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (8386 ms) 2024-03-04T15:56:06.805 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-03-04T15:56:06.805 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (8629 ms) 2024-03-04T15:56:06.805 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-03-04T15:56:06.805 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (5591 ms) 2024-03-04T15:56:06.805 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-03-04T15:56:06.805 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (5017 ms) 2024-03-04T15:56:06.805 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-03-04T15:56:06.805 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3257 ms) 2024-03-04T15:56:06.805 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-03-04T15:56:06.805 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (2682 ms) 2024-03-04T15:56:06.806 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-03-04T15:56:06.806 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (3404 ms) 2024-03-04T15:56:06.806 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-03-04T15:56:06.806 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (3698 ms) 2024-03-04T15:56:06.806 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-03-04T15:56:06.806 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3024 ms) 2024-03-04T15:56:06.806 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-03-04T15:56:06.806 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (2508 ms) 2024-03-04T15:56:06.806 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-03-04T15:56:06.806 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.Flush (2986 ms) 2024-03-04T15:56:06.806 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-03-04T15:56:06.806 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (3495 ms) 2024-03-04T15:56:06.806 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-03-04T15:56:06.806 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (3841 ms) 2024-03-04T15:56:06.807 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-03-04T15:56:06.807 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (2998 ms) 2024-03-04T15:56:06.807 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-03-04T15:56:06.807 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3050 ms) 2024-03-04T15:56:06.807 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-03-04T15:56:06.807 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3456 ms) 2024-03-04T15:56:06.807 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.Operate2Mtime 2024-03-04T15:56:06.807 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.Operate2Mtime (2563 ms) 2024-03-04T15:56:06.807 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-03-04T15:56:06.807 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3345 ms) 2024-03-04T15:56:06.807 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-03-04T15:56:06.807 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (2591 ms) 2024-03-04T15:56:06.807 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-03-04T15:56:06.807 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3032 ms) 2024-03-04T15:56:06.807 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-03-04T15:56:06.808 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3460 ms) 2024-03-04T15:56:06.808 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-03-04T15:56:06.808 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (2576 ms) 2024-03-04T15:56:06.808 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [----------] 26 tests from LibRadosAio (126922 ms total) 2024-03-04T15:56:06.808 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: 2024-03-04T15:56:06.808 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-03-04T15:56:06.932 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:56:06 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:56:06.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:06 smithi151 bash[23756]: audit 2024-03-04T15:56:05.657892+0000 mon.a (mon.0) 1564 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:06.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:06 smithi151 bash[23756]: audit 2024-03-04T15:56:05.783715+0000 mon.a (mon.0) 1565 : audit 1 from='client.? 172.21.15.117:0/1269143627' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi117-63915-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:06.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:06 smithi151 bash[23756]: cluster 2024-03-04T15:56:05.789861+0000 mon.a (mon.0) 1566 : cluster 0 osdmap e177: 8 total, 8 up, 8 in 2024-03-04T15:56:07.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:06 smithi117 bash[21480]: audit 2024-03-04T15:56:05.657892+0000 mon.a (mon.0) 1564 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:07.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:06 smithi117 bash[21480]: audit 2024-03-04T15:56:05.783715+0000 mon.a (mon.0) 1565 : audit 1 from='client.? 172.21.15.117:0/1269143627' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi117-63915-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:07.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:06 smithi117 bash[21480]: cluster 2024-03-04T15:56:05.789861+0000 mon.a (mon.0) 1566 : cluster 0 osdmap e177: 8 total, 8 up, 8 in 2024-03-04T15:56:07.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:06 smithi117 bash[28843]: audit 2024-03-04T15:56:05.657892+0000 mon.a (mon.0) 1564 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:07.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:06 smithi117 bash[28843]: audit 2024-03-04T15:56:05.783715+0000 mon.a (mon.0) 1565 : audit 1 from='client.? 172.21.15.117:0/1269143627' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi117-63915-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:07.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:06 smithi117 bash[28843]: cluster 2024-03-04T15:56:05.789861+0000 mon.a (mon.0) 1566 : cluster 0 osdmap e177: 8 total, 8 up, 8 in 2024-03-04T15:56:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:07 smithi151 bash[23756]: audit 2024-03-04T15:56:06.574319+0000 mgr.y (mgr.24428) 268 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:56:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:07 smithi151 bash[23756]: audit 2024-03-04T15:56:06.660028+0000 mon.a (mon.0) 1567 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:07 smithi151 bash[23756]: cluster 2024-03-04T15:56:06.796738+0000 mon.a (mon.0) 1568 : cluster 0 osdmap e178: 8 total, 8 up, 8 in 2024-03-04T15:56:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:07 smithi151 bash[23756]: audit 2024-03-04T15:56:06.805719+0000 mon.a (mon.0) 1569 : audit 1 from='client.? 172.21.15.117:0/4020656977' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi117-63923-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:07 smithi151 bash[23756]: cluster 2024-03-04T15:56:06.844979+0000 mgr.y (mgr.24428) 269 : cluster 0 pgmap v273: 300 pgs: 32 unknown, 268 active+clean; 599 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-03-04T15:56:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:07 smithi151 bash[23756]: audit 2024-03-04T15:56:06.845747+0000 mon.c (mon.2) 266 : audit 1 from='client.? 172.21.15.117:0/3409444435' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:07 smithi151 bash[23756]: audit 2024-03-04T15:56:06.847159+0000 mon.a (mon.0) 1570 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:07 smithi151 bash[23756]: audit 2024-03-04T15:56:06.848113+0000 mon.c (mon.2) 267 : audit 1 from='client.? 172.21.15.117:0/3409444435' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:07 smithi151 bash[23756]: audit 2024-03-04T15:56:06.848598+0000 mon.a (mon.0) 1571 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:07 smithi151 bash[23756]: audit 2024-03-04T15:56:06.849207+0000 mon.c (mon.2) 268 : audit 1 from='client.? 172.21.15.117:0/3409444435' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi117-63915-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:07.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:07 smithi151 bash[23756]: audit 2024-03-04T15:56:06.849723+0000 mon.a (mon.0) 1572 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi117-63915-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:08.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[21480]: audit 2024-03-04T15:56:06.574319+0000 mgr.y (mgr.24428) 268 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:56:08.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[21480]: audit 2024-03-04T15:56:06.660028+0000 mon.a (mon.0) 1567 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:08.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[21480]: cluster 2024-03-04T15:56:06.796738+0000 mon.a (mon.0) 1568 : cluster 0 osdmap e178: 8 total, 8 up, 8 in 2024-03-04T15:56:08.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[21480]: audit 2024-03-04T15:56:06.805719+0000 mon.a (mon.0) 1569 : audit 1 from='client.? 172.21.15.117:0/4020656977' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi117-63923-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:08.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[21480]: cluster 2024-03-04T15:56:06.844979+0000 mgr.y (mgr.24428) 269 : cluster 0 pgmap v273: 300 pgs: 32 unknown, 268 active+clean; 599 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-03-04T15:56:08.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[21480]: audit 2024-03-04T15:56:06.845747+0000 mon.c (mon.2) 266 : audit 1 from='client.? 172.21.15.117:0/3409444435' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:08.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[21480]: audit 2024-03-04T15:56:06.847159+0000 mon.a (mon.0) 1570 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:08.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[21480]: audit 2024-03-04T15:56:06.848113+0000 mon.c (mon.2) 267 : audit 1 from='client.? 172.21.15.117:0/3409444435' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:08.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[21480]: audit 2024-03-04T15:56:06.848598+0000 mon.a (mon.0) 1571 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:08.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[21480]: audit 2024-03-04T15:56:06.849207+0000 mon.c (mon.2) 268 : audit 1 from='client.? 172.21.15.117:0/3409444435' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi117-63915-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:08.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[21480]: audit 2024-03-04T15:56:06.849723+0000 mon.a (mon.0) 1572 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi117-63915-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:08.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[28843]: audit 2024-03-04T15:56:06.574319+0000 mgr.y (mgr.24428) 268 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:56:08.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[28843]: audit 2024-03-04T15:56:06.660028+0000 mon.a (mon.0) 1567 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:08.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[28843]: cluster 2024-03-04T15:56:06.796738+0000 mon.a (mon.0) 1568 : cluster 0 osdmap e178: 8 total, 8 up, 8 in 2024-03-04T15:56:08.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[28843]: audit 2024-03-04T15:56:06.805719+0000 mon.a (mon.0) 1569 : audit 1 from='client.? 172.21.15.117:0/4020656977' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi117-63923-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:08.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[28843]: cluster 2024-03-04T15:56:06.844979+0000 mgr.y (mgr.24428) 269 : cluster 0 pgmap v273: 300 pgs: 32 unknown, 268 active+clean; 599 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-03-04T15:56:08.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[28843]: audit 2024-03-04T15:56:06.845747+0000 mon.c (mon.2) 266 : audit 1 from='client.? 172.21.15.117:0/3409444435' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:08.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[28843]: audit 2024-03-04T15:56:06.847159+0000 mon.a (mon.0) 1570 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:08.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[28843]: audit 2024-03-04T15:56:06.848113+0000 mon.c (mon.2) 267 : audit 1 from='client.? 172.21.15.117:0/3409444435' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:08.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[28843]: audit 2024-03-04T15:56:06.848598+0000 mon.a (mon.0) 1571 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:08.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[28843]: audit 2024-03-04T15:56:06.849207+0000 mon.c (mon.2) 268 : audit 1 from='client.? 172.21.15.117:0/3409444435' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi117-63915-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:08.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:07 smithi117 bash[28843]: audit 2024-03-04T15:56:06.849723+0000 mon.a (mon.0) 1572 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi117-63915-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:08.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:08 smithi151 bash[23756]: audit 2024-03-04T15:56:07.662185+0000 mon.a (mon.0) 1573 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:08.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:08 smithi151 bash[23756]: cluster 2024-03-04T15:56:07.696173+0000 mon.a (mon.0) 1574 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:08.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:08 smithi151 bash[23756]: audit 2024-03-04T15:56:07.797934+0000 mon.a (mon.0) 1575 : audit 1 from='client.? 172.21.15.117:0/4020656977' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi117-63923-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:08.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:08 smithi151 bash[23756]: audit 2024-03-04T15:56:07.798114+0000 mon.a (mon.0) 1576 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi117-63915-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:08.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:08 smithi151 bash[23756]: cluster 2024-03-04T15:56:07.804670+0000 mon.a (mon.0) 1577 : cluster 0 osdmap e179: 8 total, 8 up, 8 in 2024-03-04T15:56:08.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:08 smithi151 bash[23756]: audit 2024-03-04T15:56:07.806789+0000 mon.c (mon.2) 269 : audit 1 from='client.? 172.21.15.117:0/3409444435' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi117-63915-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:08.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:08 smithi151 bash[23756]: audit 2024-03-04T15:56:07.808198+0000 mon.a (mon.0) 1578 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi117-63915-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:08.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:08 smithi151 bash[23756]: audit 2024-03-04T15:56:08.663810+0000 mon.a (mon.0) 1579 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:09.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:08 smithi117 bash[21480]: audit 2024-03-04T15:56:07.662185+0000 mon.a (mon.0) 1573 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:09.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:08 smithi117 bash[21480]: cluster 2024-03-04T15:56:07.696173+0000 mon.a (mon.0) 1574 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:09.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:08 smithi117 bash[21480]: audit 2024-03-04T15:56:07.797934+0000 mon.a (mon.0) 1575 : audit 1 from='client.? 172.21.15.117:0/4020656977' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi117-63923-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:09.071 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:08 smithi117 bash[21480]: audit 2024-03-04T15:56:07.798114+0000 mon.a (mon.0) 1576 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi117-63915-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:09.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:08 smithi117 bash[21480]: cluster 2024-03-04T15:56:07.804670+0000 mon.a (mon.0) 1577 : cluster 0 osdmap e179: 8 total, 8 up, 8 in 2024-03-04T15:56:09.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:08 smithi117 bash[21480]: audit 2024-03-04T15:56:07.806789+0000 mon.c (mon.2) 269 : audit 1 from='client.? 172.21.15.117:0/3409444435' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi117-63915-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:09.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:08 smithi117 bash[21480]: audit 2024-03-04T15:56:07.808198+0000 mon.a (mon.0) 1578 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi117-63915-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:09.072 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:08 smithi117 bash[21480]: audit 2024-03-04T15:56:08.663810+0000 mon.a (mon.0) 1579 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:09.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:08 smithi117 bash[28843]: audit 2024-03-04T15:56:07.662185+0000 mon.a (mon.0) 1573 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:09.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:08 smithi117 bash[28843]: cluster 2024-03-04T15:56:07.696173+0000 mon.a (mon.0) 1574 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:09.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:08 smithi117 bash[28843]: audit 2024-03-04T15:56:07.797934+0000 mon.a (mon.0) 1575 : audit 1 from='client.? 172.21.15.117:0/4020656977' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi117-63923-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:09.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:08 smithi117 bash[28843]: audit 2024-03-04T15:56:07.798114+0000 mon.a (mon.0) 1576 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi117-63915-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:09.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:08 smithi117 bash[28843]: cluster 2024-03-04T15:56:07.804670+0000 mon.a (mon.0) 1577 : cluster 0 osdmap e179: 8 total, 8 up, 8 in 2024-03-04T15:56:09.072 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:08 smithi117 bash[28843]: audit 2024-03-04T15:56:07.806789+0000 mon.c (mon.2) 269 : audit 1 from='client.? 172.21.15.117:0/3409444435' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi117-63915-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:09.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:08 smithi117 bash[28843]: audit 2024-03-04T15:56:07.808198+0000 mon.a (mon.0) 1578 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi117-63915-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:09.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:08 smithi117 bash[28843]: audit 2024-03-04T15:56:08.663810+0000 mon.a (mon.0) 1579 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:10.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:09 smithi117 bash[21480]: cluster 2024-03-04T15:56:08.845939+0000 mgr.y (mgr.24428) 270 : cluster 0 pgmap v276: 268 pgs: 268 active+clean; 599 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-03-04T15:56:10.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:09 smithi117 bash[21480]: cluster 2024-03-04T15:56:09.263030+0000 mon.a (mon.0) 1580 : cluster 0 osdmap e180: 8 total, 8 up, 8 in 2024-03-04T15:56:10.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:09 smithi117 bash[21480]: audit 2024-03-04T15:56:09.526131+0000 mon.c (mon.2) 270 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:10.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:09 smithi117 bash[21480]: audit 2024-03-04T15:56:09.724739+0000 mon.a (mon.0) 1581 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:10.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:09 smithi117 bash[21480]: audit 2024-03-04T15:56:09.725396+0000 mon.a (mon.0) 1582 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:10.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:09 smithi117 bash[28843]: cluster 2024-03-04T15:56:08.845939+0000 mgr.y (mgr.24428) 270 : cluster 0 pgmap v276: 268 pgs: 268 active+clean; 599 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-03-04T15:56:10.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:09 smithi117 bash[28843]: cluster 2024-03-04T15:56:09.263030+0000 mon.a (mon.0) 1580 : cluster 0 osdmap e180: 8 total, 8 up, 8 in 2024-03-04T15:56:10.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:09 smithi117 bash[28843]: audit 2024-03-04T15:56:09.526131+0000 mon.c (mon.2) 270 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:10.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:09 smithi117 bash[28843]: audit 2024-03-04T15:56:09.724739+0000 mon.a (mon.0) 1581 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:10.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:09 smithi117 bash[28843]: audit 2024-03-04T15:56:09.725396+0000 mon.a (mon.0) 1582 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:09 smithi151 bash[23756]: cluster 2024-03-04T15:56:08.845939+0000 mgr.y (mgr.24428) 270 : cluster 0 pgmap v276: 268 pgs: 268 active+clean; 599 KiB data, 548 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-03-04T15:56:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:09 smithi151 bash[23756]: cluster 2024-03-04T15:56:09.263030+0000 mon.a (mon.0) 1580 : cluster 0 osdmap e180: 8 total, 8 up, 8 in 2024-03-04T15:56:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:09 smithi151 bash[23756]: audit 2024-03-04T15:56:09.526131+0000 mon.c (mon.2) 270 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:09 smithi151 bash[23756]: audit 2024-03-04T15:56:09.724739+0000 mon.a (mon.0) 1581 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:09 smithi151 bash[23756]: audit 2024-03-04T15:56:09.725396+0000 mon.a (mon.0) 1582 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:11.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:10 smithi151 bash[23756]: audit 2024-03-04T15:56:09.816484+0000 mon.a (mon.0) 1583 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi117-63915-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi117-63915-27"}]': finished 2024-03-04T15:56:11.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:10 smithi151 bash[23756]: audit 2024-03-04T15:56:09.816710+0000 mon.a (mon.0) 1584 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:56:11.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:10 smithi151 bash[23756]: cluster 2024-03-04T15:56:09.825810+0000 mon.a (mon.0) 1585 : cluster 0 osdmap e181: 8 total, 8 up, 8 in 2024-03-04T15:56:11.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:10 smithi151 bash[23756]: audit 2024-03-04T15:56:09.827560+0000 mon.c (mon.2) 271 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-13"}]: dispatch 2024-03-04T15:56:11.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:10 smithi151 bash[23756]: audit 2024-03-04T15:56:09.828082+0000 mon.c (mon.2) 272 : audit 1 from='client.? 172.21.15.117:0/1821547806' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi117-63923-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:11.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:10 smithi151 bash[23756]: audit 2024-03-04T15:56:09.829748+0000 mon.a (mon.0) 1586 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-13"}]: dispatch 2024-03-04T15:56:11.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:10 smithi151 bash[23756]: audit 2024-03-04T15:56:09.831044+0000 mon.a (mon.0) 1587 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi117-63923-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:11.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:10 smithi151 bash[23756]: audit 2024-03-04T15:56:10.727076+0000 mon.a (mon.0) 1588 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:11.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:10 smithi151 bash[23756]: audit 2024-03-04T15:56:10.728448+0000 mon.a (mon.0) 1589 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pg_num","val":"11"}]: dispatch 2024-03-04T15:56:11.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:10 smithi117 bash[28843]: audit 2024-03-04T15:56:09.816484+0000 mon.a (mon.0) 1583 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi117-63915-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi117-63915-27"}]': finished 2024-03-04T15:56:11.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:10 smithi117 bash[28843]: audit 2024-03-04T15:56:09.816710+0000 mon.a (mon.0) 1584 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:56:11.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:10 smithi117 bash[28843]: cluster 2024-03-04T15:56:09.825810+0000 mon.a (mon.0) 1585 : cluster 0 osdmap e181: 8 total, 8 up, 8 in 2024-03-04T15:56:11.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:10 smithi117 bash[28843]: audit 2024-03-04T15:56:09.827560+0000 mon.c (mon.2) 271 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-13"}]: dispatch 2024-03-04T15:56:11.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:10 smithi117 bash[28843]: audit 2024-03-04T15:56:09.828082+0000 mon.c (mon.2) 272 : audit 1 from='client.? 172.21.15.117:0/1821547806' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi117-63923-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:11.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:10 smithi117 bash[28843]: audit 2024-03-04T15:56:09.829748+0000 mon.a (mon.0) 1586 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-13"}]: dispatch 2024-03-04T15:56:11.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:10 smithi117 bash[28843]: audit 2024-03-04T15:56:09.831044+0000 mon.a (mon.0) 1587 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi117-63923-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:11.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:10 smithi117 bash[28843]: audit 2024-03-04T15:56:10.727076+0000 mon.a (mon.0) 1588 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:11.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:10 smithi117 bash[28843]: audit 2024-03-04T15:56:10.728448+0000 mon.a (mon.0) 1589 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pg_num","val":"11"}]: dispatch 2024-03-04T15:56:11.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:10 smithi117 bash[21480]: audit 2024-03-04T15:56:09.816484+0000 mon.a (mon.0) 1583 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi117-63915-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi117-63915-27"}]': finished 2024-03-04T15:56:11.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:10 smithi117 bash[21480]: audit 2024-03-04T15:56:09.816710+0000 mon.a (mon.0) 1584 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:56:11.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:10 smithi117 bash[21480]: cluster 2024-03-04T15:56:09.825810+0000 mon.a (mon.0) 1585 : cluster 0 osdmap e181: 8 total, 8 up, 8 in 2024-03-04T15:56:11.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:10 smithi117 bash[21480]: audit 2024-03-04T15:56:09.827560+0000 mon.c (mon.2) 271 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-13"}]: dispatch 2024-03-04T15:56:11.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:10 smithi117 bash[21480]: audit 2024-03-04T15:56:09.828082+0000 mon.c (mon.2) 272 : audit 1 from='client.? 172.21.15.117:0/1821547806' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi117-63923-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:11.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:10 smithi117 bash[21480]: audit 2024-03-04T15:56:09.829748+0000 mon.a (mon.0) 1586 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-13"}]: dispatch 2024-03-04T15:56:11.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:10 smithi117 bash[21480]: audit 2024-03-04T15:56:09.831044+0000 mon.a (mon.0) 1587 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi117-63923-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:11.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:10 smithi117 bash[21480]: audit 2024-03-04T15:56:10.727076+0000 mon.a (mon.0) 1588 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:11.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:10 smithi117 bash[21480]: audit 2024-03-04T15:56:10.728448+0000 mon.a (mon.0) 1589 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pg_num","val":"11"}]: dispatch 2024-03-04T15:56:12.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[21480]: cluster 2024-03-04T15:56:10.817268+0000 mon.a (mon.0) 1590 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:56:12.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[21480]: audit 2024-03-04T15:56:10.819816+0000 mon.a (mon.0) 1591 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-13"}]': finished 2024-03-04T15:56:12.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[21480]: audit 2024-03-04T15:56:10.819974+0000 mon.a (mon.0) 1592 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi117-63923-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:12.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[21480]: audit 2024-03-04T15:56:10.820200+0000 mon.a (mon.0) 1593 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pg_num","val":"11"}]': finished 2024-03-04T15:56:12.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[21480]: cluster 2024-03-04T15:56:10.828998+0000 mon.a (mon.0) 1594 : cluster 0 osdmap e182: 8 total, 8 up, 8 in 2024-03-04T15:56:12.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[21480]: audit 2024-03-04T15:56:10.835464+0000 mon.a (mon.0) 1595 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:12.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[21480]: audit 2024-03-04T15:56:10.836953+0000 mon.a (mon.0) 1596 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:12.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[21480]: audit 2024-03-04T15:56:10.838173+0000 mon.a (mon.0) 1597 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-04T15:56:12.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[21480]: cluster 2024-03-04T15:56:10.847161+0000 mgr.y (mgr.24428) 271 : cluster 0 pgmap v279: 308 pgs: 40 unknown, 268 active+clean; 599 KiB data, 548 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:56:12.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[21480]: audit 2024-03-04T15:56:11.822875+0000 mon.a (mon.0) 1598 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pgp_num","val":"11"}]': finished 2024-03-04T15:56:12.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[21480]: cluster 2024-03-04T15:56:11.831005+0000 mon.a (mon.0) 1599 : cluster 0 osdmap e183: 8 total, 8 up, 8 in 2024-03-04T15:56:12.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[21480]: audit 2024-03-04T15:56:11.834609+0000 mon.a (mon.0) 1600 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:12.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[21480]: audit 2024-03-04T15:56:11.837770+0000 mon.c (mon.2) 273 : audit 1 from='client.? 172.21.15.117:0/3409444435' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:12.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[28843]: cluster 2024-03-04T15:56:10.817268+0000 mon.a (mon.0) 1590 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:56:12.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[28843]: audit 2024-03-04T15:56:10.819816+0000 mon.a (mon.0) 1591 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-13"}]': finished 2024-03-04T15:56:12.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[28843]: audit 2024-03-04T15:56:10.819974+0000 mon.a (mon.0) 1592 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi117-63923-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:12.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[28843]: audit 2024-03-04T15:56:10.820200+0000 mon.a (mon.0) 1593 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pg_num","val":"11"}]': finished 2024-03-04T15:56:12.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[28843]: cluster 2024-03-04T15:56:10.828998+0000 mon.a (mon.0) 1594 : cluster 0 osdmap e182: 8 total, 8 up, 8 in 2024-03-04T15:56:12.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[28843]: audit 2024-03-04T15:56:10.835464+0000 mon.a (mon.0) 1595 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:12.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[28843]: audit 2024-03-04T15:56:10.836953+0000 mon.a (mon.0) 1596 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:12.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[28843]: audit 2024-03-04T15:56:10.838173+0000 mon.a (mon.0) 1597 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-04T15:56:12.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[28843]: cluster 2024-03-04T15:56:10.847161+0000 mgr.y (mgr.24428) 271 : cluster 0 pgmap v279: 308 pgs: 40 unknown, 268 active+clean; 599 KiB data, 548 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:56:12.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[28843]: audit 2024-03-04T15:56:11.822875+0000 mon.a (mon.0) 1598 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pgp_num","val":"11"}]': finished 2024-03-04T15:56:12.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[28843]: cluster 2024-03-04T15:56:11.831005+0000 mon.a (mon.0) 1599 : cluster 0 osdmap e183: 8 total, 8 up, 8 in 2024-03-04T15:56:12.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[28843]: audit 2024-03-04T15:56:11.834609+0000 mon.a (mon.0) 1600 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:12.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:12 smithi117 bash[28843]: audit 2024-03-04T15:56:11.837770+0000 mon.c (mon.2) 273 : audit 1 from='client.? 172.21.15.117:0/3409444435' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:12.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:12 smithi151 bash[23756]: cluster 2024-03-04T15:56:10.817268+0000 mon.a (mon.0) 1590 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:56:12.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:12 smithi151 bash[23756]: audit 2024-03-04T15:56:10.819816+0000 mon.a (mon.0) 1591 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-13"}]': finished 2024-03-04T15:56:12.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:12 smithi151 bash[23756]: audit 2024-03-04T15:56:10.819974+0000 mon.a (mon.0) 1592 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi117-63923-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:12.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:12 smithi151 bash[23756]: audit 2024-03-04T15:56:10.820200+0000 mon.a (mon.0) 1593 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pg_num","val":"11"}]': finished 2024-03-04T15:56:12.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:12 smithi151 bash[23756]: cluster 2024-03-04T15:56:10.828998+0000 mon.a (mon.0) 1594 : cluster 0 osdmap e182: 8 total, 8 up, 8 in 2024-03-04T15:56:12.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:12 smithi151 bash[23756]: audit 2024-03-04T15:56:10.835464+0000 mon.a (mon.0) 1595 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:12.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:12 smithi151 bash[23756]: audit 2024-03-04T15:56:10.836953+0000 mon.a (mon.0) 1596 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:12.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:12 smithi151 bash[23756]: audit 2024-03-04T15:56:10.838173+0000 mon.a (mon.0) 1597 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-04T15:56:12.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:12 smithi151 bash[23756]: cluster 2024-03-04T15:56:10.847161+0000 mgr.y (mgr.24428) 271 : cluster 0 pgmap v279: 308 pgs: 40 unknown, 268 active+clean; 599 KiB data, 548 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:56:12.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:12 smithi151 bash[23756]: audit 2024-03-04T15:56:11.822875+0000 mon.a (mon.0) 1598 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pgp_num","val":"11"}]': finished 2024-03-04T15:56:12.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:12 smithi151 bash[23756]: cluster 2024-03-04T15:56:11.831005+0000 mon.a (mon.0) 1599 : cluster 0 osdmap e183: 8 total, 8 up, 8 in 2024-03-04T15:56:12.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:12 smithi151 bash[23756]: audit 2024-03-04T15:56:11.834609+0000 mon.a (mon.0) 1600 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:12.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:12 smithi151 bash[23756]: audit 2024-03-04T15:56:11.837770+0000 mon.c (mon.2) 273 : audit 1 from='client.? 172.21.15.117:0/3409444435' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:13.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[21480]: audit 2024-03-04T15:56:12.300434+0000 mon.a (mon.0) 1601 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:13.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[21480]: audit 2024-03-04T15:56:12.301890+0000 mon.a (mon.0) 1602 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:13.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[21480]: audit 2024-03-04T15:56:12.310335+0000 mon.a (mon.0) 1603 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pg_num","val":"11"}]: dispatch 2024-03-04T15:56:13.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[21480]: audit 2024-03-04T15:56:12.734520+0000 mon.a (mon.0) 1604 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi117-63915-27"}]': finished 2024-03-04T15:56:13.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[21480]: audit 2024-03-04T15:56:12.734715+0000 mon.a (mon.0) 1605 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pg_num","val":"11"}]': finished 2024-03-04T15:56:13.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[21480]: audit 2024-03-04T15:56:12.739988+0000 mon.c (mon.2) 274 : audit 1 from='client.? 172.21.15.117:0/3409444435' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:13.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[21480]: cluster 2024-03-04T15:56:12.742918+0000 mon.a (mon.0) 1606 : cluster 0 osdmap e184: 8 total, 8 up, 8 in 2024-03-04T15:56:13.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[21480]: audit 2024-03-04T15:56:12.745388+0000 mon.a (mon.0) 1607 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:13.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[21480]: audit 2024-03-04T15:56:12.745552+0000 mon.c (mon.2) 275 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:13.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[21480]: audit 2024-03-04T15:56:12.745909+0000 mon.a (mon.0) 1608 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:13.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[21480]: audit 2024-03-04T15:56:12.746263+0000 mon.a (mon.0) 1609 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:13.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[21480]: audit 2024-03-04T15:56:12.748653+0000 mon.c (mon.2) 276 : audit 1 from='client.? 172.21.15.117:0/2193782967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi117-63923-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:13.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[21480]: audit 2024-03-04T15:56:12.749246+0000 mon.a (mon.0) 1610 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi117-63923-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:13.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[21480]: cluster 2024-03-04T15:56:12.848307+0000 mgr.y (mgr.24428) 272 : cluster 0 pgmap v282: 300 pgs: 64 unknown, 236 active+clean; 609 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-03-04T15:56:13.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[21480]: audit 2024-03-04T15:56:13.271656+0000 mon.a (mon.0) 1611 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:56:13.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[21480]: audit 2024-03-04T15:56:13.274879+0000 mon.c (mon.2) 277 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:56:13.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[28843]: audit 2024-03-04T15:56:12.300434+0000 mon.a (mon.0) 1601 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:13.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[28843]: audit 2024-03-04T15:56:12.301890+0000 mon.a (mon.0) 1602 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:13.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[28843]: audit 2024-03-04T15:56:12.310335+0000 mon.a (mon.0) 1603 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pg_num","val":"11"}]: dispatch 2024-03-04T15:56:13.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[28843]: audit 2024-03-04T15:56:12.734520+0000 mon.a (mon.0) 1604 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi117-63915-27"}]': finished 2024-03-04T15:56:13.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[28843]: audit 2024-03-04T15:56:12.734715+0000 mon.a (mon.0) 1605 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pg_num","val":"11"}]': finished 2024-03-04T15:56:13.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[28843]: audit 2024-03-04T15:56:12.739988+0000 mon.c (mon.2) 274 : audit 1 from='client.? 172.21.15.117:0/3409444435' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:13.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[28843]: cluster 2024-03-04T15:56:12.742918+0000 mon.a (mon.0) 1606 : cluster 0 osdmap e184: 8 total, 8 up, 8 in 2024-03-04T15:56:13.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[28843]: audit 2024-03-04T15:56:12.745388+0000 mon.a (mon.0) 1607 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:13.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[28843]: audit 2024-03-04T15:56:12.745552+0000 mon.c (mon.2) 275 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:13.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[28843]: audit 2024-03-04T15:56:12.745909+0000 mon.a (mon.0) 1608 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:13.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[28843]: audit 2024-03-04T15:56:12.746263+0000 mon.a (mon.0) 1609 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:13.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[28843]: audit 2024-03-04T15:56:12.748653+0000 mon.c (mon.2) 276 : audit 1 from='client.? 172.21.15.117:0/2193782967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi117-63923-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:13.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[28843]: audit 2024-03-04T15:56:12.749246+0000 mon.a (mon.0) 1610 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi117-63923-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:13.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[28843]: cluster 2024-03-04T15:56:12.848307+0000 mgr.y (mgr.24428) 272 : cluster 0 pgmap v282: 300 pgs: 64 unknown, 236 active+clean; 609 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-03-04T15:56:13.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[28843]: audit 2024-03-04T15:56:13.271656+0000 mon.a (mon.0) 1611 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:56:13.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:13 smithi117 bash[28843]: audit 2024-03-04T15:56:13.274879+0000 mon.c (mon.2) 277 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:56:13.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:13 smithi151 bash[23756]: audit 2024-03-04T15:56:12.300434+0000 mon.a (mon.0) 1601 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:13.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:13 smithi151 bash[23756]: audit 2024-03-04T15:56:12.301890+0000 mon.a (mon.0) 1602 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:13.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:13 smithi151 bash[23756]: audit 2024-03-04T15:56:12.310335+0000 mon.a (mon.0) 1603 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pg_num","val":"11"}]: dispatch 2024-03-04T15:56:13.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:13 smithi151 bash[23756]: audit 2024-03-04T15:56:12.734520+0000 mon.a (mon.0) 1604 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi117-63915-27"}]': finished 2024-03-04T15:56:13.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:13 smithi151 bash[23756]: audit 2024-03-04T15:56:12.734715+0000 mon.a (mon.0) 1605 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pg_num","val":"11"}]': finished 2024-03-04T15:56:13.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:13 smithi151 bash[23756]: audit 2024-03-04T15:56:12.739988+0000 mon.c (mon.2) 274 : audit 1 from='client.? 172.21.15.117:0/3409444435' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:13.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:13 smithi151 bash[23756]: cluster 2024-03-04T15:56:12.742918+0000 mon.a (mon.0) 1606 : cluster 0 osdmap e184: 8 total, 8 up, 8 in 2024-03-04T15:56:13.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:13 smithi151 bash[23756]: audit 2024-03-04T15:56:12.745388+0000 mon.a (mon.0) 1607 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:13.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:13 smithi151 bash[23756]: audit 2024-03-04T15:56:12.745552+0000 mon.c (mon.2) 275 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:13.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:13 smithi151 bash[23756]: audit 2024-03-04T15:56:12.745909+0000 mon.a (mon.0) 1608 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi117-63915-27"}]: dispatch 2024-03-04T15:56:13.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:13 smithi151 bash[23756]: audit 2024-03-04T15:56:12.746263+0000 mon.a (mon.0) 1609 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:13.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:13 smithi151 bash[23756]: audit 2024-03-04T15:56:12.748653+0000 mon.c (mon.2) 276 : audit 1 from='client.? 172.21.15.117:0/2193782967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi117-63923-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:13.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:13 smithi151 bash[23756]: audit 2024-03-04T15:56:12.749246+0000 mon.a (mon.0) 1610 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi117-63923-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:13.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:13 smithi151 bash[23756]: cluster 2024-03-04T15:56:12.848307+0000 mgr.y (mgr.24428) 272 : cluster 0 pgmap v282: 300 pgs: 64 unknown, 236 active+clean; 609 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-03-04T15:56:13.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:13 smithi151 bash[23756]: audit 2024-03-04T15:56:13.271656+0000 mon.a (mon.0) 1611 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:56:13.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:13 smithi151 bash[23756]: audit 2024-03-04T15:56:13.274879+0000 mon.c (mon.2) 277 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:56:14.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[21480]: cluster 2024-03-04T15:56:13.736899+0000 mon.a (mon.0) 1612 : cluster 3 Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:14.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[21480]: audit 2024-03-04T15:56:13.738790+0000 mon.a (mon.0) 1613 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi117-63915-27"}]': finished 2024-03-04T15:56:14.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[21480]: audit 2024-03-04T15:56:13.739015+0000 mon.a (mon.0) 1614 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:14.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[21480]: audit 2024-03-04T15:56:13.739203+0000 mon.a (mon.0) 1615 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi117-63923-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:14.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[21480]: cluster 2024-03-04T15:56:13.747411+0000 mon.a (mon.0) 1616 : cluster 0 osdmap e185: 8 total, 8 up, 8 in 2024-03-04T15:56:14.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[21480]: audit 2024-03-04T15:56:13.751596+0000 mon.a (mon.0) 1617 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:14.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[21480]: audit 2024-03-04T15:56:13.814335+0000 mon.c (mon.2) 278 : audit 1 from='client.? 172.21.15.117:0/543328654' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:14.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[21480]: audit 2024-03-04T15:56:13.815635+0000 mon.a (mon.0) 1618 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:14.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[21480]: audit 2024-03-04T15:56:13.821434+0000 mon.c (mon.2) 279 : audit 1 from='client.? 172.21.15.117:0/543328654' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:14.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[21480]: audit 2024-03-04T15:56:13.821869+0000 mon.a (mon.0) 1619 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:14.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[21480]: audit 2024-03-04T15:56:13.822391+0000 mon.c (mon.2) 280 : audit 1 from='client.? 172.21.15.117:0/543328654' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi117-63915-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:14.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[21480]: audit 2024-03-04T15:56:13.822794+0000 mon.a (mon.0) 1620 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi117-63915-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:14.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[28843]: cluster 2024-03-04T15:56:13.736899+0000 mon.a (mon.0) 1612 : cluster 3 Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:14.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[28843]: audit 2024-03-04T15:56:13.738790+0000 mon.a (mon.0) 1613 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi117-63915-27"}]': finished 2024-03-04T15:56:14.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[28843]: audit 2024-03-04T15:56:13.739015+0000 mon.a (mon.0) 1614 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:14.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[28843]: audit 2024-03-04T15:56:13.739203+0000 mon.a (mon.0) 1615 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi117-63923-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:14.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[28843]: cluster 2024-03-04T15:56:13.747411+0000 mon.a (mon.0) 1616 : cluster 0 osdmap e185: 8 total, 8 up, 8 in 2024-03-04T15:56:14.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[28843]: audit 2024-03-04T15:56:13.751596+0000 mon.a (mon.0) 1617 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:14.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[28843]: audit 2024-03-04T15:56:13.814335+0000 mon.c (mon.2) 278 : audit 1 from='client.? 172.21.15.117:0/543328654' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:14.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[28843]: audit 2024-03-04T15:56:13.815635+0000 mon.a (mon.0) 1618 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:14.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[28843]: audit 2024-03-04T15:56:13.821434+0000 mon.c (mon.2) 279 : audit 1 from='client.? 172.21.15.117:0/543328654' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:14.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[28843]: audit 2024-03-04T15:56:13.821869+0000 mon.a (mon.0) 1619 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:14.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[28843]: audit 2024-03-04T15:56:13.822391+0000 mon.c (mon.2) 280 : audit 1 from='client.? 172.21.15.117:0/543328654' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi117-63915-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:14.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:14 smithi117 bash[28843]: audit 2024-03-04T15:56:13.822794+0000 mon.a (mon.0) 1620 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi117-63915-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:14.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:14 smithi151 bash[23756]: cluster 2024-03-04T15:56:13.736899+0000 mon.a (mon.0) 1612 : cluster 3 Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:14.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:14 smithi151 bash[23756]: audit 2024-03-04T15:56:13.738790+0000 mon.a (mon.0) 1613 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi117-63915-27"}]': finished 2024-03-04T15:56:14.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:14 smithi151 bash[23756]: audit 2024-03-04T15:56:13.739015+0000 mon.a (mon.0) 1614 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:14.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:14 smithi151 bash[23756]: audit 2024-03-04T15:56:13.739203+0000 mon.a (mon.0) 1615 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi117-63923-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:14.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:14 smithi151 bash[23756]: cluster 2024-03-04T15:56:13.747411+0000 mon.a (mon.0) 1616 : cluster 0 osdmap e185: 8 total, 8 up, 8 in 2024-03-04T15:56:14.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:14 smithi151 bash[23756]: audit 2024-03-04T15:56:13.751596+0000 mon.a (mon.0) 1617 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:14.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:14 smithi151 bash[23756]: audit 2024-03-04T15:56:13.814335+0000 mon.c (mon.2) 278 : audit 1 from='client.? 172.21.15.117:0/543328654' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:14.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:14 smithi151 bash[23756]: audit 2024-03-04T15:56:13.815635+0000 mon.a (mon.0) 1618 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:14.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:14 smithi151 bash[23756]: audit 2024-03-04T15:56:13.821434+0000 mon.c (mon.2) 279 : audit 1 from='client.? 172.21.15.117:0/543328654' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:14.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:14 smithi151 bash[23756]: audit 2024-03-04T15:56:13.821869+0000 mon.a (mon.0) 1619 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:14.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:14 smithi151 bash[23756]: audit 2024-03-04T15:56:13.822391+0000 mon.c (mon.2) 280 : audit 1 from='client.? 172.21.15.117:0/543328654' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi117-63915-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:14.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:14 smithi151 bash[23756]: audit 2024-03-04T15:56:13.822794+0000 mon.a (mon.0) 1620 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi117-63915-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:15.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:15 smithi117 bash[21480]: audit 2024-03-04T15:56:14.753792+0000 mon.a (mon.0) 1621 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:15.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:15 smithi117 bash[21480]: audit 2024-03-04T15:56:14.764582+0000 mon.a (mon.0) 1622 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi117-63915-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:15.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:15 smithi117 bash[21480]: cluster 2024-03-04T15:56:14.772339+0000 mon.a (mon.0) 1623 : cluster 0 osdmap e186: 8 total, 8 up, 8 in 2024-03-04T15:56:15.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:15 smithi117 bash[21480]: audit 2024-03-04T15:56:14.774562+0000 mon.c (mon.2) 281 : audit 1 from='client.? 172.21.15.117:0/543328654' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi117-63915-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:15.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:15 smithi117 bash[21480]: audit 2024-03-04T15:56:14.776914+0000 mon.a (mon.0) 1624 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi117-63915-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:15.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:15 smithi117 bash[21480]: audit 2024-03-04T15:56:14.807015+0000 mon.c (mon.2) 282 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:15.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:15 smithi117 bash[21480]: audit 2024-03-04T15:56:14.808546+0000 mon.a (mon.0) 1625 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:15.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:15 smithi117 bash[21480]: cluster 2024-03-04T15:56:14.848804+0000 mgr.y (mgr.24428) 273 : cluster 0 pgmap v285: 268 pgs: 32 unknown, 236 active+clean; 609 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-03-04T15:56:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:15 smithi117 bash[28843]: audit 2024-03-04T15:56:14.753792+0000 mon.a (mon.0) 1621 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:15 smithi117 bash[28843]: audit 2024-03-04T15:56:14.764582+0000 mon.a (mon.0) 1622 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi117-63915-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:15 smithi117 bash[28843]: cluster 2024-03-04T15:56:14.772339+0000 mon.a (mon.0) 1623 : cluster 0 osdmap e186: 8 total, 8 up, 8 in 2024-03-04T15:56:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:15 smithi117 bash[28843]: audit 2024-03-04T15:56:14.774562+0000 mon.c (mon.2) 281 : audit 1 from='client.? 172.21.15.117:0/543328654' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi117-63915-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:15 smithi117 bash[28843]: audit 2024-03-04T15:56:14.776914+0000 mon.a (mon.0) 1624 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi117-63915-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:15 smithi117 bash[28843]: audit 2024-03-04T15:56:14.807015+0000 mon.c (mon.2) 282 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:15.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:15 smithi117 bash[28843]: audit 2024-03-04T15:56:14.808546+0000 mon.a (mon.0) 1625 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:15.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:15 smithi117 bash[28843]: cluster 2024-03-04T15:56:14.848804+0000 mgr.y (mgr.24428) 273 : cluster 0 pgmap v285: 268 pgs: 32 unknown, 236 active+clean; 609 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-03-04T15:56:15.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:15 smithi151 bash[23756]: audit 2024-03-04T15:56:14.753792+0000 mon.a (mon.0) 1621 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:15.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:15 smithi151 bash[23756]: audit 2024-03-04T15:56:14.764582+0000 mon.a (mon.0) 1622 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi117-63915-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:15.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:15 smithi151 bash[23756]: cluster 2024-03-04T15:56:14.772339+0000 mon.a (mon.0) 1623 : cluster 0 osdmap e186: 8 total, 8 up, 8 in 2024-03-04T15:56:15.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:15 smithi151 bash[23756]: audit 2024-03-04T15:56:14.774562+0000 mon.c (mon.2) 281 : audit 1 from='client.? 172.21.15.117:0/543328654' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi117-63915-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:15.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:15 smithi151 bash[23756]: audit 2024-03-04T15:56:14.776914+0000 mon.a (mon.0) 1624 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi117-63915-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:15.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:15 smithi151 bash[23756]: audit 2024-03-04T15:56:14.807015+0000 mon.c (mon.2) 282 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:15.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:15 smithi151 bash[23756]: audit 2024-03-04T15:56:14.808546+0000 mon.a (mon.0) 1625 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:15.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:15 smithi151 bash[23756]: cluster 2024-03-04T15:56:14.848804+0000 mgr.y (mgr.24428) 273 : cluster 0 pgmap v285: 268 pgs: 32 unknown, 236 active+clean; 609 KiB data, 554 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-03-04T15:56:16.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:56:16 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:56:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:56:16.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:16 smithi117 bash[21480]: audit 2024-03-04T15:56:15.755448+0000 mon.a (mon.0) 1626 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:16.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:16 smithi117 bash[21480]: audit 2024-03-04T15:56:15.777636+0000 mon.a (mon.0) 1627 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-15", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:56:16.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:16 smithi117 bash[21480]: audit 2024-03-04T15:56:15.787915+0000 mon.c (mon.2) 283 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-15"}]: dispatch 2024-03-04T15:56:16.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:16 smithi117 bash[21480]: cluster 2024-03-04T15:56:16.236779+0000 mon.a (mon.0) 1628 : cluster 0 osdmap e187: 8 total, 8 up, 8 in 2024-03-04T15:56:16.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:16 smithi117 bash[21480]: audit 2024-03-04T15:56:16.238150+0000 mon.a (mon.0) 1629 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-15"}]: dispatch 2024-03-04T15:56:16.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:16 smithi117 bash[21480]: audit 2024-03-04T15:56:16.240046+0000 mon.a (mon.0) 1630 : audit 1 from='client.? 172.21.15.117:0/4084577413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi117-63923-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:16.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:16 smithi117 bash[28843]: audit 2024-03-04T15:56:15.755448+0000 mon.a (mon.0) 1626 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:16.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:16 smithi117 bash[28843]: audit 2024-03-04T15:56:15.777636+0000 mon.a (mon.0) 1627 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-15", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:56:16.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:16 smithi117 bash[28843]: audit 2024-03-04T15:56:15.787915+0000 mon.c (mon.2) 283 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-15"}]: dispatch 2024-03-04T15:56:16.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:16 smithi117 bash[28843]: cluster 2024-03-04T15:56:16.236779+0000 mon.a (mon.0) 1628 : cluster 0 osdmap e187: 8 total, 8 up, 8 in 2024-03-04T15:56:16.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:16 smithi117 bash[28843]: audit 2024-03-04T15:56:16.238150+0000 mon.a (mon.0) 1629 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-15"}]: dispatch 2024-03-04T15:56:16.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:16 smithi117 bash[28843]: audit 2024-03-04T15:56:16.240046+0000 mon.a (mon.0) 1630 : audit 1 from='client.? 172.21.15.117:0/4084577413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi117-63923-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:16.575 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:16 smithi151 bash[23756]: audit 2024-03-04T15:56:15.755448+0000 mon.a (mon.0) 1626 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:16.575 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:16 smithi151 bash[23756]: audit 2024-03-04T15:56:15.777636+0000 mon.a (mon.0) 1627 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-15", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:56:16.575 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:16 smithi151 bash[23756]: audit 2024-03-04T15:56:15.787915+0000 mon.c (mon.2) 283 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-15"}]: dispatch 2024-03-04T15:56:16.575 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:16 smithi151 bash[23756]: cluster 2024-03-04T15:56:16.236779+0000 mon.a (mon.0) 1628 : cluster 0 osdmap e187: 8 total, 8 up, 8 in 2024-03-04T15:56:16.575 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:16 smithi151 bash[23756]: audit 2024-03-04T15:56:16.238150+0000 mon.a (mon.0) 1629 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-15"}]: dispatch 2024-03-04T15:56:16.576 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:16 smithi151 bash[23756]: audit 2024-03-04T15:56:16.240046+0000 mon.a (mon.0) 1630 : audit 1 from='client.? 172.21.15.117:0/4084577413' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi117-63923-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:16.932 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:56:16 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:56:17.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:17 smithi117 bash[21480]: audit 2024-03-04T15:56:16.577264+0000 mgr.y (mgr.24428) 274 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:56:17.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:17 smithi117 bash[21480]: audit 2024-03-04T15:56:16.757188+0000 mon.a (mon.0) 1631 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:17.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:17 smithi117 bash[21480]: audit 2024-03-04T15:56:16.781295+0000 mon.a (mon.0) 1632 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi117-63915-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi117-63915-28"}]': finished 2024-03-04T15:56:17.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:17 smithi117 bash[21480]: audit 2024-03-04T15:56:16.781564+0000 mon.a (mon.0) 1633 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-15"}]': finished 2024-03-04T15:56:17.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:17 smithi117 bash[21480]: audit 2024-03-04T15:56:16.781797+0000 mon.a (mon.0) 1634 : audit 1 from='client.? 172.21.15.117:0/4084577413' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi117-63923-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:17.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:17 smithi117 bash[21480]: audit 2024-03-04T15:56:16.791257+0000 mon.c (mon.2) 284 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-15", "mode": "writeback"}]: dispatch 2024-03-04T15:56:17.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:17 smithi117 bash[21480]: cluster 2024-03-04T15:56:16.792178+0000 mon.a (mon.0) 1635 : cluster 0 osdmap e188: 8 total, 8 up, 8 in 2024-03-04T15:56:17.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:17 smithi117 bash[21480]: audit 2024-03-04T15:56:16.811150+0000 mon.a (mon.0) 1636 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-15", "mode": "writeback"}]: dispatch 2024-03-04T15:56:17.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:17 smithi117 bash[21480]: cluster 2024-03-04T15:56:16.849386+0000 mgr.y (mgr.24428) 275 : cluster 0 pgmap v288: 308 pgs: 8 unknown, 32 creating+peering, 268 active+clean; 609 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-04T15:56:17.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:17 smithi117 bash[28843]: audit 2024-03-04T15:56:16.577264+0000 mgr.y (mgr.24428) 274 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:56:17.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:17 smithi117 bash[28843]: audit 2024-03-04T15:56:16.757188+0000 mon.a (mon.0) 1631 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:17.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:17 smithi117 bash[28843]: audit 2024-03-04T15:56:16.781295+0000 mon.a (mon.0) 1632 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi117-63915-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi117-63915-28"}]': finished 2024-03-04T15:56:17.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:17 smithi117 bash[28843]: audit 2024-03-04T15:56:16.781564+0000 mon.a (mon.0) 1633 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-15"}]': finished 2024-03-04T15:56:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:17 smithi117 bash[28843]: audit 2024-03-04T15:56:16.781797+0000 mon.a (mon.0) 1634 : audit 1 from='client.? 172.21.15.117:0/4084577413' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi117-63923-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:17 smithi117 bash[28843]: audit 2024-03-04T15:56:16.791257+0000 mon.c (mon.2) 284 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-15", "mode": "writeback"}]: dispatch 2024-03-04T15:56:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:17 smithi117 bash[28843]: cluster 2024-03-04T15:56:16.792178+0000 mon.a (mon.0) 1635 : cluster 0 osdmap e188: 8 total, 8 up, 8 in 2024-03-04T15:56:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:17 smithi117 bash[28843]: audit 2024-03-04T15:56:16.811150+0000 mon.a (mon.0) 1636 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-15", "mode": "writeback"}]: dispatch 2024-03-04T15:56:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:17 smithi117 bash[28843]: cluster 2024-03-04T15:56:16.849386+0000 mgr.y (mgr.24428) 275 : cluster 0 pgmap v288: 308 pgs: 8 unknown, 32 creating+peering, 268 active+clean; 609 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-04T15:56:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:17 smithi151 bash[23756]: audit 2024-03-04T15:56:16.577264+0000 mgr.y (mgr.24428) 274 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:56:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:17 smithi151 bash[23756]: audit 2024-03-04T15:56:16.757188+0000 mon.a (mon.0) 1631 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:17 smithi151 bash[23756]: audit 2024-03-04T15:56:16.781295+0000 mon.a (mon.0) 1632 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi117-63915-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi117-63915-28"}]': finished 2024-03-04T15:56:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:17 smithi151 bash[23756]: audit 2024-03-04T15:56:16.781564+0000 mon.a (mon.0) 1633 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-15"}]': finished 2024-03-04T15:56:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:17 smithi151 bash[23756]: audit 2024-03-04T15:56:16.781797+0000 mon.a (mon.0) 1634 : audit 1 from='client.? 172.21.15.117:0/4084577413' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi117-63923-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:17 smithi151 bash[23756]: audit 2024-03-04T15:56:16.791257+0000 mon.c (mon.2) 284 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-15", "mode": "writeback"}]: dispatch 2024-03-04T15:56:17.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:17 smithi151 bash[23756]: cluster 2024-03-04T15:56:16.792178+0000 mon.a (mon.0) 1635 : cluster 0 osdmap e188: 8 total, 8 up, 8 in 2024-03-04T15:56:17.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:17 smithi151 bash[23756]: audit 2024-03-04T15:56:16.811150+0000 mon.a (mon.0) 1636 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-15", "mode": "writeback"}]: dispatch 2024-03-04T15:56:17.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:17 smithi151 bash[23756]: cluster 2024-03-04T15:56:16.849386+0000 mgr.y (mgr.24428) 275 : cluster 0 pgmap v288: 308 pgs: 8 unknown, 32 creating+peering, 268 active+clean; 609 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 10 op/s 2024-03-04T15:56:18.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:18 smithi117 bash[28843]: cluster 2024-03-04T15:56:17.732134+0000 mon.a (mon.0) 1637 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:56:18.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:18 smithi117 bash[28843]: audit 2024-03-04T15:56:17.738325+0000 mon.a (mon.0) 1638 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-15", "mode": "writeback"}]': finished 2024-03-04T15:56:18.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:18 smithi117 bash[28843]: cluster 2024-03-04T15:56:17.746055+0000 mon.a (mon.0) 1639 : cluster 0 osdmap e189: 8 total, 8 up, 8 in 2024-03-04T15:56:18.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:18 smithi117 bash[28843]: audit 2024-03-04T15:56:17.760946+0000 mon.a (mon.0) 1640 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:18.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:18 smithi117 bash[28843]: audit 2024-03-04T15:56:17.767805+0000 mon.a (mon.0) 1641 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]: dispatch 2024-03-04T15:56:18.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:18 smithi117 bash[21480]: cluster 2024-03-04T15:56:17.732134+0000 mon.a (mon.0) 1637 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:56:18.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:18 smithi117 bash[21480]: audit 2024-03-04T15:56:17.738325+0000 mon.a (mon.0) 1638 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-15", "mode": "writeback"}]': finished 2024-03-04T15:56:18.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:18 smithi117 bash[21480]: cluster 2024-03-04T15:56:17.746055+0000 mon.a (mon.0) 1639 : cluster 0 osdmap e189: 8 total, 8 up, 8 in 2024-03-04T15:56:18.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:18 smithi117 bash[21480]: audit 2024-03-04T15:56:17.760946+0000 mon.a (mon.0) 1640 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:18.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:18 smithi117 bash[21480]: audit 2024-03-04T15:56:17.767805+0000 mon.a (mon.0) 1641 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]: dispatch 2024-03-04T15:56:18.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:18 smithi151 bash[23756]: cluster 2024-03-04T15:56:17.732134+0000 mon.a (mon.0) 1637 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:56:18.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:18 smithi151 bash[23756]: audit 2024-03-04T15:56:17.738325+0000 mon.a (mon.0) 1638 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-15", "mode": "writeback"}]': finished 2024-03-04T15:56:18.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:18 smithi151 bash[23756]: cluster 2024-03-04T15:56:17.746055+0000 mon.a (mon.0) 1639 : cluster 0 osdmap e189: 8 total, 8 up, 8 in 2024-03-04T15:56:18.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:18 smithi151 bash[23756]: audit 2024-03-04T15:56:17.760946+0000 mon.a (mon.0) 1640 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:18.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:18 smithi151 bash[23756]: audit 2024-03-04T15:56:17.767805+0000 mon.a (mon.0) 1641 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]: dispatch 2024-03-04T15:56:19.772 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWritpp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-03-04T15:56:19.772 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (2005 ms) 2024-03-04T15:56:19.773 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (8941 ms total) 2024-03-04T15:56:19.773 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: 2024-03-04T15:56:19.773 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-03-04T15:56:19.773 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-03-04T15:56:19.773 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4490 ms) 2024-03-04T15:56:19.773 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-03-04T15:56:19.773 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3477 ms) 2024-03-04T15:56:19.773 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-03-04T15:56:19.773 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (4066 ms) 2024-03-04T15:56:19.773 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (12033 ms total) 2024-03-04T15:56:19.773 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: 2024-03-04T15:56:19.773 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-03-04T15:56:19.773 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [==========] 21 tests from 5 test suites ran. (139774 ms total) 2024-03-04T15:56:19.773 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-03-04T15:56:19.773 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ SKIPPED ] 1 test, listed below: 2024-03-04T15:56:19.773 INFO:tasks.workunit.client.0.smithi117.stdout: api_snapshots_pp: [ SKIPPED ] LibRadosSnapshotsSelfManagedPP.WriteRollback 2024-03-04T15:56:20.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[21480]: audit 2024-03-04T15:56:18.742186+0000 mon.a (mon.0) 1642 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]': finished 2024-03-04T15:56:20.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[21480]: cluster 2024-03-04T15:56:18.749500+0000 mon.a (mon.0) 1643 : cluster 0 osdmap e190: 8 total, 8 up, 8 in 2024-03-04T15:56:20.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[21480]: audit 2024-03-04T15:56:18.752777+0000 mon.a (mon.0) 1644 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]: dispatch 2024-03-04T15:56:20.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[21480]: audit 2024-03-04T15:56:18.754601+0000 mon.c (mon.2) 285 : audit 1 from='client.? 172.21.15.117:0/2550563509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi117-63923-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:20.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[21480]: audit 2024-03-04T15:56:18.755036+0000 mon.c (mon.2) 286 : audit 1 from='client.? 172.21.15.117:0/543328654' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:20.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[21480]: audit 2024-03-04T15:56:18.755548+0000 mon.a (mon.0) 1645 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi117-63923-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:20.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[21480]: audit 2024-03-04T15:56:18.757110+0000 mon.a (mon.0) 1646 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:20.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[21480]: audit 2024-03-04T15:56:18.763297+0000 mon.a (mon.0) 1647 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:20.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[21480]: audit 2024-03-04T15:56:18.814348+0000 mon.c (mon.2) 287 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:20.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[21480]: audit 2024-03-04T15:56:18.814805+0000 mon.a (mon.0) 1648 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:20.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[21480]: cluster 2024-03-04T15:56:18.850191+0000 mgr.y (mgr.24428) 276 : cluster 0 pgmap v291: 292 pgs: 32 unknown, 260 active+clean; 585 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 6 op/s 2024-03-04T15:56:20.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[28843]: audit 2024-03-04T15:56:18.742186+0000 mon.a (mon.0) 1642 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]': finished 2024-03-04T15:56:20.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[28843]: cluster 2024-03-04T15:56:18.749500+0000 mon.a (mon.0) 1643 : cluster 0 osdmap e190: 8 total, 8 up, 8 in 2024-03-04T15:56:20.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[28843]: audit 2024-03-04T15:56:18.752777+0000 mon.a (mon.0) 1644 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]: dispatch 2024-03-04T15:56:20.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[28843]: audit 2024-03-04T15:56:18.754601+0000 mon.c (mon.2) 285 : audit 1 from='client.? 172.21.15.117:0/2550563509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi117-63923-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:20.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[28843]: audit 2024-03-04T15:56:18.755036+0000 mon.c (mon.2) 286 : audit 1 from='client.? 172.21.15.117:0/543328654' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:20.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[28843]: audit 2024-03-04T15:56:18.755548+0000 mon.a (mon.0) 1645 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi117-63923-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:20.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[28843]: audit 2024-03-04T15:56:18.757110+0000 mon.a (mon.0) 1646 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:20.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[28843]: audit 2024-03-04T15:56:18.763297+0000 mon.a (mon.0) 1647 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:20.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[28843]: audit 2024-03-04T15:56:18.814348+0000 mon.c (mon.2) 287 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:20.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[28843]: audit 2024-03-04T15:56:18.814805+0000 mon.a (mon.0) 1648 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:20.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:19 smithi117 bash[28843]: cluster 2024-03-04T15:56:18.850191+0000 mgr.y (mgr.24428) 276 : cluster 0 pgmap v291: 292 pgs: 32 unknown, 260 active+clean; 585 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 6 op/s 2024-03-04T15:56:20.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:19 smithi151 bash[23756]: audit 2024-03-04T15:56:18.742186+0000 mon.a (mon.0) 1642 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]': finished 2024-03-04T15:56:20.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:19 smithi151 bash[23756]: cluster 2024-03-04T15:56:18.749500+0000 mon.a (mon.0) 1643 : cluster 0 osdmap e190: 8 total, 8 up, 8 in 2024-03-04T15:56:20.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:19 smithi151 bash[23756]: audit 2024-03-04T15:56:18.752777+0000 mon.a (mon.0) 1644 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]: dispatch 2024-03-04T15:56:20.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:19 smithi151 bash[23756]: audit 2024-03-04T15:56:18.754601+0000 mon.c (mon.2) 285 : audit 1 from='client.? 172.21.15.117:0/2550563509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi117-63923-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:20.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:19 smithi151 bash[23756]: audit 2024-03-04T15:56:18.755036+0000 mon.c (mon.2) 286 : audit 1 from='client.? 172.21.15.117:0/543328654' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:20.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:19 smithi151 bash[23756]: audit 2024-03-04T15:56:18.755548+0000 mon.a (mon.0) 1645 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi117-63923-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:20.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:19 smithi151 bash[23756]: audit 2024-03-04T15:56:18.757110+0000 mon.a (mon.0) 1646 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:20.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:19 smithi151 bash[23756]: audit 2024-03-04T15:56:18.763297+0000 mon.a (mon.0) 1647 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:20.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:19 smithi151 bash[23756]: audit 2024-03-04T15:56:18.814348+0000 mon.c (mon.2) 287 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:20.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:19 smithi151 bash[23756]: audit 2024-03-04T15:56:18.814805+0000 mon.a (mon.0) 1648 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:20.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:19 smithi151 bash[23756]: cluster 2024-03-04T15:56:18.850191+0000 mgr.y (mgr.24428) 276 : cluster 0 pgmap v291: 292 pgs: 32 unknown, 260 active+clean; 585 KiB data, 555 MiB used, 715 GiB / 715 GiB avail; 0 B/s wr, 6 op/s 2024-03-04T15:56:20.778 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-03-04T15:56:20.778 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-03-04T15:56:20.778 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-03-04T15:56:20.778 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-03-04T15:56:20.778 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-03-04T15:56:20.778 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (16820 ms) 2024-03-04T15:56:20.778 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-03-04T15:56:20.778 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (31443 ms) 2024-03-04T15:56:20.778 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-03-04T15:56:20.779 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (12090 ms) 2024-03-04T15:56:20.779 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-03-04T15:56:20.779 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (5591 ms) 2024-03-04T15:56:20.779 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-03-04T15:56:20.779 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (5018 ms) 2024-03-04T15:56:20.779 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-03-04T15:56:20.779 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (4258 ms) 2024-03-04T15:56:20.779 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-03-04T15:56:20.779 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (3065 ms) 2024-03-04T15:56:20.779 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-03-04T15:56:20.779 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (4711 ms) 2024-03-04T15:56:20.779 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-03-04T15:56:20.779 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (3022 ms) 2024-03-04T15:56:20.779 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-03-04T15:56:20.779 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (2499 ms) 2024-03-04T15:56:20.780 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-03-04T15:56:20.780 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (3016 ms) 2024-03-04T15:56:20.780 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-03-04T15:56:20.780 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (2991 ms) 2024-03-04T15:56:20.780 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-03-04T15:56:20.780 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (4304 ms) 2024-03-04T15:56:20.780 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-03-04T15:56:20.780 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3019 ms) 2024-03-04T15:56:20.780 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-03-04T15:56:20.780 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3042 ms) 2024-03-04T15:56:20.780 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-03-04T15:56:20.780 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3006 ms) 2024-03-04T15:56:20.780 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-03-04T15:56:20.780 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (3018 ms) 2024-03-04T15:56:20.780 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-03-04T15:56:20.781 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (2929 ms) 2024-03-04T15:56:20.781 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-03-04T15:56:20.781 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3001 ms) 2024-03-04T15:56:20.781 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-03-04T15:56:20.781 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (3031 ms) 2024-03-04T15:56:20.781 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-03-04T15:56:20.781 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3010 ms) 2024-03-04T15:56:20.781 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-03-04T15:56:20.781 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3024 ms) 2024-03-04T15:56:20.781 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-03-04T15:56:20.781 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3472 ms) 2024-03-04T15:56:20.781 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-03-04T15:56:20.781 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (2572 ms) 2024-03-04T15:56:20.781 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-03-04T15:56:20.781 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (2945 ms) 2024-03-04T15:56:20.781 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-03-04T15:56:20.782 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (2980 ms) 2024-03-04T15:56:20.782 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-03-04T15:56:21.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[21480]: cluster 2024-03-04T15:56:19.742129+0000 mon.a (mon.0) 1649 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:21.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[21480]: audit 2024-03-04T15:56:19.752866+0000 mon.a (mon.0) 1650 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]': finished 2024-03-04T15:56:21.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[21480]: audit 2024-03-04T15:56:19.753118+0000 mon.a (mon.0) 1651 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi117-63923-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:21.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[21480]: audit 2024-03-04T15:56:19.753345+0000 mon.a (mon.0) 1652 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi117-63915-28"}]': finished 2024-03-04T15:56:21.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[21480]: audit 2024-03-04T15:56:19.753498+0000 mon.a (mon.0) 1653 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:56:21.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[21480]: cluster 2024-03-04T15:56:19.763636+0000 mon.a (mon.0) 1654 : cluster 0 osdmap e191: 8 total, 8 up, 8 in 2024-03-04T15:56:21.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[21480]: audit 2024-03-04T15:56:19.766194+0000 mon.a (mon.0) 1655 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:21.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[21480]: audit 2024-03-04T15:56:19.768603+0000 mon.c (mon.2) 288 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-15"}]: dispatch 2024-03-04T15:56:21.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[21480]: audit 2024-03-04T15:56:19.769043+0000 mon.c (mon.2) 289 : audit 1 from='client.? 172.21.15.117:0/543328654' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:21.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[21480]: audit 2024-03-04T15:56:19.769764+0000 mon.a (mon.0) 1656 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-15"}]: dispatch 2024-03-04T15:56:21.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[21480]: audit 2024-03-04T15:56:19.770275+0000 mon.a (mon.0) 1657 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:21.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[28843]: cluster 2024-03-04T15:56:19.742129+0000 mon.a (mon.0) 1649 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:21.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[28843]: audit 2024-03-04T15:56:19.752866+0000 mon.a (mon.0) 1650 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]': finished 2024-03-04T15:56:21.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[28843]: audit 2024-03-04T15:56:19.753118+0000 mon.a (mon.0) 1651 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi117-63923-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:21.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[28843]: audit 2024-03-04T15:56:19.753345+0000 mon.a (mon.0) 1652 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi117-63915-28"}]': finished 2024-03-04T15:56:21.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[28843]: audit 2024-03-04T15:56:19.753498+0000 mon.a (mon.0) 1653 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:56:21.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[28843]: cluster 2024-03-04T15:56:19.763636+0000 mon.a (mon.0) 1654 : cluster 0 osdmap e191: 8 total, 8 up, 8 in 2024-03-04T15:56:21.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[28843]: audit 2024-03-04T15:56:19.766194+0000 mon.a (mon.0) 1655 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:21.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[28843]: audit 2024-03-04T15:56:19.768603+0000 mon.c (mon.2) 288 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-15"}]: dispatch 2024-03-04T15:56:21.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[28843]: audit 2024-03-04T15:56:19.769043+0000 mon.c (mon.2) 289 : audit 1 from='client.? 172.21.15.117:0/543328654' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:21.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[28843]: audit 2024-03-04T15:56:19.769764+0000 mon.a (mon.0) 1656 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-15"}]: dispatch 2024-03-04T15:56:21.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:20 smithi117 bash[28843]: audit 2024-03-04T15:56:19.770275+0000 mon.a (mon.0) 1657 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:21.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:20 smithi151 bash[23756]: cluster 2024-03-04T15:56:19.742129+0000 mon.a (mon.0) 1649 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:21.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:20 smithi151 bash[23756]: audit 2024-03-04T15:56:19.752866+0000 mon.a (mon.0) 1650 : audit 1 from='client.? 172.21.15.117:0/3922160823' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi117-64156-21"}]': finished 2024-03-04T15:56:21.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:20 smithi151 bash[23756]: audit 2024-03-04T15:56:19.753118+0000 mon.a (mon.0) 1651 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi117-63923-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:21.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:20 smithi151 bash[23756]: audit 2024-03-04T15:56:19.753345+0000 mon.a (mon.0) 1652 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi117-63915-28"}]': finished 2024-03-04T15:56:21.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:20 smithi151 bash[23756]: audit 2024-03-04T15:56:19.753498+0000 mon.a (mon.0) 1653 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:56:21.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:20 smithi151 bash[23756]: cluster 2024-03-04T15:56:19.763636+0000 mon.a (mon.0) 1654 : cluster 0 osdmap e191: 8 total, 8 up, 8 in 2024-03-04T15:56:21.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:20 smithi151 bash[23756]: audit 2024-03-04T15:56:19.766194+0000 mon.a (mon.0) 1655 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:21.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:20 smithi151 bash[23756]: audit 2024-03-04T15:56:19.768603+0000 mon.c (mon.2) 288 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-15"}]: dispatch 2024-03-04T15:56:21.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:20 smithi151 bash[23756]: audit 2024-03-04T15:56:19.769043+0000 mon.c (mon.2) 289 : audit 1 from='client.? 172.21.15.117:0/543328654' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:21.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:20 smithi151 bash[23756]: audit 2024-03-04T15:56:19.769764+0000 mon.a (mon.0) 1656 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-15"}]: dispatch 2024-03-04T15:56:21.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:20 smithi151 bash[23756]: audit 2024-03-04T15:56:19.770275+0000 mon.a (mon.0) 1657 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi117-63915-28"}]: dispatch 2024-03-04T15:56:22.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[28843]: cluster 2024-03-04T15:56:20.754940+0000 mon.a (mon.0) 1658 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:56:22.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[28843]: audit 2024-03-04T15:56:20.766846+0000 mon.a (mon.0) 1659 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-15"}]': finished 2024-03-04T15:56:22.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[28843]: audit 2024-03-04T15:56:20.766981+0000 mon.a (mon.0) 1660 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi117-63915-28"}]': finished 2024-03-04T15:56:22.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[28843]: cluster 2024-03-04T15:56:20.772828+0000 mon.a (mon.0) 1661 : cluster 0 osdmap e192: 8 total, 8 up, 8 in 2024-03-04T15:56:22.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[28843]: audit 2024-03-04T15:56:20.773989+0000 mon.a (mon.0) 1662 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:22.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[28843]: audit 2024-03-04T15:56:20.818312+0000 mon.c (mon.2) 290 : audit 1 from='client.? 172.21.15.117:0/4129928719' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:22.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[28843]: cluster 2024-03-04T15:56:20.850701+0000 mgr.y (mgr.24428) 277 : cluster 0 pgmap v294: 260 pgs: 260 active+clean; 585 KiB data, 555 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:56:22.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[28843]: audit 2024-03-04T15:56:20.851132+0000 mon.c (mon.2) 291 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi117-63955-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-04T15:56:22.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[28843]: audit 2024-03-04T15:56:21.255381+0000 mon.a (mon.0) 1663 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:22.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[28843]: audit 2024-03-04T15:56:21.256322+0000 mon.c (mon.2) 292 : audit 1 from='client.? 172.21.15.117:0/4129928719' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:22.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[28843]: audit 2024-03-04T15:56:21.258384+0000 mon.a (mon.0) 1664 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi117-63955-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-04T15:56:22.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[28843]: audit 2024-03-04T15:56:21.259003+0000 mon.a (mon.0) 1665 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:22.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[28843]: audit 2024-03-04T15:56:21.259700+0000 mon.c (mon.2) 293 : audit 1 from='client.? 172.21.15.117:0/4129928719' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi117-63915-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:22.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[28843]: audit 2024-03-04T15:56:21.260189+0000 mon.a (mon.0) 1666 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi117-63915-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:22.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[21480]: cluster 2024-03-04T15:56:20.754940+0000 mon.a (mon.0) 1658 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:56:22.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[21480]: audit 2024-03-04T15:56:20.766846+0000 mon.a (mon.0) 1659 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-15"}]': finished 2024-03-04T15:56:22.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[21480]: audit 2024-03-04T15:56:20.766981+0000 mon.a (mon.0) 1660 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi117-63915-28"}]': finished 2024-03-04T15:56:22.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[21480]: cluster 2024-03-04T15:56:20.772828+0000 mon.a (mon.0) 1661 : cluster 0 osdmap e192: 8 total, 8 up, 8 in 2024-03-04T15:56:22.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[21480]: audit 2024-03-04T15:56:20.773989+0000 mon.a (mon.0) 1662 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:22.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[21480]: audit 2024-03-04T15:56:20.818312+0000 mon.c (mon.2) 290 : audit 1 from='client.? 172.21.15.117:0/4129928719' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:22.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[21480]: cluster 2024-03-04T15:56:20.850701+0000 mgr.y (mgr.24428) 277 : cluster 0 pgmap v294: 260 pgs: 260 active+clean; 585 KiB data, 555 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:56:22.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[21480]: audit 2024-03-04T15:56:20.851132+0000 mon.c (mon.2) 291 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi117-63955-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-04T15:56:22.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[21480]: audit 2024-03-04T15:56:21.255381+0000 mon.a (mon.0) 1663 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:22.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[21480]: audit 2024-03-04T15:56:21.256322+0000 mon.c (mon.2) 292 : audit 1 from='client.? 172.21.15.117:0/4129928719' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:22.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[21480]: audit 2024-03-04T15:56:21.258384+0000 mon.a (mon.0) 1664 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi117-63955-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-04T15:56:22.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[21480]: audit 2024-03-04T15:56:21.259003+0000 mon.a (mon.0) 1665 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:22.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[21480]: audit 2024-03-04T15:56:21.259700+0000 mon.c (mon.2) 293 : audit 1 from='client.? 172.21.15.117:0/4129928719' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi117-63915-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:22.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:21 smithi117 bash[21480]: audit 2024-03-04T15:56:21.260189+0000 mon.a (mon.0) 1666 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi117-63915-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:22.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:21 smithi151 bash[23756]: cluster 2024-03-04T15:56:20.754940+0000 mon.a (mon.0) 1658 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:56:22.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:21 smithi151 bash[23756]: audit 2024-03-04T15:56:20.766846+0000 mon.a (mon.0) 1659 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-15"}]': finished 2024-03-04T15:56:22.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:21 smithi151 bash[23756]: audit 2024-03-04T15:56:20.766981+0000 mon.a (mon.0) 1660 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi117-63915-28"}]': finished 2024-03-04T15:56:22.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:21 smithi151 bash[23756]: cluster 2024-03-04T15:56:20.772828+0000 mon.a (mon.0) 1661 : cluster 0 osdmap e192: 8 total, 8 up, 8 in 2024-03-04T15:56:22.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:21 smithi151 bash[23756]: audit 2024-03-04T15:56:20.773989+0000 mon.a (mon.0) 1662 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:22.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:21 smithi151 bash[23756]: audit 2024-03-04T15:56:20.818312+0000 mon.c (mon.2) 290 : audit 1 from='client.? 172.21.15.117:0/4129928719' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:22.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:21 smithi151 bash[23756]: cluster 2024-03-04T15:56:20.850701+0000 mgr.y (mgr.24428) 277 : cluster 0 pgmap v294: 260 pgs: 260 active+clean; 585 KiB data, 555 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:56:22.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:21 smithi151 bash[23756]: audit 2024-03-04T15:56:20.851132+0000 mon.c (mon.2) 291 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi117-63955-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-04T15:56:22.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:21 smithi151 bash[23756]: audit 2024-03-04T15:56:21.255381+0000 mon.a (mon.0) 1663 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:22.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:21 smithi151 bash[23756]: audit 2024-03-04T15:56:21.256322+0000 mon.c (mon.2) 292 : audit 1 from='client.? 172.21.15.117:0/4129928719' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:22.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:21 smithi151 bash[23756]: audit 2024-03-04T15:56:21.258384+0000 mon.a (mon.0) 1664 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi117-63955-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-03-04T15:56:22.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:21 smithi151 bash[23756]: audit 2024-03-04T15:56:21.259003+0000 mon.a (mon.0) 1665 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:22.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:21 smithi151 bash[23756]: audit 2024-03-04T15:56:21.259700+0000 mon.c (mon.2) 293 : audit 1 from='client.? 172.21.15.117:0/4129928719' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi117-63915-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:22.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:21 smithi151 bash[23756]: audit 2024-03-04T15:56:21.260189+0000 mon.a (mon.0) 1666 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi117-63915-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:23.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:22 smithi117 bash[28843]: audit 2024-03-04T15:56:21.778555+0000 mon.a (mon.0) 1667 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:23.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:22 smithi117 bash[28843]: audit 2024-03-04T15:56:21.779969+0000 mon.a (mon.0) 1668 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:23.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:22 smithi117 bash[28843]: audit 2024-03-04T15:56:21.781292+0000 mon.a (mon.0) 1669 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-04T15:56:23.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:22 smithi117 bash[28843]: audit 2024-03-04T15:56:21.782115+0000 mon.a (mon.0) 1670 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi117-63955-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-03-04T15:56:23.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:22 smithi117 bash[28843]: audit 2024-03-04T15:56:21.782291+0000 mon.a (mon.0) 1671 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi117-63915-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:23.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:22 smithi117 bash[28843]: cluster 2024-03-04T15:56:21.791854+0000 mon.a (mon.0) 1672 : cluster 0 osdmap e193: 8 total, 8 up, 8 in 2024-03-04T15:56:23.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:22 smithi117 bash[28843]: audit 2024-03-04T15:56:21.794830+0000 mon.c (mon.2) 294 : audit 1 from='client.? 172.21.15.117:0/4129928719' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi117-63915-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:23.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:22 smithi117 bash[28843]: audit 2024-03-04T15:56:21.809135+0000 mon.a (mon.0) 1673 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi117-63915-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:23.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:22 smithi117 bash[28843]: audit 2024-03-04T15:56:21.814106+0000 mon.c (mon.2) 295 : audit 1 from='client.? 172.21.15.117:0/3600239616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi117-63923-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:23.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:22 smithi117 bash[28843]: audit 2024-03-04T15:56:22.269301+0000 mon.a (mon.0) 1674 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi117-63923-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:23.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:22 smithi117 bash[21480]: audit 2024-03-04T15:56:21.778555+0000 mon.a (mon.0) 1667 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:23.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:22 smithi117 bash[21480]: audit 2024-03-04T15:56:21.779969+0000 mon.a (mon.0) 1668 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:23.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:22 smithi117 bash[21480]: audit 2024-03-04T15:56:21.781292+0000 mon.a (mon.0) 1669 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-04T15:56:23.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:22 smithi117 bash[21480]: audit 2024-03-04T15:56:21.782115+0000 mon.a (mon.0) 1670 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi117-63955-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-03-04T15:56:23.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:22 smithi117 bash[21480]: audit 2024-03-04T15:56:21.782291+0000 mon.a (mon.0) 1671 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi117-63915-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:23.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:22 smithi117 bash[21480]: cluster 2024-03-04T15:56:21.791854+0000 mon.a (mon.0) 1672 : cluster 0 osdmap e193: 8 total, 8 up, 8 in 2024-03-04T15:56:23.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:22 smithi117 bash[21480]: audit 2024-03-04T15:56:21.794830+0000 mon.c (mon.2) 294 : audit 1 from='client.? 172.21.15.117:0/4129928719' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi117-63915-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:23.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:22 smithi117 bash[21480]: audit 2024-03-04T15:56:21.809135+0000 mon.a (mon.0) 1673 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi117-63915-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:23.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:22 smithi117 bash[21480]: audit 2024-03-04T15:56:21.814106+0000 mon.c (mon.2) 295 : audit 1 from='client.? 172.21.15.117:0/3600239616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi117-63923-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:23.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:22 smithi117 bash[21480]: audit 2024-03-04T15:56:22.269301+0000 mon.a (mon.0) 1674 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi117-63923-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:23.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:22 smithi151 bash[23756]: audit 2024-03-04T15:56:21.778555+0000 mon.a (mon.0) 1667 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:23.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:22 smithi151 bash[23756]: audit 2024-03-04T15:56:21.779969+0000 mon.a (mon.0) 1668 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:23.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:22 smithi151 bash[23756]: audit 2024-03-04T15:56:21.781292+0000 mon.a (mon.0) 1669 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pgp_num","val":"11"}]: dispatch 2024-03-04T15:56:23.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:22 smithi151 bash[23756]: audit 2024-03-04T15:56:21.782115+0000 mon.a (mon.0) 1670 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi117-63955-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-03-04T15:56:23.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:22 smithi151 bash[23756]: audit 2024-03-04T15:56:21.782291+0000 mon.a (mon.0) 1671 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi117-63915-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:23.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:22 smithi151 bash[23756]: cluster 2024-03-04T15:56:21.791854+0000 mon.a (mon.0) 1672 : cluster 0 osdmap e193: 8 total, 8 up, 8 in 2024-03-04T15:56:23.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:22 smithi151 bash[23756]: audit 2024-03-04T15:56:21.794830+0000 mon.c (mon.2) 294 : audit 1 from='client.? 172.21.15.117:0/4129928719' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi117-63915-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:23.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:22 smithi151 bash[23756]: audit 2024-03-04T15:56:21.809135+0000 mon.a (mon.0) 1673 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi117-63915-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:23.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:22 smithi151 bash[23756]: audit 2024-03-04T15:56:21.814106+0000 mon.c (mon.2) 295 : audit 1 from='client.? 172.21.15.117:0/3600239616' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi117-63923-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:23.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:22 smithi151 bash[23756]: audit 2024-03-04T15:56:22.269301+0000 mon.a (mon.0) 1674 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi117-63923-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:24.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:23 smithi117 bash[21480]: audit 2024-03-04T15:56:22.790809+0000 mon.a (mon.0) 1675 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pgp_num","val":"11"}]': finished 2024-03-04T15:56:24.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:23 smithi117 bash[21480]: audit 2024-03-04T15:56:22.791004+0000 mon.a (mon.0) 1676 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi117-63923-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:24.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:23 smithi117 bash[21480]: cluster 2024-03-04T15:56:22.851798+0000 mgr.y (mgr.24428) 278 : cluster 0 pgmap v297: 292 pgs: 64 unknown, 228 active+clean; 585 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:56:24.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:23 smithi117 bash[21480]: cluster 2024-03-04T15:56:23.255358+0000 mon.a (mon.0) 1677 : cluster 0 osdmap e194: 8 total, 8 up, 8 in 2024-03-04T15:56:24.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:23 smithi117 bash[21480]: audit 2024-03-04T15:56:23.256877+0000 mon.a (mon.0) 1678 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:24.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:23 smithi117 bash[21480]: audit 2024-03-04T15:56:23.257256+0000 mon.c (mon.2) 296 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:24.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:23 smithi117 bash[21480]: audit 2024-03-04T15:56:23.268734+0000 mon.a (mon.0) 1679 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:23 smithi117 bash[28843]: audit 2024-03-04T15:56:22.790809+0000 mon.a (mon.0) 1675 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pgp_num","val":"11"}]': finished 2024-03-04T15:56:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:23 smithi117 bash[28843]: audit 2024-03-04T15:56:22.791004+0000 mon.a (mon.0) 1676 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi117-63923-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:23 smithi117 bash[28843]: cluster 2024-03-04T15:56:22.851798+0000 mgr.y (mgr.24428) 278 : cluster 0 pgmap v297: 292 pgs: 64 unknown, 228 active+clean; 585 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:56:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:23 smithi117 bash[28843]: cluster 2024-03-04T15:56:23.255358+0000 mon.a (mon.0) 1677 : cluster 0 osdmap e194: 8 total, 8 up, 8 in 2024-03-04T15:56:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:23 smithi117 bash[28843]: audit 2024-03-04T15:56:23.256877+0000 mon.a (mon.0) 1678 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:23 smithi117 bash[28843]: audit 2024-03-04T15:56:23.257256+0000 mon.c (mon.2) 296 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:24.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:23 smithi117 bash[28843]: audit 2024-03-04T15:56:23.268734+0000 mon.a (mon.0) 1679 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:23 smithi151 bash[23756]: audit 2024-03-04T15:56:22.790809+0000 mon.a (mon.0) 1675 : audit 1 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi117-63955-1","var":"pgp_num","val":"11"}]': finished 2024-03-04T15:56:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:23 smithi151 bash[23756]: audit 2024-03-04T15:56:22.791004+0000 mon.a (mon.0) 1676 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi117-63923-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:23 smithi151 bash[23756]: cluster 2024-03-04T15:56:22.851798+0000 mgr.y (mgr.24428) 278 : cluster 0 pgmap v297: 292 pgs: 64 unknown, 228 active+clean; 585 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:56:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:23 smithi151 bash[23756]: cluster 2024-03-04T15:56:23.255358+0000 mon.a (mon.0) 1677 : cluster 0 osdmap e194: 8 total, 8 up, 8 in 2024-03-04T15:56:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:23 smithi151 bash[23756]: audit 2024-03-04T15:56:23.256877+0000 mon.a (mon.0) 1678 : audit 0 from='client.? 172.21.15.117:0/2298950076' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-03-04T15:56:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:23 smithi151 bash[23756]: audit 2024-03-04T15:56:23.257256+0000 mon.c (mon.2) 296 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:23 smithi151 bash[23756]: audit 2024-03-04T15:56:23.268734+0000 mon.a (mon.0) 1679 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:25.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:24 smithi151 bash[23756]: audit 2024-03-04T15:56:23.803109+0000 mon.a (mon.0) 1680 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi117-63915-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi117-63915-29"}]': finished 2024-03-04T15:56:25.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:24 smithi151 bash[23756]: audit 2024-03-04T15:56:23.803371+0000 mon.a (mon.0) 1681 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:25.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:24 smithi151 bash[23756]: cluster 2024-03-04T15:56:23.815252+0000 mon.a (mon.0) 1682 : cluster 0 osdmap e195: 8 total, 8 up, 8 in 2024-03-04T15:56:25.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:24 smithi151 bash[23756]: audit 2024-03-04T15:56:23.887423+0000 mon.c (mon.2) 297 : audit 1 from='client.? 172.21.15.117:0/783728534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:25.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:24 smithi151 bash[23756]: audit 2024-03-04T15:56:23.890084+0000 mon.c (mon.2) 298 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:25.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:24 smithi151 bash[23756]: audit 2024-03-04T15:56:24.314444+0000 mon.a (mon.0) 1683 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:25.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:24 smithi151 bash[23756]: audit 2024-03-04T15:56:24.315226+0000 mon.a (mon.0) 1684 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:25.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:24 smithi151 bash[23756]: audit 2024-03-04T15:56:24.315742+0000 mon.c (mon.2) 299 : audit 1 from='client.? 172.21.15.117:0/783728534' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:25.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:24 smithi151 bash[23756]: audit 2024-03-04T15:56:24.316495+0000 mon.a (mon.0) 1685 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:25.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:24 smithi151 bash[23756]: audit 2024-03-04T15:56:24.317243+0000 mon.c (mon.2) 300 : audit 1 from='client.? 172.21.15.117:0/783728534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi117-63955-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:25.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:24 smithi151 bash[23756]: audit 2024-03-04T15:56:24.317671+0000 mon.a (mon.0) 1686 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi117-63955-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:25.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[21480]: audit 2024-03-04T15:56:23.803109+0000 mon.a (mon.0) 1680 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi117-63915-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi117-63915-29"}]': finished 2024-03-04T15:56:25.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[21480]: audit 2024-03-04T15:56:23.803371+0000 mon.a (mon.0) 1681 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:25.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[21480]: cluster 2024-03-04T15:56:23.815252+0000 mon.a (mon.0) 1682 : cluster 0 osdmap e195: 8 total, 8 up, 8 in 2024-03-04T15:56:25.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[21480]: audit 2024-03-04T15:56:23.887423+0000 mon.c (mon.2) 297 : audit 1 from='client.? 172.21.15.117:0/783728534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:25.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[21480]: audit 2024-03-04T15:56:23.890084+0000 mon.c (mon.2) 298 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:25.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[21480]: audit 2024-03-04T15:56:24.314444+0000 mon.a (mon.0) 1683 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:25.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[21480]: audit 2024-03-04T15:56:24.315226+0000 mon.a (mon.0) 1684 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:25.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[21480]: audit 2024-03-04T15:56:24.315742+0000 mon.c (mon.2) 299 : audit 1 from='client.? 172.21.15.117:0/783728534' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:25.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[21480]: audit 2024-03-04T15:56:24.316495+0000 mon.a (mon.0) 1685 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:25.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[21480]: audit 2024-03-04T15:56:24.317243+0000 mon.c (mon.2) 300 : audit 1 from='client.? 172.21.15.117:0/783728534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi117-63955-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:25.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[21480]: audit 2024-03-04T15:56:24.317671+0000 mon.a (mon.0) 1686 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi117-63955-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:25.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[28843]: audit 2024-03-04T15:56:23.803109+0000 mon.a (mon.0) 1680 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi117-63915-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi117-63915-29"}]': finished 2024-03-04T15:56:25.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[28843]: audit 2024-03-04T15:56:23.803371+0000 mon.a (mon.0) 1681 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:25.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[28843]: cluster 2024-03-04T15:56:23.815252+0000 mon.a (mon.0) 1682 : cluster 0 osdmap e195: 8 total, 8 up, 8 in 2024-03-04T15:56:25.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[28843]: audit 2024-03-04T15:56:23.887423+0000 mon.c (mon.2) 297 : audit 1 from='client.? 172.21.15.117:0/783728534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:25.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[28843]: audit 2024-03-04T15:56:23.890084+0000 mon.c (mon.2) 298 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:25.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[28843]: audit 2024-03-04T15:56:24.314444+0000 mon.a (mon.0) 1683 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:25.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[28843]: audit 2024-03-04T15:56:24.315226+0000 mon.a (mon.0) 1684 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:25.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[28843]: audit 2024-03-04T15:56:24.315742+0000 mon.c (mon.2) 299 : audit 1 from='client.? 172.21.15.117:0/783728534' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:25.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[28843]: audit 2024-03-04T15:56:24.316495+0000 mon.a (mon.0) 1685 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:25.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[28843]: audit 2024-03-04T15:56:24.317243+0000 mon.c (mon.2) 300 : audit 1 from='client.? 172.21.15.117:0/783728534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi117-63955-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:25.321 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:24 smithi117 bash[28843]: audit 2024-03-04T15:56:24.317671+0000 mon.a (mon.0) 1686 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi117-63955-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:26.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:56:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:56:26.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[21480]: audit 2024-03-04T15:56:24.807349+0000 mon.a (mon.0) 1687 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-17", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:56:26.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[21480]: audit 2024-03-04T15:56:24.807623+0000 mon.a (mon.0) 1688 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi117-63955-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:26.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[21480]: cluster 2024-03-04T15:56:24.818321+0000 mon.a (mon.0) 1689 : cluster 0 osdmap e196: 8 total, 8 up, 8 in 2024-03-04T15:56:26.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[21480]: audit 2024-03-04T15:56:24.820376+0000 mon.c (mon.2) 301 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-17"}]: dispatch 2024-03-04T15:56:26.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[21480]: audit 2024-03-04T15:56:24.823076+0000 mon.c (mon.2) 302 : audit 1 from='client.? 172.21.15.117:0/783728534' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi117-63955-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:26.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[21480]: audit 2024-03-04T15:56:24.824825+0000 mon.a (mon.0) 1690 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-17"}]: dispatch 2024-03-04T15:56:26.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[21480]: audit 2024-03-04T15:56:24.825537+0000 mon.c (mon.2) 303 : audit 1 from='client.? 172.21.15.117:0/1728395483' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi117-63923-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:26.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[21480]: audit 2024-03-04T15:56:24.838096+0000 mon.a (mon.0) 1691 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi117-63955-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:26.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[21480]: audit 2024-03-04T15:56:24.840031+0000 mon.a (mon.0) 1692 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi117-63923-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:26.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[21480]: cluster 2024-03-04T15:56:24.852591+0000 mgr.y (mgr.24428) 279 : cluster 0 pgmap v300: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:56:26.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[21480]: audit 2024-03-04T15:56:25.811258+0000 mon.a (mon.0) 1693 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-17"}]': finished 2024-03-04T15:56:26.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[28843]: audit 2024-03-04T15:56:24.807349+0000 mon.a (mon.0) 1687 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-17", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:56:26.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[28843]: audit 2024-03-04T15:56:24.807623+0000 mon.a (mon.0) 1688 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi117-63955-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:26.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[28843]: cluster 2024-03-04T15:56:24.818321+0000 mon.a (mon.0) 1689 : cluster 0 osdmap e196: 8 total, 8 up, 8 in 2024-03-04T15:56:26.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[28843]: audit 2024-03-04T15:56:24.820376+0000 mon.c (mon.2) 301 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-17"}]: dispatch 2024-03-04T15:56:26.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[28843]: audit 2024-03-04T15:56:24.823076+0000 mon.c (mon.2) 302 : audit 1 from='client.? 172.21.15.117:0/783728534' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi117-63955-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:26.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[28843]: audit 2024-03-04T15:56:24.824825+0000 mon.a (mon.0) 1690 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-17"}]: dispatch 2024-03-04T15:56:26.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[28843]: audit 2024-03-04T15:56:24.825537+0000 mon.c (mon.2) 303 : audit 1 from='client.? 172.21.15.117:0/1728395483' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi117-63923-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:26.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[28843]: audit 2024-03-04T15:56:24.838096+0000 mon.a (mon.0) 1691 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi117-63955-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:26.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[28843]: audit 2024-03-04T15:56:24.840031+0000 mon.a (mon.0) 1692 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi117-63923-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:26.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[28843]: cluster 2024-03-04T15:56:24.852591+0000 mgr.y (mgr.24428) 279 : cluster 0 pgmap v300: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:56:26.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:26 smithi117 bash[28843]: audit 2024-03-04T15:56:25.811258+0000 mon.a (mon.0) 1693 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-17"}]': finished 2024-03-04T15:56:26.578 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:26 smithi151 bash[23756]: audit 2024-03-04T15:56:24.807349+0000 mon.a (mon.0) 1687 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-17", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:56:26.578 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:26 smithi151 bash[23756]: audit 2024-03-04T15:56:24.807623+0000 mon.a (mon.0) 1688 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi117-63955-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:26.578 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:26 smithi151 bash[23756]: cluster 2024-03-04T15:56:24.818321+0000 mon.a (mon.0) 1689 : cluster 0 osdmap e196: 8 total, 8 up, 8 in 2024-03-04T15:56:26.578 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:26 smithi151 bash[23756]: audit 2024-03-04T15:56:24.820376+0000 mon.c (mon.2) 301 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-17"}]: dispatch 2024-03-04T15:56:26.578 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:26 smithi151 bash[23756]: audit 2024-03-04T15:56:24.823076+0000 mon.c (mon.2) 302 : audit 1 from='client.? 172.21.15.117:0/783728534' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi117-63955-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:26.578 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:26 smithi151 bash[23756]: audit 2024-03-04T15:56:24.824825+0000 mon.a (mon.0) 1690 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-17"}]: dispatch 2024-03-04T15:56:26.578 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:26 smithi151 bash[23756]: audit 2024-03-04T15:56:24.825537+0000 mon.c (mon.2) 303 : audit 1 from='client.? 172.21.15.117:0/1728395483' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi117-63923-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:26.578 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:26 smithi151 bash[23756]: audit 2024-03-04T15:56:24.838096+0000 mon.a (mon.0) 1691 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi117-63955-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:26.579 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:26 smithi151 bash[23756]: audit 2024-03-04T15:56:24.840031+0000 mon.a (mon.0) 1692 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi117-63923-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:26.579 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:26 smithi151 bash[23756]: cluster 2024-03-04T15:56:24.852591+0000 mgr.y (mgr.24428) 279 : cluster 0 pgmap v300: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:56:26.579 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:26 smithi151 bash[23756]: audit 2024-03-04T15:56:25.811258+0000 mon.a (mon.0) 1693 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-17"}]': finished 2024-03-04T15:56:26.932 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:56:26 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:56:27.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[21480]: audit 2024-03-04T15:56:25.811522+0000 mon.a (mon.0) 1694 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi117-63923-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:27.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[21480]: audit 2024-03-04T15:56:25.822001+0000 mon.c (mon.2) 304 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-17", "mode": "writeback"}]: dispatch 2024-03-04T15:56:27.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[21480]: cluster 2024-03-04T15:56:26.272750+0000 mon.a (mon.0) 1695 : cluster 0 osdmap e197: 8 total, 8 up, 8 in 2024-03-04T15:56:27.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[21480]: cluster 2024-03-04T15:56:26.273220+0000 mon.a (mon.0) 1696 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:27.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[21480]: audit 2024-03-04T15:56:26.274658+0000 mon.a (mon.0) 1697 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-17", "mode": "writeback"}]: dispatch 2024-03-04T15:56:27.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[21480]: audit 2024-03-04T15:56:26.274887+0000 mon.c (mon.2) 305 : audit 1 from='client.? 172.21.15.117:0/4129928719' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:27.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[21480]: audit 2024-03-04T15:56:26.294345+0000 mon.a (mon.0) 1698 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:27.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[21480]: cluster 2024-03-04T15:56:26.811996+0000 mon.a (mon.0) 1699 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:56:27.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[21480]: audit 2024-03-04T15:56:26.814548+0000 mon.a (mon.0) 1700 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi117-63955-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi117-63955-2"}]': finished 2024-03-04T15:56:27.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[21480]: audit 2024-03-04T15:56:26.814841+0000 mon.a (mon.0) 1701 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-17", "mode": "writeback"}]': finished 2024-03-04T15:56:27.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[21480]: audit 2024-03-04T15:56:26.815033+0000 mon.a (mon.0) 1702 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi117-63915-29"}]': finished 2024-03-04T15:56:27.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[21480]: cluster 2024-03-04T15:56:26.823688+0000 mon.a (mon.0) 1703 : cluster 0 osdmap e198: 8 total, 8 up, 8 in 2024-03-04T15:56:27.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[21480]: audit 2024-03-04T15:56:26.825477+0000 mon.c (mon.2) 306 : audit 1 from='client.? 172.21.15.117:0/4129928719' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:27.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[21480]: audit 2024-03-04T15:56:26.826714+0000 mon.a (mon.0) 1704 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:27.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[28843]: audit 2024-03-04T15:56:25.811522+0000 mon.a (mon.0) 1694 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi117-63923-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:27.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[28843]: audit 2024-03-04T15:56:25.822001+0000 mon.c (mon.2) 304 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-17", "mode": "writeback"}]: dispatch 2024-03-04T15:56:27.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[28843]: cluster 2024-03-04T15:56:26.272750+0000 mon.a (mon.0) 1695 : cluster 0 osdmap e197: 8 total, 8 up, 8 in 2024-03-04T15:56:27.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[28843]: cluster 2024-03-04T15:56:26.273220+0000 mon.a (mon.0) 1696 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:27.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[28843]: audit 2024-03-04T15:56:26.274658+0000 mon.a (mon.0) 1697 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-17", "mode": "writeback"}]: dispatch 2024-03-04T15:56:27.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[28843]: audit 2024-03-04T15:56:26.274887+0000 mon.c (mon.2) 305 : audit 1 from='client.? 172.21.15.117:0/4129928719' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:27.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[28843]: audit 2024-03-04T15:56:26.294345+0000 mon.a (mon.0) 1698 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:27.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[28843]: cluster 2024-03-04T15:56:26.811996+0000 mon.a (mon.0) 1699 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:56:27.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[28843]: audit 2024-03-04T15:56:26.814548+0000 mon.a (mon.0) 1700 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi117-63955-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi117-63955-2"}]': finished 2024-03-04T15:56:27.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[28843]: audit 2024-03-04T15:56:26.814841+0000 mon.a (mon.0) 1701 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-17", "mode": "writeback"}]': finished 2024-03-04T15:56:27.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[28843]: audit 2024-03-04T15:56:26.815033+0000 mon.a (mon.0) 1702 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi117-63915-29"}]': finished 2024-03-04T15:56:27.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[28843]: cluster 2024-03-04T15:56:26.823688+0000 mon.a (mon.0) 1703 : cluster 0 osdmap e198: 8 total, 8 up, 8 in 2024-03-04T15:56:27.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[28843]: audit 2024-03-04T15:56:26.825477+0000 mon.c (mon.2) 306 : audit 1 from='client.? 172.21.15.117:0/4129928719' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:27.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:27 smithi117 bash[28843]: audit 2024-03-04T15:56:26.826714+0000 mon.a (mon.0) 1704 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:27.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:27 smithi151 bash[23756]: audit 2024-03-04T15:56:25.811522+0000 mon.a (mon.0) 1694 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi117-63923-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:27.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:27 smithi151 bash[23756]: audit 2024-03-04T15:56:25.822001+0000 mon.c (mon.2) 304 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-17", "mode": "writeback"}]: dispatch 2024-03-04T15:56:27.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:27 smithi151 bash[23756]: cluster 2024-03-04T15:56:26.272750+0000 mon.a (mon.0) 1695 : cluster 0 osdmap e197: 8 total, 8 up, 8 in 2024-03-04T15:56:27.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:27 smithi151 bash[23756]: cluster 2024-03-04T15:56:26.273220+0000 mon.a (mon.0) 1696 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:27.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:27 smithi151 bash[23756]: audit 2024-03-04T15:56:26.274658+0000 mon.a (mon.0) 1697 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-17", "mode": "writeback"}]: dispatch 2024-03-04T15:56:27.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:27 smithi151 bash[23756]: audit 2024-03-04T15:56:26.274887+0000 mon.c (mon.2) 305 : audit 1 from='client.? 172.21.15.117:0/4129928719' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:27.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:27 smithi151 bash[23756]: audit 2024-03-04T15:56:26.294345+0000 mon.a (mon.0) 1698 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:27.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:27 smithi151 bash[23756]: cluster 2024-03-04T15:56:26.811996+0000 mon.a (mon.0) 1699 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:56:27.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:27 smithi151 bash[23756]: audit 2024-03-04T15:56:26.814548+0000 mon.a (mon.0) 1700 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi117-63955-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi117-63955-2"}]': finished 2024-03-04T15:56:27.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:27 smithi151 bash[23756]: audit 2024-03-04T15:56:26.814841+0000 mon.a (mon.0) 1701 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-17", "mode": "writeback"}]': finished 2024-03-04T15:56:27.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:27 smithi151 bash[23756]: audit 2024-03-04T15:56:26.815033+0000 mon.a (mon.0) 1702 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi117-63915-29"}]': finished 2024-03-04T15:56:27.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:27 smithi151 bash[23756]: cluster 2024-03-04T15:56:26.823688+0000 mon.a (mon.0) 1703 : cluster 0 osdmap e198: 8 total, 8 up, 8 in 2024-03-04T15:56:27.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:27 smithi151 bash[23756]: audit 2024-03-04T15:56:26.825477+0000 mon.c (mon.2) 306 : audit 1 from='client.? 172.21.15.117:0/4129928719' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:27.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:27 smithi151 bash[23756]: audit 2024-03-04T15:56:26.826714+0000 mon.a (mon.0) 1704 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi117-63915-29"}]: dispatch 2024-03-04T15:56:27.861 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAed=7 2024-03-04T15:56:27.861 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:02547ec2:::1:head 2024-03-04T15:56:27.861 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : cursor()=39:02547ec2:::1:head expected=39:02547ec2:::1:head 2024-03-04T15:56:27.861 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:02547ec2:::1:head -> 1 2024-03-04T15:56:27.861 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : entry=1 expected=1 2024-03-04T15:56:27.861 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:52ea6a34:::10:head 2024-03-04T15:56:27.861 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : cursor()=39:52ea6a34:::10:head expected=39:52ea6a34:::10:head 2024-03-04T15:56:27.861 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:52ea6a34:::10:head -> 10 2024-03-04T15:56:27.862 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : entry=10 expected=10 2024-03-04T15:56:27.862 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:b29083e3:::5:head 2024-03-04T15:56:27.862 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : cursor()=39:b29083e3:::5:head expected=39:b29083e3:::5:head 2024-03-04T15:56:27.862 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:b29083e3:::5:head -> 5 2024-03-04T15:56:27.862 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : entry=5 expected=5 2024-03-04T15:56:27.862 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:89d3ae78:::11:head 2024-03-04T15:56:27.862 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : cursor()=39:89d3ae78:::11:head expected=39:89d3ae78:::11:head 2024-03-04T15:56:27.862 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:89d3ae78:::11:head -> 11 2024-03-04T15:56:27.862 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : entry=11 expected=11 2024-03-04T15:56:27.862 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:863748b0:::15:head 2024-03-04T15:56:27.862 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : cursor()=39:863748b0:::15:head expected=39:863748b0:::15:head 2024-03-04T15:56:27.862 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:863748b0:::15:head -> 15 2024-03-04T15:56:27.862 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : entry=15 expected=15 2024-03-04T15:56:27.862 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:6cac518f:::0:head 2024-03-04T15:56:27.863 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : cursor()=39:6cac518f:::0:head expected=39:6cac518f:::0:head 2024-03-04T15:56:27.863 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:6cac518f:::0:head -> 0 2024-03-04T15:56:27.863 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : entry=0 expected=0 2024-03-04T15:56:27.863 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:62a1935d:::14:head 2024-03-04T15:56:27.863 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : cursor()=39:62a1935d:::14:head expected=39:62a1935d:::14:head 2024-03-04T15:56:27.863 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:62a1935d:::14:head -> 14 2024-03-04T15:56:27.863 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : entry=14 expected=14 2024-03-04T15:56:27.863 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : seek to 39:566253c9:::13:head 2024-03-04T15:56:27.863 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : cursor()=39:566253c9:::13:head expected=39:566253c9:::13:head 2024-03-04T15:56:27.863 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: > 39:566253c9:::13:head -> 13 2024-03-04T15:56:27.863 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: : entry=13 expected=13 2024-03-04T15:56:27.863 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (381 ms) 2024-03-04T15:56:27.863 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-03-04T15:56:27.863 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (112575 ms) 2024-03-04T15:56:27.864 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-03-04T15:56:27.864 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: split 0/5 -> MIN 39:33333333::::head 2024-03-04T15:56:27.864 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: split 1/5 -> 39:33333333::::head 39:66666666::::head 2024-03-04T15:56:27.864 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: split 2/5 -> 39:66666666::::head 39:99999999::::head 2024-03-04T15:56:27.864 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: split 3/5 -> 39:99999999::::head 39:cccccccc::::head 2024-03-04T15:56:27.864 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: split 4/5 -> 39:cccccccc::::head MAX 2024-03-04T15:56:27.864 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (11382 ms) 2024-03-04T15:56:27.864 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [----------] 7 tests from LibRadosList (124644 ms total) 2024-03-04T15:56:27.864 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 2024-03-04T15:56:27.864 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-03-04T15:56:27.864 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-03-04T15:56:27.864 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (986 ms) 2024-03-04T15:56:27.864 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-03-04T15:56:27.864 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: myset foo1,foo2,foo3 2024-03-04T15:56:27.865 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: foo1 2024-03-04T15:56:27.865 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: foo2 2024-03-04T15:56:27.865 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: foo3 2024-03-04T15:56:27.865 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: myset foo1,foo4,foo5 2024-03-04T15:56:27.865 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: foo4 2024-03-04T15:56:27.865 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: foo5 2024-03-04T15:56:27.865 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: foo1 2024-03-04T15:56:27.865 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: myset foo6,foo7 2024-03-04T15:56:27.865 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: foo7 2024-03-04T15:56:27.865 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: foo6 2024-03-04T15:56:27.865 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-03-04T15:56:27.865 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: ns1:foo4 2024-03-04T15:56:27.865 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: ns1:foo5 2024-03-04T15:56:27.865 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: ns2:foo7 2024-03-04T15:56:27.865 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: ns2:foo6 2024-03-04T15:56:27.866 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: ns1:foo1 2024-03-04T15:56:27.866 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: :foo1 2024-03-04T15:56:27.866 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: :foo2 2024-03-04T15:56:27.866 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: :foo3 2024-03-04T15:56:27.866 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (52 ms) 2024-03-04T15:56:28.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:28 smithi151 bash[23756]: audit 2024-03-04T15:56:26.579608+0000 mgr.y (mgr.24428) 280 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:56:28.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:28 smithi151 bash[23756]: cluster 2024-03-04T15:56:26.853236+0000 mgr.y (mgr.24428) 281 : cluster 0 pgmap v303: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:56:28.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:28 smithi151 bash[23756]: audit 2024-03-04T15:56:26.937040+0000 mon.c (mon.2) 307 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:28.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:28 smithi151 bash[23756]: audit 2024-03-04T15:56:27.311431+0000 mon.a (mon.0) 1705 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:28.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:28 smithi151 bash[23756]: audit 2024-03-04T15:56:27.744161+0000 mon.a (mon.0) 1706 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi117-63915-29"}]': finished 2024-03-04T15:56:28.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:28 smithi151 bash[23756]: audit 2024-03-04T15:56:27.744363+0000 mon.a (mon.0) 1707 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:56:28.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:28 smithi151 bash[23756]: cluster 2024-03-04T15:56:27.755135+0000 mon.a (mon.0) 1708 : cluster 0 osdmap e199: 8 total, 8 up, 8 in 2024-03-04T15:56:28.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:28 smithi151 bash[23756]: audit 2024-03-04T15:56:27.756317+0000 mon.c (mon.2) 308 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-17"}]: dispatch 2024-03-04T15:56:28.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:28 smithi151 bash[23756]: audit 2024-03-04T15:56:27.758283+0000 mon.c (mon.2) 309 : audit 1 from='client.? 172.21.15.117:0/3150207567' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi117-63923-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:28.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:28 smithi151 bash[23756]: audit 2024-03-04T15:56:27.760792+0000 mon.a (mon.0) 1709 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-17"}]: dispatch 2024-03-04T15:56:28.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:28 smithi151 bash[23756]: audit 2024-03-04T15:56:27.761648+0000 mon.a (mon.0) 1710 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi117-63923-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:28.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:28 smithi151 bash[23756]: audit 2024-03-04T15:56:27.790733+0000 mon.a (mon.0) 1711 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi117-63915-30"}]: dispatch 2024-03-04T15:56:28.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:28 smithi151 bash[23756]: audit 2024-03-04T15:56:27.791940+0000 mon.a (mon.0) 1712 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi117-63915-30"}]: dispatch 2024-03-04T15:56:28.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:28 smithi151 bash[23756]: audit 2024-03-04T15:56:27.792425+0000 mon.a (mon.0) 1713 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi117-63915-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:28.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[28843]: audit 2024-03-04T15:56:26.579608+0000 mgr.y (mgr.24428) 280 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:56:28.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[28843]: cluster 2024-03-04T15:56:26.853236+0000 mgr.y (mgr.24428) 281 : cluster 0 pgmap v303: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:56:28.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[28843]: audit 2024-03-04T15:56:26.937040+0000 mon.c (mon.2) 307 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:28.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[28843]: audit 2024-03-04T15:56:27.311431+0000 mon.a (mon.0) 1705 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:28.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[28843]: audit 2024-03-04T15:56:27.744161+0000 mon.a (mon.0) 1706 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi117-63915-29"}]': finished 2024-03-04T15:56:28.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[28843]: audit 2024-03-04T15:56:27.744363+0000 mon.a (mon.0) 1707 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:56:28.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[28843]: cluster 2024-03-04T15:56:27.755135+0000 mon.a (mon.0) 1708 : cluster 0 osdmap e199: 8 total, 8 up, 8 in 2024-03-04T15:56:28.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[28843]: audit 2024-03-04T15:56:27.756317+0000 mon.c (mon.2) 308 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-17"}]: dispatch 2024-03-04T15:56:28.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[28843]: audit 2024-03-04T15:56:27.758283+0000 mon.c (mon.2) 309 : audit 1 from='client.? 172.21.15.117:0/3150207567' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi117-63923-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:28.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[28843]: audit 2024-03-04T15:56:27.760792+0000 mon.a (mon.0) 1709 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-17"}]: dispatch 2024-03-04T15:56:28.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[28843]: audit 2024-03-04T15:56:27.761648+0000 mon.a (mon.0) 1710 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi117-63923-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:28.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[28843]: audit 2024-03-04T15:56:27.790733+0000 mon.a (mon.0) 1711 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi117-63915-30"}]: dispatch 2024-03-04T15:56:28.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[28843]: audit 2024-03-04T15:56:27.791940+0000 mon.a (mon.0) 1712 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi117-63915-30"}]: dispatch 2024-03-04T15:56:28.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[28843]: audit 2024-03-04T15:56:27.792425+0000 mon.a (mon.0) 1713 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi117-63915-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:28.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[21480]: audit 2024-03-04T15:56:26.579608+0000 mgr.y (mgr.24428) 280 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:56:28.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[21480]: cluster 2024-03-04T15:56:26.853236+0000 mgr.y (mgr.24428) 281 : cluster 0 pgmap v303: 236 pgs: 8 unknown, 228 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:56:28.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[21480]: audit 2024-03-04T15:56:26.937040+0000 mon.c (mon.2) 307 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:28.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[21480]: audit 2024-03-04T15:56:27.311431+0000 mon.a (mon.0) 1705 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:28.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[21480]: audit 2024-03-04T15:56:27.744161+0000 mon.a (mon.0) 1706 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi117-63915-29"}]': finished 2024-03-04T15:56:28.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[21480]: audit 2024-03-04T15:56:27.744363+0000 mon.a (mon.0) 1707 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:56:28.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[21480]: cluster 2024-03-04T15:56:27.755135+0000 mon.a (mon.0) 1708 : cluster 0 osdmap e199: 8 total, 8 up, 8 in 2024-03-04T15:56:28.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[21480]: audit 2024-03-04T15:56:27.756317+0000 mon.c (mon.2) 308 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-17"}]: dispatch 2024-03-04T15:56:28.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[21480]: audit 2024-03-04T15:56:27.758283+0000 mon.c (mon.2) 309 : audit 1 from='client.? 172.21.15.117:0/3150207567' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi117-63923-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:28.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[21480]: audit 2024-03-04T15:56:27.760792+0000 mon.a (mon.0) 1709 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-17"}]: dispatch 2024-03-04T15:56:28.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[21480]: audit 2024-03-04T15:56:27.761648+0000 mon.a (mon.0) 1710 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi117-63923-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:28.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[21480]: audit 2024-03-04T15:56:27.790733+0000 mon.a (mon.0) 1711 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi117-63915-30"}]: dispatch 2024-03-04T15:56:28.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[21480]: audit 2024-03-04T15:56:27.791940+0000 mon.a (mon.0) 1712 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi117-63915-30"}]: dispatch 2024-03-04T15:56:28.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:28 smithi117 bash[21480]: audit 2024-03-04T15:56:27.792425+0000 mon.a (mon.0) 1713 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi117-63915-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:29.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:29 smithi151 bash[23756]: audit 2024-03-04T15:56:28.317370+0000 mon.a (mon.0) 1714 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:56:29.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:29 smithi151 bash[23756]: audit 2024-03-04T15:56:28.321269+0000 mon.c (mon.2) 310 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:56:29.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:29 smithi151 bash[23756]: cluster 2024-03-04T15:56:28.744745+0000 mon.a (mon.0) 1715 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:56:29.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:29 smithi151 bash[23756]: audit 2024-03-04T15:56:28.747635+0000 mon.a (mon.0) 1716 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-17"}]': finished 2024-03-04T15:56:29.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:29 smithi151 bash[23756]: audit 2024-03-04T15:56:28.747908+0000 mon.a (mon.0) 1717 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi117-63923-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:29.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:29 smithi151 bash[23756]: audit 2024-03-04T15:56:28.748151+0000 mon.a (mon.0) 1718 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi117-63915-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:29.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:29 smithi151 bash[23756]: cluster 2024-03-04T15:56:28.761162+0000 mon.a (mon.0) 1719 : cluster 0 osdmap e200: 8 total, 8 up, 8 in 2024-03-04T15:56:29.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:29 smithi151 bash[23756]: audit 2024-03-04T15:56:28.767197+0000 mon.c (mon.2) 311 : audit 1 from='client.? 172.21.15.117:0/783728534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:29.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:29 smithi151 bash[23756]: audit 2024-03-04T15:56:28.768107+0000 mon.a (mon.0) 1720 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi117-63915-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi117-63915-30"}]: dispatch 2024-03-04T15:56:29.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:29 smithi151 bash[23756]: audit 2024-03-04T15:56:28.768553+0000 mon.a (mon.0) 1721 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:29.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:29 smithi151 bash[23756]: cluster 2024-03-04T15:56:28.854023+0000 mgr.y (mgr.24428) 282 : cluster 0 pgmap v306: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:56:29.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[28843]: audit 2024-03-04T15:56:28.317370+0000 mon.a (mon.0) 1714 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:56:29.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[28843]: audit 2024-03-04T15:56:28.321269+0000 mon.c (mon.2) 310 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:56:29.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[28843]: cluster 2024-03-04T15:56:28.744745+0000 mon.a (mon.0) 1715 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:56:29.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[28843]: audit 2024-03-04T15:56:28.747635+0000 mon.a (mon.0) 1716 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-17"}]': finished 2024-03-04T15:56:29.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[28843]: audit 2024-03-04T15:56:28.747908+0000 mon.a (mon.0) 1717 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi117-63923-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:29.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[28843]: audit 2024-03-04T15:56:28.748151+0000 mon.a (mon.0) 1718 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi117-63915-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:29.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[28843]: cluster 2024-03-04T15:56:28.761162+0000 mon.a (mon.0) 1719 : cluster 0 osdmap e200: 8 total, 8 up, 8 in 2024-03-04T15:56:29.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[28843]: audit 2024-03-04T15:56:28.767197+0000 mon.c (mon.2) 311 : audit 1 from='client.? 172.21.15.117:0/783728534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:29.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[28843]: audit 2024-03-04T15:56:28.768107+0000 mon.a (mon.0) 1720 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi117-63915-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi117-63915-30"}]: dispatch 2024-03-04T15:56:29.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[28843]: audit 2024-03-04T15:56:28.768553+0000 mon.a (mon.0) 1721 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:29.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[28843]: cluster 2024-03-04T15:56:28.854023+0000 mgr.y (mgr.24428) 282 : cluster 0 pgmap v306: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:56:29.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[21480]: audit 2024-03-04T15:56:28.317370+0000 mon.a (mon.0) 1714 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:56:29.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[21480]: audit 2024-03-04T15:56:28.321269+0000 mon.c (mon.2) 310 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:56:29.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[21480]: cluster 2024-03-04T15:56:28.744745+0000 mon.a (mon.0) 1715 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:56:29.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[21480]: audit 2024-03-04T15:56:28.747635+0000 mon.a (mon.0) 1716 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-17"}]': finished 2024-03-04T15:56:29.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[21480]: audit 2024-03-04T15:56:28.747908+0000 mon.a (mon.0) 1717 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi117-63923-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:29.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[21480]: audit 2024-03-04T15:56:28.748151+0000 mon.a (mon.0) 1718 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi117-63915-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:29.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[21480]: cluster 2024-03-04T15:56:28.761162+0000 mon.a (mon.0) 1719 : cluster 0 osdmap e200: 8 total, 8 up, 8 in 2024-03-04T15:56:29.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[21480]: audit 2024-03-04T15:56:28.767197+0000 mon.c (mon.2) 311 : audit 1 from='client.? 172.21.15.117:0/783728534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:29.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[21480]: audit 2024-03-04T15:56:28.768107+0000 mon.a (mon.0) 1720 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi117-63915-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi117-63915-30"}]: dispatch 2024-03-04T15:56:29.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[21480]: audit 2024-03-04T15:56:28.768553+0000 mon.a (mon.0) 1721 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:29.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:29 smithi117 bash[21480]: cluster 2024-03-04T15:56:28.854023+0000 mgr.y (mgr.24428) 282 : cluster 0 pgmap v306: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:56:31.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:31 smithi151 bash[23756]: audit 2024-03-04T15:56:29.757490+0000 mon.a (mon.0) 1722 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi117-63955-2"}]': finished 2024-03-04T15:56:31.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:31 smithi151 bash[23756]: audit 2024-03-04T15:56:29.767014+0000 mon.c (mon.2) 312 : audit 1 from='client.? 172.21.15.117:0/783728534' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:31.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:31 smithi151 bash[23756]: cluster 2024-03-04T15:56:30.220437+0000 mon.a (mon.0) 1723 : cluster 0 osdmap e201: 8 total, 8 up, 8 in 2024-03-04T15:56:31.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:31 smithi151 bash[23756]: audit 2024-03-04T15:56:30.222258+0000 mon.a (mon.0) 1724 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:31.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:31 smithi117 bash[28843]: audit 2024-03-04T15:56:29.757490+0000 mon.a (mon.0) 1722 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi117-63955-2"}]': finished 2024-03-04T15:56:31.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:31 smithi117 bash[28843]: audit 2024-03-04T15:56:29.767014+0000 mon.c (mon.2) 312 : audit 1 from='client.? 172.21.15.117:0/783728534' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:31.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:31 smithi117 bash[28843]: cluster 2024-03-04T15:56:30.220437+0000 mon.a (mon.0) 1723 : cluster 0 osdmap e201: 8 total, 8 up, 8 in 2024-03-04T15:56:31.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:31 smithi117 bash[28843]: audit 2024-03-04T15:56:30.222258+0000 mon.a (mon.0) 1724 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:31.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:31 smithi117 bash[21480]: audit 2024-03-04T15:56:29.757490+0000 mon.a (mon.0) 1722 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi117-63955-2"}]': finished 2024-03-04T15:56:31.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:31 smithi117 bash[21480]: audit 2024-03-04T15:56:29.767014+0000 mon.c (mon.2) 312 : audit 1 from='client.? 172.21.15.117:0/783728534' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:31.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:31 smithi117 bash[21480]: cluster 2024-03-04T15:56:30.220437+0000 mon.a (mon.0) 1723 : cluster 0 osdmap e201: 8 total, 8 up, 8 in 2024-03-04T15:56:31.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:31 smithi117 bash[21480]: audit 2024-03-04T15:56:30.222258+0000 mon.a (mon.0) 1724 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi117-63955-2"}]: dispatch 2024-03-04T15:56:32.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:32 smithi151 bash[23756]: cluster 2024-03-04T15:56:30.854752+0000 mgr.y (mgr.24428) 283 : cluster 0 pgmap v308: 196 pgs: 196 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:56:32.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:32 smithi151 bash[23756]: audit 2024-03-04T15:56:31.111168+0000 mon.a (mon.0) 1725 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi117-63915-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi117-63915-30"}]': finished 2024-03-04T15:56:32.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:32 smithi151 bash[23756]: audit 2024-03-04T15:56:31.111397+0000 mon.a (mon.0) 1726 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi117-63955-2"}]': finished 2024-03-04T15:56:32.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:32 smithi151 bash[23756]: cluster 2024-03-04T15:56:31.118025+0000 mon.a (mon.0) 1727 : cluster 0 osdmap e202: 8 total, 8 up, 8 in 2024-03-04T15:56:32.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:32 smithi151 bash[23756]: audit 2024-03-04T15:56:31.123541+0000 mon.c (mon.2) 313 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:32.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:32 smithi151 bash[23756]: audit 2024-03-04T15:56:31.134117+0000 mon.c (mon.2) 314 : audit 1 from='client.? 172.21.15.117:0/3259867706' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi117-63923-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:32.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:32 smithi151 bash[23756]: audit 2024-03-04T15:56:31.135361+0000 mon.a (mon.0) 1728 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:32.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:32 smithi151 bash[23756]: audit 2024-03-04T15:56:31.136135+0000 mon.a (mon.0) 1729 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi117-63923-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:32.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:32 smithi151 bash[23756]: audit 2024-03-04T15:56:32.115857+0000 mon.a (mon.0) 1730 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:32.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:32 smithi117 bash[28843]: cluster 2024-03-04T15:56:30.854752+0000 mgr.y (mgr.24428) 283 : cluster 0 pgmap v308: 196 pgs: 196 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:56:32.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:32 smithi117 bash[28843]: audit 2024-03-04T15:56:31.111168+0000 mon.a (mon.0) 1725 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi117-63915-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi117-63915-30"}]': finished 2024-03-04T15:56:32.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:32 smithi117 bash[28843]: audit 2024-03-04T15:56:31.111397+0000 mon.a (mon.0) 1726 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi117-63955-2"}]': finished 2024-03-04T15:56:32.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:32 smithi117 bash[28843]: cluster 2024-03-04T15:56:31.118025+0000 mon.a (mon.0) 1727 : cluster 0 osdmap e202: 8 total, 8 up, 8 in 2024-03-04T15:56:32.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:32 smithi117 bash[28843]: audit 2024-03-04T15:56:31.123541+0000 mon.c (mon.2) 313 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:32.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:32 smithi117 bash[28843]: audit 2024-03-04T15:56:31.134117+0000 mon.c (mon.2) 314 : audit 1 from='client.? 172.21.15.117:0/3259867706' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi117-63923-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:32.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:32 smithi117 bash[28843]: audit 2024-03-04T15:56:31.135361+0000 mon.a (mon.0) 1728 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:32.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:32 smithi117 bash[28843]: audit 2024-03-04T15:56:31.136135+0000 mon.a (mon.0) 1729 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi117-63923-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:32.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:32 smithi117 bash[28843]: audit 2024-03-04T15:56:32.115857+0000 mon.a (mon.0) 1730 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:32.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:32 smithi117 bash[21480]: cluster 2024-03-04T15:56:30.854752+0000 mgr.y (mgr.24428) 283 : cluster 0 pgmap v308: 196 pgs: 196 active+clean; 583 KiB data, 564 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:56:32.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:32 smithi117 bash[21480]: audit 2024-03-04T15:56:31.111168+0000 mon.a (mon.0) 1725 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi117-63915-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi117-63915-30"}]': finished 2024-03-04T15:56:32.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:32 smithi117 bash[21480]: audit 2024-03-04T15:56:31.111397+0000 mon.a (mon.0) 1726 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi117-63955-2"}]': finished 2024-03-04T15:56:32.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:32 smithi117 bash[21480]: cluster 2024-03-04T15:56:31.118025+0000 mon.a (mon.0) 1727 : cluster 0 osdmap e202: 8 total, 8 up, 8 in 2024-03-04T15:56:32.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:32 smithi117 bash[21480]: audit 2024-03-04T15:56:31.123541+0000 mon.c (mon.2) 313 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:32.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:32 smithi117 bash[21480]: audit 2024-03-04T15:56:31.134117+0000 mon.c (mon.2) 314 : audit 1 from='client.? 172.21.15.117:0/3259867706' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi117-63923-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:32.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:32 smithi117 bash[21480]: audit 2024-03-04T15:56:31.135361+0000 mon.a (mon.0) 1728 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:32.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:32 smithi117 bash[21480]: audit 2024-03-04T15:56:31.136135+0000 mon.a (mon.0) 1729 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi117-63923-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:32.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:32 smithi117 bash[21480]: audit 2024-03-04T15:56:32.115857+0000 mon.a (mon.0) 1730 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:33.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:32.116084+0000 mon.a (mon.0) 1731 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi117-63923-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:33.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: cluster 2024-03-04T15:56:32.125385+0000 mon.a (mon.0) 1732 : cluster 0 osdmap e203: 8 total, 8 up, 8 in 2024-03-04T15:56:33.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:32.135555+0000 mon.c (mon.2) 315 : audit 1 from='client.? 172.21.15.117:0/1969575866' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63955-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:33.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:32.151701+0000 mon.c (mon.2) 316 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:33.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:32.153777+0000 mon.a (mon.0) 1733 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63955-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:33.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:32.154535+0000 mon.a (mon.0) 1734 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:33.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:32.633105+0000 mon.c (mon.2) 317 : audit 1 from='client.? 172.21.15.117:0/3259867706' entity='client.admin' cmd=[{ 2024-03-04T15:56:33.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:32.633193+0000 mon.c (mon.2) 318 : audit 1 "prefix": "osd pool set", 2024-03-04T15:56:33.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:32.633232+0000 mon.c (mon.2) 319 : audit 1 "pool": "PoolEIOFlag_smithi117-63923-33", 2024-03-04T15:56:33.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:32.633296+0000 mon.c (mon.2) 320 : audit 1 "var": "eio", 2024-03-04T15:56:33.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:32.633341+0000 mon.c (mon.2) 321 : audit 1 "val": "true" 2024-03-04T15:56:33.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:32.633385+0000 mon.c (mon.2) 322 : audit 1 }]: dispatch 2024-03-04T15:56:33.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:32.634268+0000 mon.a (mon.0) 1735 : audit 1 from='client.? ' entity='client.admin' cmd=[{ 2024-03-04T15:56:33.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:32.634315+0000 mon.a (mon.0) 1736 : audit 1 "prefix": "osd pool set", 2024-03-04T15:56:33.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:32.634351+0000 mon.a (mon.0) 1737 : audit 1 "pool": "PoolEIOFlag_smithi117-63923-33", 2024-03-04T15:56:33.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:32.634398+0000 mon.a (mon.0) 1738 : audit 1 "var": "eio", 2024-03-04T15:56:33.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:32.634442+0000 mon.a (mon.0) 1739 : audit 1 "val": "true" 2024-03-04T15:56:33.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:32.634485+0000 mon.a (mon.0) 1740 : audit 1 }]: dispatch 2024-03-04T15:56:33.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: cluster 2024-03-04T15:56:32.740256+0000 mon.a (mon.0) 1741 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:33.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:33.119263+0000 mon.a (mon.0) 1742 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63955-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:33.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:33.119491+0000 mon.a (mon.0) 1743 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-19", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:56:33.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:33.119631+0000 mon.a (mon.0) 1744 : audit 1 from='client.? ' entity='client.admin' cmd='[{ 2024-03-04T15:56:33.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:33.119660+0000 mon.a (mon.0) 1745 : audit 1 "prefix": "osd pool set", 2024-03-04T15:56:33.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:33.119680+0000 mon.a (mon.0) 1746 : audit 1 "pool": "PoolEIOFlag_smithi117-63923-33", 2024-03-04T15:56:33.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:33.119724+0000 mon.a (mon.0) 1747 : audit 1 "var": "eio", 2024-03-04T15:56:33.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:33.119746+0000 mon.a (mon.0) 1748 : audit 1 "val": "true" 2024-03-04T15:56:33.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:33.119767+0000 mon.a (mon.0) 1749 : audit 1 }]': finished 2024-03-04T15:56:33.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: cluster 2024-03-04T15:56:33.127983+0000 mon.a (mon.0) 1750 : cluster 0 osdmap e204: 8 total, 8 up, 8 in 2024-03-04T15:56:33.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:33 smithi151 bash[23756]: audit 2024-03-04T15:56:33.134266+0000 mon.a (mon.0) 1751 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi117-63915-30"}]: dispatch 2024-03-04T15:56:33.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:32.116084+0000 mon.a (mon.0) 1731 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi117-63923-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:33.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: cluster 2024-03-04T15:56:32.125385+0000 mon.a (mon.0) 1732 : cluster 0 osdmap e203: 8 total, 8 up, 8 in 2024-03-04T15:56:33.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:32.135555+0000 mon.c (mon.2) 315 : audit 1 from='client.? 172.21.15.117:0/1969575866' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63955-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:33.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:32.151701+0000 mon.c (mon.2) 316 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:33.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:32.153777+0000 mon.a (mon.0) 1733 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63955-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:33.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:32.154535+0000 mon.a (mon.0) 1734 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:33.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:32.633105+0000 mon.c (mon.2) 317 : audit 1 from='client.? 172.21.15.117:0/3259867706' entity='client.admin' cmd=[{ 2024-03-04T15:56:33.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:32.633193+0000 mon.c (mon.2) 318 : audit 1 "prefix": "osd pool set", 2024-03-04T15:56:33.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:32.633232+0000 mon.c (mon.2) 319 : audit 1 "pool": "PoolEIOFlag_smithi117-63923-33", 2024-03-04T15:56:33.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:32.633296+0000 mon.c (mon.2) 320 : audit 1 "var": "eio", 2024-03-04T15:56:33.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:32.633341+0000 mon.c (mon.2) 321 : audit 1 "val": "true" 2024-03-04T15:56:33.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:32.633385+0000 mon.c (mon.2) 322 : audit 1 }]: dispatch 2024-03-04T15:56:33.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:32.634268+0000 mon.a (mon.0) 1735 : audit 1 from='client.? ' entity='client.admin' cmd=[{ 2024-03-04T15:56:33.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:32.634315+0000 mon.a (mon.0) 1736 : audit 1 "prefix": "osd pool set", 2024-03-04T15:56:33.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:32.634351+0000 mon.a (mon.0) 1737 : audit 1 "pool": "PoolEIOFlag_smithi117-63923-33", 2024-03-04T15:56:33.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:32.634398+0000 mon.a (mon.0) 1738 : audit 1 "var": "eio", 2024-03-04T15:56:33.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:32.634442+0000 mon.a (mon.0) 1739 : audit 1 "val": "true" 2024-03-04T15:56:33.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:32.634485+0000 mon.a (mon.0) 1740 : audit 1 }]: dispatch 2024-03-04T15:56:33.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: cluster 2024-03-04T15:56:32.740256+0000 mon.a (mon.0) 1741 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:33.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:33.119263+0000 mon.a (mon.0) 1742 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63955-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:33.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:32.116084+0000 mon.a (mon.0) 1731 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi117-63923-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:33.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: cluster 2024-03-04T15:56:32.125385+0000 mon.a (mon.0) 1732 : cluster 0 osdmap e203: 8 total, 8 up, 8 in 2024-03-04T15:56:33.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:32.135555+0000 mon.c (mon.2) 315 : audit 1 from='client.? 172.21.15.117:0/1969575866' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63955-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:33.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:32.151701+0000 mon.c (mon.2) 316 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:33.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:32.153777+0000 mon.a (mon.0) 1733 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63955-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:33.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:32.154535+0000 mon.a (mon.0) 1734 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:33.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:32.633105+0000 mon.c (mon.2) 317 : audit 1 from='client.? 172.21.15.117:0/3259867706' entity='client.admin' cmd=[{ 2024-03-04T15:56:33.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:32.633193+0000 mon.c (mon.2) 318 : audit 1 "prefix": "osd pool set", 2024-03-04T15:56:33.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:32.633232+0000 mon.c (mon.2) 319 : audit 1 "pool": "PoolEIOFlag_smithi117-63923-33", 2024-03-04T15:56:33.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:32.633296+0000 mon.c (mon.2) 320 : audit 1 "var": "eio", 2024-03-04T15:56:33.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:32.633341+0000 mon.c (mon.2) 321 : audit 1 "val": "true" 2024-03-04T15:56:33.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:32.633385+0000 mon.c (mon.2) 322 : audit 1 }]: dispatch 2024-03-04T15:56:33.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:32.634268+0000 mon.a (mon.0) 1735 : audit 1 from='client.? ' entity='client.admin' cmd=[{ 2024-03-04T15:56:33.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:32.634315+0000 mon.a (mon.0) 1736 : audit 1 "prefix": "osd pool set", 2024-03-04T15:56:33.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:32.634351+0000 mon.a (mon.0) 1737 : audit 1 "pool": "PoolEIOFlag_smithi117-63923-33", 2024-03-04T15:56:33.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:32.634398+0000 mon.a (mon.0) 1738 : audit 1 "var": "eio", 2024-03-04T15:56:33.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:32.634442+0000 mon.a (mon.0) 1739 : audit 1 "val": "true" 2024-03-04T15:56:33.575 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:32.634485+0000 mon.a (mon.0) 1740 : audit 1 }]: dispatch 2024-03-04T15:56:33.575 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: cluster 2024-03-04T15:56:32.740256+0000 mon.a (mon.0) 1741 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:33.575 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:33.119263+0000 mon.a (mon.0) 1742 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-63955-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:33.577 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:33.119491+0000 mon.a (mon.0) 1743 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-19", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:56:33.577 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:33.119631+0000 mon.a (mon.0) 1744 : audit 1 from='client.? ' entity='client.admin' cmd='[{ 2024-03-04T15:56:33.577 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:33.119660+0000 mon.a (mon.0) 1745 : audit 1 "prefix": "osd pool set", 2024-03-04T15:56:33.577 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:33.119680+0000 mon.a (mon.0) 1746 : audit 1 "pool": "PoolEIOFlag_smithi117-63923-33", 2024-03-04T15:56:33.577 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:33.119724+0000 mon.a (mon.0) 1747 : audit 1 "var": "eio", 2024-03-04T15:56:33.577 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:33.119746+0000 mon.a (mon.0) 1748 : audit 1 "val": "true" 2024-03-04T15:56:33.577 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:33.119767+0000 mon.a (mon.0) 1749 : audit 1 }]': finished 2024-03-04T15:56:33.577 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: cluster 2024-03-04T15:56:33.127983+0000 mon.a (mon.0) 1750 : cluster 0 osdmap e204: 8 total, 8 up, 8 in 2024-03-04T15:56:33.577 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[21480]: audit 2024-03-04T15:56:33.134266+0000 mon.a (mon.0) 1751 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi117-63915-30"}]: dispatch 2024-03-04T15:56:33.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:33.119491+0000 mon.a (mon.0) 1743 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-19", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:56:33.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:33.119631+0000 mon.a (mon.0) 1744 : audit 1 from='client.? ' entity='client.admin' cmd='[{ 2024-03-04T15:56:33.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:33.119660+0000 mon.a (mon.0) 1745 : audit 1 "prefix": "osd pool set", 2024-03-04T15:56:33.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:33.119680+0000 mon.a (mon.0) 1746 : audit 1 "pool": "PoolEIOFlag_smithi117-63923-33", 2024-03-04T15:56:33.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:33.119724+0000 mon.a (mon.0) 1747 : audit 1 "var": "eio", 2024-03-04T15:56:33.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:33.119746+0000 mon.a (mon.0) 1748 : audit 1 "val": "true" 2024-03-04T15:56:33.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:33.119767+0000 mon.a (mon.0) 1749 : audit 1 }]': finished 2024-03-04T15:56:33.578 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: cluster 2024-03-04T15:56:33.127983+0000 mon.a (mon.0) 1750 : cluster 0 osdmap e204: 8 total, 8 up, 8 in 2024-03-04T15:56:33.579 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:33 smithi117 bash[28843]: audit 2024-03-04T15:56:33.134266+0000 mon.a (mon.0) 1751 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi117-63915-30"}]: dispatch 2024-03-04T15:56:34.150 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsStart 2024-03-04T15:56:34.150 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 1 0 2024-03-04T15:56:34.150 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 10 0 2024-03-04T15:56:34.150 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 13 0 2024-03-04T15:56:34.150 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 7 0 2024-03-04T15:56:34.150 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 14 0 2024-03-04T15:56:34.151 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 0 0 2024-03-04T15:56:34.151 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 15 0 2024-03-04T15:56:34.151 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 11 0 2024-03-04T15:56:34.151 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 5 0 2024-03-04T15:56:34.151 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 8 0 2024-03-04T15:56:34.151 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 6 0 2024-03-04T15:56:34.151 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 3 0 2024-03-04T15:56:34.151 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 4 0 2024-03-04T15:56:34.151 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 12 0 2024-03-04T15:56:34.151 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 9 0 2024-03-04T15:56:34.151 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 2 0 2024-03-04T15:56:34.151 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-03-04T15:56:34.151 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (80 ms) 2024-03-04T15:56:34.151 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [----------] 3 tests from LibRadosListEC (1118 ms total) 2024-03-04T15:56:34.151 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 2024-03-04T15:56:34.151 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-03-04T15:56:34.152 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-03-04T15:56:34.152 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3027 ms) 2024-03-04T15:56:34.152 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [----------] 1 test from LibRadosListNP (3027 ms total) 2024-03-04T15:56:34.152 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: 2024-03-04T15:56:34.152 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [----------] Global test environment tear-down 2024-03-04T15:56:34.152 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [==========] 11 tests from 3 test suites ran. (154226 ms total) 2024-03-04T15:56:34.152 INFO:tasks.workunit.client.0.smithi117.stdout: api_list: [ PASSED ] 11 tests. 2024-03-04T15:56:34.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:34 smithi151 bash[23756]: cluster 2024-03-04T15:56:32.855712+0000 mgr.y (mgr.24428) 284 : cluster 0 pgmap v311: 300 pgs: 32 unknown, 72 creating+peering, 196 active+clean; 583 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 249 B/s wr, 1 op/s 2024-03-04T15:56:34.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:34 smithi151 bash[23756]: audit 2024-03-04T15:56:33.138731+0000 mon.c (mon.2) 323 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-19"}]: dispatch 2024-03-04T15:56:34.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:34 smithi151 bash[23756]: audit 2024-03-04T15:56:33.145387+0000 mon.a (mon.0) 1752 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-19"}]: dispatch 2024-03-04T15:56:34.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:34 smithi151 bash[23756]: audit 2024-03-04T15:56:33.185578+0000 mon.c (mon.2) 324 : audit 1 from='client.? 172.21.15.117:0/1969575866' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi117-63955-3","pool2":"test-rados-api-smithi117-63955-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-04T15:56:34.436 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:34 smithi151 bash[23756]: audit 2024-03-04T15:56:33.186063+0000 mon.a (mon.0) 1753 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi117-63955-3","pool2":"test-rados-api-smithi117-63955-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-04T15:56:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:34 smithi151 bash[23756]: audit 2024-03-04T15:56:34.122103+0000 mon.a (mon.0) 1754 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi117-63915-30"}]': finished 2024-03-04T15:56:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:34 smithi151 bash[23756]: audit 2024-03-04T15:56:34.122223+0000 mon.a (mon.0) 1755 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-19"}]': finished 2024-03-04T15:56:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:34 smithi151 bash[23756]: audit 2024-03-04T15:56:34.122448+0000 mon.a (mon.0) 1756 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi117-63955-3","pool2":"test-rados-api-smithi117-63955-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-03-04T15:56:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:34 smithi151 bash[23756]: cluster 2024-03-04T15:56:34.129620+0000 mon.a (mon.0) 1757 : cluster 0 osdmap e205: 8 total, 8 up, 8 in 2024-03-04T15:56:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:34 smithi151 bash[23756]: audit 2024-03-04T15:56:34.132011+0000 mon.a (mon.0) 1758 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi117-63915-30"}]: dispatch 2024-03-04T15:56:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:34 smithi151 bash[23756]: audit 2024-03-04T15:56:34.139369+0000 mon.c (mon.2) 325 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-19", "mode": "writeback"}]: dispatch 2024-03-04T15:56:34.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:34 smithi151 bash[23756]: audit 2024-03-04T15:56:34.140689+0000 mon.a (mon.0) 1759 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-19", "mode": "writeback"}]: dispatch 2024-03-04T15:56:34.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[21480]: cluster 2024-03-04T15:56:32.855712+0000 mgr.y (mgr.24428) 284 : cluster 0 pgmap v311: 300 pgs: 32 unknown, 72 creating+peering, 196 active+clean; 583 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 249 B/s wr, 1 op/s 2024-03-04T15:56:34.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[21480]: audit 2024-03-04T15:56:33.138731+0000 mon.c (mon.2) 323 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-19"}]: dispatch 2024-03-04T15:56:34.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[21480]: audit 2024-03-04T15:56:33.145387+0000 mon.a (mon.0) 1752 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-19"}]: dispatch 2024-03-04T15:56:34.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[21480]: audit 2024-03-04T15:56:33.185578+0000 mon.c (mon.2) 324 : audit 1 from='client.? 172.21.15.117:0/1969575866' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi117-63955-3","pool2":"test-rados-api-smithi117-63955-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-04T15:56:34.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[21480]: audit 2024-03-04T15:56:33.186063+0000 mon.a (mon.0) 1753 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi117-63955-3","pool2":"test-rados-api-smithi117-63955-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-04T15:56:34.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[21480]: audit 2024-03-04T15:56:34.122103+0000 mon.a (mon.0) 1754 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi117-63915-30"}]': finished 2024-03-04T15:56:34.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[21480]: audit 2024-03-04T15:56:34.122223+0000 mon.a (mon.0) 1755 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-19"}]': finished 2024-03-04T15:56:34.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[21480]: audit 2024-03-04T15:56:34.122448+0000 mon.a (mon.0) 1756 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi117-63955-3","pool2":"test-rados-api-smithi117-63955-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-03-04T15:56:34.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[21480]: cluster 2024-03-04T15:56:34.129620+0000 mon.a (mon.0) 1757 : cluster 0 osdmap e205: 8 total, 8 up, 8 in 2024-03-04T15:56:34.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[21480]: audit 2024-03-04T15:56:34.132011+0000 mon.a (mon.0) 1758 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi117-63915-30"}]: dispatch 2024-03-04T15:56:34.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[21480]: audit 2024-03-04T15:56:34.139369+0000 mon.c (mon.2) 325 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-19", "mode": "writeback"}]: dispatch 2024-03-04T15:56:34.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[21480]: audit 2024-03-04T15:56:34.140689+0000 mon.a (mon.0) 1759 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-19", "mode": "writeback"}]: dispatch 2024-03-04T15:56:34.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[28843]: cluster 2024-03-04T15:56:32.855712+0000 mgr.y (mgr.24428) 284 : cluster 0 pgmap v311: 300 pgs: 32 unknown, 72 creating+peering, 196 active+clean; 583 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 249 B/s wr, 1 op/s 2024-03-04T15:56:34.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[28843]: audit 2024-03-04T15:56:33.138731+0000 mon.c (mon.2) 323 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-19"}]: dispatch 2024-03-04T15:56:34.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[28843]: audit 2024-03-04T15:56:33.145387+0000 mon.a (mon.0) 1752 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-19"}]: dispatch 2024-03-04T15:56:34.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[28843]: audit 2024-03-04T15:56:33.185578+0000 mon.c (mon.2) 324 : audit 1 from='client.? 172.21.15.117:0/1969575866' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi117-63955-3","pool2":"test-rados-api-smithi117-63955-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-04T15:56:34.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[28843]: audit 2024-03-04T15:56:33.186063+0000 mon.a (mon.0) 1753 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi117-63955-3","pool2":"test-rados-api-smithi117-63955-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-03-04T15:56:34.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[28843]: audit 2024-03-04T15:56:34.122103+0000 mon.a (mon.0) 1754 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi117-63915-30"}]': finished 2024-03-04T15:56:34.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[28843]: audit 2024-03-04T15:56:34.122223+0000 mon.a (mon.0) 1755 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-19"}]': finished 2024-03-04T15:56:34.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[28843]: audit 2024-03-04T15:56:34.122448+0000 mon.a (mon.0) 1756 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi117-63955-3","pool2":"test-rados-api-smithi117-63955-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-03-04T15:56:34.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[28843]: cluster 2024-03-04T15:56:34.129620+0000 mon.a (mon.0) 1757 : cluster 0 osdmap e205: 8 total, 8 up, 8 in 2024-03-04T15:56:34.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[28843]: audit 2024-03-04T15:56:34.132011+0000 mon.a (mon.0) 1758 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi117-63915-30"}]: dispatch 2024-03-04T15:56:34.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[28843]: audit 2024-03-04T15:56:34.139369+0000 mon.c (mon.2) 325 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-19", "mode": "writeback"}]: dispatch 2024-03-04T15:56:34.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:34 smithi117 bash[28843]: audit 2024-03-04T15:56:34.140689+0000 mon.a (mon.0) 1759 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-19", "mode": "writeback"}]: dispatch 2024-03-04T15:56:35.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:35 smithi151 bash[23756]: cluster 2024-03-04T15:56:35.124601+0000 mon.a (mon.0) 1760 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:56:35.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:35 smithi151 bash[23756]: audit 2024-03-04T15:56:35.127393+0000 mon.a (mon.0) 1761 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi117-63915-30"}]': finished 2024-03-04T15:56:35.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:35 smithi151 bash[23756]: audit 2024-03-04T15:56:35.127594+0000 mon.a (mon.0) 1762 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-19", "mode": "writeback"}]': finished 2024-03-04T15:56:35.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:35 smithi151 bash[23756]: cluster 2024-03-04T15:56:35.139209+0000 mon.a (mon.0) 1763 : cluster 0 osdmap e206: 8 total, 8 up, 8 in 2024-03-04T15:56:35.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:35 smithi151 bash[23756]: audit 2024-03-04T15:56:35.147277+0000 mon.c (mon.2) 326 : audit 1 from='client.? 172.21.15.117:0/2958265763' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi117-63923-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:35.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:35 smithi151 bash[23756]: audit 2024-03-04T15:56:35.149793+0000 mon.a (mon.0) 1764 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi117-63923-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:35.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:35 smithi117 bash[21480]: cluster 2024-03-04T15:56:35.124601+0000 mon.a (mon.0) 1760 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:56:35.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:35 smithi117 bash[21480]: audit 2024-03-04T15:56:35.127393+0000 mon.a (mon.0) 1761 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi117-63915-30"}]': finished 2024-03-04T15:56:35.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:35 smithi117 bash[21480]: audit 2024-03-04T15:56:35.127594+0000 mon.a (mon.0) 1762 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-19", "mode": "writeback"}]': finished 2024-03-04T15:56:35.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:35 smithi117 bash[21480]: cluster 2024-03-04T15:56:35.139209+0000 mon.a (mon.0) 1763 : cluster 0 osdmap e206: 8 total, 8 up, 8 in 2024-03-04T15:56:35.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:35 smithi117 bash[21480]: audit 2024-03-04T15:56:35.147277+0000 mon.c (mon.2) 326 : audit 1 from='client.? 172.21.15.117:0/2958265763' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi117-63923-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:35.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:35 smithi117 bash[21480]: audit 2024-03-04T15:56:35.149793+0000 mon.a (mon.0) 1764 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi117-63923-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:35.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:35 smithi117 bash[28843]: cluster 2024-03-04T15:56:35.124601+0000 mon.a (mon.0) 1760 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:56:35.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:35 smithi117 bash[28843]: audit 2024-03-04T15:56:35.127393+0000 mon.a (mon.0) 1761 : audit 1 from='client.? 172.21.15.117:0/63169825' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi117-63915-30"}]': finished 2024-03-04T15:56:35.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:35 smithi117 bash[28843]: audit 2024-03-04T15:56:35.127594+0000 mon.a (mon.0) 1762 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-19", "mode": "writeback"}]': finished 2024-03-04T15:56:35.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:35 smithi117 bash[28843]: cluster 2024-03-04T15:56:35.139209+0000 mon.a (mon.0) 1763 : cluster 0 osdmap e206: 8 total, 8 up, 8 in 2024-03-04T15:56:35.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:35 smithi117 bash[28843]: audit 2024-03-04T15:56:35.147277+0000 mon.c (mon.2) 326 : audit 1 from='client.? 172.21.15.117:0/2958265763' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi117-63923-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:35.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:35 smithi117 bash[28843]: audit 2024-03-04T15:56:35.149793+0000 mon.a (mon.0) 1764 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi117-63923-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:36.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:36 smithi151 bash[23756]: cluster 2024-03-04T15:56:34.856536+0000 mgr.y (mgr.24428) 285 : cluster 0 pgmap v314: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:56:36.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:36 smithi151 bash[23756]: audit 2024-03-04T15:56:35.173694+0000 mon.a (mon.0) 1765 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi117-63915-31"}]: dispatch 2024-03-04T15:56:36.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:36 smithi151 bash[23756]: audit 2024-03-04T15:56:35.178767+0000 mon.a (mon.0) 1766 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi117-63915-31"}]: dispatch 2024-03-04T15:56:36.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:36 smithi151 bash[23756]: audit 2024-03-04T15:56:35.179248+0000 mon.a (mon.0) 1767 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi117-63915-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:36.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:36 smithi151 bash[23756]: audit 2024-03-04T15:56:35.191426+0000 mon.c (mon.2) 327 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:36.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:36 smithi151 bash[23756]: audit 2024-03-04T15:56:35.191726+0000 mon.a (mon.0) 1768 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:36.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:36 smithi151 bash[23756]: audit 2024-03-04T15:56:36.130879+0000 mon.a (mon.0) 1769 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi117-63923-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:36.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:36 smithi151 bash[23756]: audit 2024-03-04T15:56:36.131123+0000 mon.a (mon.0) 1770 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi117-63915-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:36.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:36 smithi151 bash[23756]: audit 2024-03-04T15:56:36.131269+0000 mon.a (mon.0) 1771 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:56:36.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:36 smithi151 bash[23756]: cluster 2024-03-04T15:56:36.136871+0000 mon.a (mon.0) 1772 : cluster 0 osdmap e207: 8 total, 8 up, 8 in 2024-03-04T15:56:36.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:36 smithi151 bash[23756]: audit 2024-03-04T15:56:36.138978+0000 mon.a (mon.0) 1773 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi117-63915-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi117-63915-31"}]: dispatch 2024-03-04T15:56:36.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:36 smithi151 bash[23756]: audit 2024-03-04T15:56:36.144732+0000 mon.c (mon.2) 328 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-19"}]: dispatch 2024-03-04T15:56:36.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:36 smithi151 bash[23756]: audit 2024-03-04T15:56:36.145469+0000 mon.a (mon.0) 1774 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-19"}]: dispatch 2024-03-04T15:56:36.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:56:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:56:36.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[21480]: cluster 2024-03-04T15:56:34.856536+0000 mgr.y (mgr.24428) 285 : cluster 0 pgmap v314: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:56:36.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[21480]: audit 2024-03-04T15:56:35.173694+0000 mon.a (mon.0) 1765 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi117-63915-31"}]: dispatch 2024-03-04T15:56:36.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[21480]: audit 2024-03-04T15:56:35.178767+0000 mon.a (mon.0) 1766 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi117-63915-31"}]: dispatch 2024-03-04T15:56:36.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[21480]: audit 2024-03-04T15:56:35.179248+0000 mon.a (mon.0) 1767 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi117-63915-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:36.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[21480]: audit 2024-03-04T15:56:35.191426+0000 mon.c (mon.2) 327 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:36.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[21480]: audit 2024-03-04T15:56:35.191726+0000 mon.a (mon.0) 1768 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:36.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[21480]: audit 2024-03-04T15:56:36.130879+0000 mon.a (mon.0) 1769 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi117-63923-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:36.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[21480]: audit 2024-03-04T15:56:36.131123+0000 mon.a (mon.0) 1770 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi117-63915-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:36.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[21480]: audit 2024-03-04T15:56:36.131269+0000 mon.a (mon.0) 1771 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:56:36.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[21480]: cluster 2024-03-04T15:56:36.136871+0000 mon.a (mon.0) 1772 : cluster 0 osdmap e207: 8 total, 8 up, 8 in 2024-03-04T15:56:36.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[21480]: audit 2024-03-04T15:56:36.138978+0000 mon.a (mon.0) 1773 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi117-63915-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi117-63915-31"}]: dispatch 2024-03-04T15:56:36.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[21480]: audit 2024-03-04T15:56:36.144732+0000 mon.c (mon.2) 328 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-19"}]: dispatch 2024-03-04T15:56:36.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[21480]: audit 2024-03-04T15:56:36.145469+0000 mon.a (mon.0) 1774 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-19"}]: dispatch 2024-03-04T15:56:36.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[28843]: cluster 2024-03-04T15:56:34.856536+0000 mgr.y (mgr.24428) 285 : cluster 0 pgmap v314: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 573 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:56:36.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[28843]: audit 2024-03-04T15:56:35.173694+0000 mon.a (mon.0) 1765 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi117-63915-31"}]: dispatch 2024-03-04T15:56:36.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[28843]: audit 2024-03-04T15:56:35.178767+0000 mon.a (mon.0) 1766 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi117-63915-31"}]: dispatch 2024-03-04T15:56:36.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[28843]: audit 2024-03-04T15:56:35.179248+0000 mon.a (mon.0) 1767 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi117-63915-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:36.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[28843]: audit 2024-03-04T15:56:35.191426+0000 mon.c (mon.2) 327 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:36.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[28843]: audit 2024-03-04T15:56:35.191726+0000 mon.a (mon.0) 1768 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:36.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[28843]: audit 2024-03-04T15:56:36.130879+0000 mon.a (mon.0) 1769 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi117-63923-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:36.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[28843]: audit 2024-03-04T15:56:36.131123+0000 mon.a (mon.0) 1770 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi117-63915-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:36.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[28843]: audit 2024-03-04T15:56:36.131269+0000 mon.a (mon.0) 1771 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:56:36.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[28843]: cluster 2024-03-04T15:56:36.136871+0000 mon.a (mon.0) 1772 : cluster 0 osdmap e207: 8 total, 8 up, 8 in 2024-03-04T15:56:36.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[28843]: audit 2024-03-04T15:56:36.138978+0000 mon.a (mon.0) 1773 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi117-63915-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi117-63915-31"}]: dispatch 2024-03-04T15:56:36.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[28843]: audit 2024-03-04T15:56:36.144732+0000 mon.c (mon.2) 328 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-19"}]: dispatch 2024-03-04T15:56:36.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:36 smithi117 bash[28843]: audit 2024-03-04T15:56:36.145469+0000 mon.a (mon.0) 1774 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-19"}]: dispatch 2024-03-04T15:56:36.932 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:56:36 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:56:37.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:37 smithi151 bash[23756]: cluster 2024-03-04T15:56:37.132039+0000 mon.a (mon.0) 1775 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:56:37.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:37 smithi151 bash[23756]: audit 2024-03-04T15:56:37.134722+0000 mon.a (mon.0) 1776 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-19"}]': finished 2024-03-04T15:56:38.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:37 smithi117 bash[21480]: cluster 2024-03-04T15:56:37.132039+0000 mon.a (mon.0) 1775 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:56:38.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:37 smithi117 bash[21480]: audit 2024-03-04T15:56:37.134722+0000 mon.a (mon.0) 1776 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-19"}]': finished 2024-03-04T15:56:38.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:37 smithi117 bash[28843]: cluster 2024-03-04T15:56:37.132039+0000 mon.a (mon.0) 1775 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:56:38.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:37 smithi117 bash[28843]: audit 2024-03-04T15:56:37.134722+0000 mon.a (mon.0) 1776 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-19"}]': finished 2024-03-04T15:56:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:38 smithi151 bash[23756]: audit 2024-03-04T15:56:36.590936+0000 mgr.y (mgr.24428) 286 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:56:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:38 smithi151 bash[23756]: cluster 2024-03-04T15:56:36.859000+0000 mgr.y (mgr.24428) 287 : cluster 0 pgmap v317: 260 pgs: 260 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 1023 B/s wr, 27 op/s 2024-03-04T15:56:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:38 smithi151 bash[23756]: cluster 2024-03-04T15:56:37.597054+0000 mon.a (mon.0) 1777 : cluster 0 osdmap e208: 8 total, 8 up, 8 in 2024-03-04T15:56:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:38 smithi151 bash[23756]: audit 2024-03-04T15:56:37.798883+0000 mon.c (mon.2) 329 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:56:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:38 smithi151 bash[23756]: audit 2024-03-04T15:56:38.138420+0000 mon.a (mon.0) 1778 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi117-63915-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi117-63915-31"}]': finished 2024-03-04T15:56:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:38 smithi151 bash[23756]: cluster 2024-03-04T15:56:38.165113+0000 mon.a (mon.0) 1779 : cluster 0 osdmap e209: 8 total, 8 up, 8 in 2024-03-04T15:56:38.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:38 smithi151 bash[23756]: audit 2024-03-04T15:56:38.166969+0000 mon.c (mon.2) 330 : audit 1 from='client.? 172.21.15.117:0/2147862292' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi117-63923-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:39.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:38 smithi117 bash[21480]: audit 2024-03-04T15:56:36.590936+0000 mgr.y (mgr.24428) 286 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:56:39.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:38 smithi117 bash[21480]: cluster 2024-03-04T15:56:36.859000+0000 mgr.y (mgr.24428) 287 : cluster 0 pgmap v317: 260 pgs: 260 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 1023 B/s wr, 27 op/s 2024-03-04T15:56:39.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:38 smithi117 bash[21480]: cluster 2024-03-04T15:56:37.597054+0000 mon.a (mon.0) 1777 : cluster 0 osdmap e208: 8 total, 8 up, 8 in 2024-03-04T15:56:39.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:38 smithi117 bash[21480]: audit 2024-03-04T15:56:37.798883+0000 mon.c (mon.2) 329 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:56:39.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:38 smithi117 bash[21480]: audit 2024-03-04T15:56:38.138420+0000 mon.a (mon.0) 1778 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi117-63915-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi117-63915-31"}]': finished 2024-03-04T15:56:39.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:38 smithi117 bash[21480]: cluster 2024-03-04T15:56:38.165113+0000 mon.a (mon.0) 1779 : cluster 0 osdmap e209: 8 total, 8 up, 8 in 2024-03-04T15:56:39.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:38 smithi117 bash[21480]: audit 2024-03-04T15:56:38.166969+0000 mon.c (mon.2) 330 : audit 1 from='client.? 172.21.15.117:0/2147862292' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi117-63923-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:39.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:38 smithi117 bash[28843]: audit 2024-03-04T15:56:36.590936+0000 mgr.y (mgr.24428) 286 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:56:39.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:38 smithi117 bash[28843]: cluster 2024-03-04T15:56:36.859000+0000 mgr.y (mgr.24428) 287 : cluster 0 pgmap v317: 260 pgs: 260 active+clean; 584 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 26 KiB/s rd, 1023 B/s wr, 27 op/s 2024-03-04T15:56:39.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:38 smithi117 bash[28843]: cluster 2024-03-04T15:56:37.597054+0000 mon.a (mon.0) 1777 : cluster 0 osdmap e208: 8 total, 8 up, 8 in 2024-03-04T15:56:39.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:38 smithi117 bash[28843]: audit 2024-03-04T15:56:37.798883+0000 mon.c (mon.2) 329 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:56:39.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:38 smithi117 bash[28843]: audit 2024-03-04T15:56:38.138420+0000 mon.a (mon.0) 1778 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi117-63915-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi117-63915-31"}]': finished 2024-03-04T15:56:39.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:38 smithi117 bash[28843]: cluster 2024-03-04T15:56:38.165113+0000 mon.a (mon.0) 1779 : cluster 0 osdmap e209: 8 total, 8 up, 8 in 2024-03-04T15:56:39.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:38 smithi117 bash[28843]: audit 2024-03-04T15:56:38.166969+0000 mon.c (mon.2) 330 : audit 1 from='client.? 172.21.15.117:0/2147862292' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi117-63923-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:39.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:39 smithi151 bash[23756]: audit 2024-03-04T15:56:38.528466+0000 mon.c (mon.2) 331 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:56:39.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:39 smithi151 bash[23756]: audit 2024-03-04T15:56:38.530132+0000 mon.c (mon.2) 332 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:56:39.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:39 smithi151 bash[23756]: audit 2024-03-04T15:56:38.612358+0000 mon.a (mon.0) 1780 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi117-63923-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:39.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:39 smithi151 bash[23756]: audit 2024-03-04T15:56:38.626939+0000 mon.a (mon.0) 1781 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:56:39.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:39 smithi151 bash[23756]: cluster 2024-03-04T15:56:38.859827+0000 mgr.y (mgr.24428) 288 : cluster 0 pgmap v320: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:56:39.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:39 smithi151 bash[23756]: audit 2024-03-04T15:56:39.143508+0000 mon.a (mon.0) 1782 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi117-63923-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:39.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:39 smithi151 bash[23756]: cluster 2024-03-04T15:56:39.157079+0000 mon.a (mon.0) 1783 : cluster 0 osdmap e210: 8 total, 8 up, 8 in 2024-03-04T15:56:39.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:39 smithi151 bash[23756]: audit 2024-03-04T15:56:39.160945+0000 mon.c (mon.2) 333 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:39.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:39 smithi151 bash[23756]: audit 2024-03-04T15:56:39.170620+0000 mon.a (mon.0) 1784 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:40.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:39 smithi117 bash[21480]: audit 2024-03-04T15:56:38.528466+0000 mon.c (mon.2) 331 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:56:40.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:39 smithi117 bash[21480]: audit 2024-03-04T15:56:38.530132+0000 mon.c (mon.2) 332 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:56:40.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:39 smithi117 bash[21480]: audit 2024-03-04T15:56:38.612358+0000 mon.a (mon.0) 1780 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi117-63923-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:40.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:39 smithi117 bash[21480]: audit 2024-03-04T15:56:38.626939+0000 mon.a (mon.0) 1781 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:56:40.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:39 smithi117 bash[21480]: cluster 2024-03-04T15:56:38.859827+0000 mgr.y (mgr.24428) 288 : cluster 0 pgmap v320: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:56:40.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:39 smithi117 bash[21480]: audit 2024-03-04T15:56:39.143508+0000 mon.a (mon.0) 1782 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi117-63923-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:40.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:39 smithi117 bash[21480]: cluster 2024-03-04T15:56:39.157079+0000 mon.a (mon.0) 1783 : cluster 0 osdmap e210: 8 total, 8 up, 8 in 2024-03-04T15:56:40.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:39 smithi117 bash[21480]: audit 2024-03-04T15:56:39.160945+0000 mon.c (mon.2) 333 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:40.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:39 smithi117 bash[21480]: audit 2024-03-04T15:56:39.170620+0000 mon.a (mon.0) 1784 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:40.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:39 smithi117 bash[28843]: audit 2024-03-04T15:56:38.528466+0000 mon.c (mon.2) 331 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:56:40.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:39 smithi117 bash[28843]: audit 2024-03-04T15:56:38.530132+0000 mon.c (mon.2) 332 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:56:40.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:39 smithi117 bash[28843]: audit 2024-03-04T15:56:38.612358+0000 mon.a (mon.0) 1780 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi117-63923-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:40.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:39 smithi117 bash[28843]: audit 2024-03-04T15:56:38.626939+0000 mon.a (mon.0) 1781 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:56:40.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:39 smithi117 bash[28843]: cluster 2024-03-04T15:56:38.859827+0000 mgr.y (mgr.24428) 288 : cluster 0 pgmap v320: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 582 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:56:40.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:39 smithi117 bash[28843]: audit 2024-03-04T15:56:39.143508+0000 mon.a (mon.0) 1782 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi117-63923-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:40.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:39 smithi117 bash[28843]: cluster 2024-03-04T15:56:39.157079+0000 mon.a (mon.0) 1783 : cluster 0 osdmap e210: 8 total, 8 up, 8 in 2024-03-04T15:56:40.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:39 smithi117 bash[28843]: audit 2024-03-04T15:56:39.160945+0000 mon.c (mon.2) 333 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:40.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:39 smithi117 bash[28843]: audit 2024-03-04T15:56:39.170620+0000 mon.a (mon.0) 1784 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:40.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:40 smithi151 bash[23756]: cluster 2024-03-04T15:56:39.624513+0000 mon.a (mon.0) 1785 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:40.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:40 smithi151 bash[23756]: audit 2024-03-04T15:56:40.146812+0000 mon.a (mon.0) 1786 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:40.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:40 smithi151 bash[23756]: cluster 2024-03-04T15:56:40.156509+0000 mon.a (mon.0) 1787 : cluster 0 osdmap e211: 8 total, 8 up, 8 in 2024-03-04T15:56:40.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:40 smithi151 bash[23756]: audit 2024-03-04T15:56:40.166118+0000 mon.a (mon.0) 1788 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi117-63915-31"}]: dispatch 2024-03-04T15:56:40.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:40 smithi151 bash[23756]: audit 2024-03-04T15:56:40.220856+0000 mon.c (mon.2) 334 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:41.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:40 smithi117 bash[21480]: cluster 2024-03-04T15:56:39.624513+0000 mon.a (mon.0) 1785 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:41.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:40 smithi117 bash[21480]: audit 2024-03-04T15:56:40.146812+0000 mon.a (mon.0) 1786 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:41.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:40 smithi117 bash[21480]: cluster 2024-03-04T15:56:40.156509+0000 mon.a (mon.0) 1787 : cluster 0 osdmap e211: 8 total, 8 up, 8 in 2024-03-04T15:56:41.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:40 smithi117 bash[21480]: audit 2024-03-04T15:56:40.166118+0000 mon.a (mon.0) 1788 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi117-63915-31"}]: dispatch 2024-03-04T15:56:41.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:40 smithi117 bash[21480]: audit 2024-03-04T15:56:40.220856+0000 mon.c (mon.2) 334 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:41.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:40 smithi117 bash[28843]: cluster 2024-03-04T15:56:39.624513+0000 mon.a (mon.0) 1785 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:41.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:40 smithi117 bash[28843]: audit 2024-03-04T15:56:40.146812+0000 mon.a (mon.0) 1786 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:41.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:40 smithi117 bash[28843]: cluster 2024-03-04T15:56:40.156509+0000 mon.a (mon.0) 1787 : cluster 0 osdmap e211: 8 total, 8 up, 8 in 2024-03-04T15:56:41.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:40 smithi117 bash[28843]: audit 2024-03-04T15:56:40.166118+0000 mon.a (mon.0) 1788 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi117-63915-31"}]: dispatch 2024-03-04T15:56:41.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:40 smithi117 bash[28843]: audit 2024-03-04T15:56:40.220856+0000 mon.c (mon.2) 334 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:41.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:41 smithi151 bash[23756]: audit 2024-03-04T15:56:40.654096+0000 mon.a (mon.0) 1789 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:41.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:41 smithi151 bash[23756]: cluster 2024-03-04T15:56:40.860760+0000 mgr.y (mgr.24428) 289 : cluster 0 pgmap v323: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 582 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:56:41.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:41 smithi151 bash[23756]: audit 2024-03-04T15:56:41.152100+0000 mon.a (mon.0) 1790 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi117-63915-31"}]': finished 2024-03-04T15:56:41.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:41 smithi151 bash[23756]: audit 2024-03-04T15:56:41.152268+0000 mon.a (mon.0) 1791 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-21", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:56:41.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:41 smithi151 bash[23756]: cluster 2024-03-04T15:56:41.160673+0000 mon.a (mon.0) 1792 : cluster 0 osdmap e212: 8 total, 8 up, 8 in 2024-03-04T15:56:41.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:41 smithi151 bash[23756]: audit 2024-03-04T15:56:41.162133+0000 mon.c (mon.2) 335 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-21"}]: dispatch 2024-03-04T15:56:41.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:41 smithi151 bash[23756]: audit 2024-03-04T15:56:41.165174+0000 mon.a (mon.0) 1793 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi117-63915-31"}]: dispatch 2024-03-04T15:56:41.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:41 smithi151 bash[23756]: audit 2024-03-04T15:56:41.166658+0000 mon.a (mon.0) 1794 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-21"}]: dispatch 2024-03-04T15:56:41.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:41 smithi151 bash[23756]: audit 2024-03-04T15:56:41.167328+0000 mon.a (mon.0) 1795 : audit 1 from='client.? 172.21.15.117:0/3257130125' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi117-63923-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:42.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:41 smithi117 bash[28843]: audit 2024-03-04T15:56:40.654096+0000 mon.a (mon.0) 1789 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:42.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:41 smithi117 bash[28843]: cluster 2024-03-04T15:56:40.860760+0000 mgr.y (mgr.24428) 289 : cluster 0 pgmap v323: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 582 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:56:42.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:41 smithi117 bash[28843]: audit 2024-03-04T15:56:41.152100+0000 mon.a (mon.0) 1790 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi117-63915-31"}]': finished 2024-03-04T15:56:42.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:41 smithi117 bash[28843]: audit 2024-03-04T15:56:41.152268+0000 mon.a (mon.0) 1791 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-21", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:56:42.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:41 smithi117 bash[28843]: cluster 2024-03-04T15:56:41.160673+0000 mon.a (mon.0) 1792 : cluster 0 osdmap e212: 8 total, 8 up, 8 in 2024-03-04T15:56:42.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:41 smithi117 bash[28843]: audit 2024-03-04T15:56:41.162133+0000 mon.c (mon.2) 335 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-21"}]: dispatch 2024-03-04T15:56:42.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:41 smithi117 bash[28843]: audit 2024-03-04T15:56:41.165174+0000 mon.a (mon.0) 1793 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi117-63915-31"}]: dispatch 2024-03-04T15:56:42.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:41 smithi117 bash[28843]: audit 2024-03-04T15:56:41.166658+0000 mon.a (mon.0) 1794 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-21"}]: dispatch 2024-03-04T15:56:42.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:41 smithi117 bash[28843]: audit 2024-03-04T15:56:41.167328+0000 mon.a (mon.0) 1795 : audit 1 from='client.? 172.21.15.117:0/3257130125' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi117-63923-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:42.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:41 smithi117 bash[21480]: audit 2024-03-04T15:56:40.654096+0000 mon.a (mon.0) 1789 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:42.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:41 smithi117 bash[21480]: cluster 2024-03-04T15:56:40.860760+0000 mgr.y (mgr.24428) 289 : cluster 0 pgmap v323: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 582 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:56:42.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:41 smithi117 bash[21480]: audit 2024-03-04T15:56:41.152100+0000 mon.a (mon.0) 1790 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi117-63915-31"}]': finished 2024-03-04T15:56:42.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:41 smithi117 bash[21480]: audit 2024-03-04T15:56:41.152268+0000 mon.a (mon.0) 1791 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-21", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:56:42.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:41 smithi117 bash[21480]: cluster 2024-03-04T15:56:41.160673+0000 mon.a (mon.0) 1792 : cluster 0 osdmap e212: 8 total, 8 up, 8 in 2024-03-04T15:56:42.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:41 smithi117 bash[21480]: audit 2024-03-04T15:56:41.162133+0000 mon.c (mon.2) 335 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-21"}]: dispatch 2024-03-04T15:56:42.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:41 smithi117 bash[21480]: audit 2024-03-04T15:56:41.165174+0000 mon.a (mon.0) 1793 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi117-63915-31"}]: dispatch 2024-03-04T15:56:42.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:41 smithi117 bash[21480]: audit 2024-03-04T15:56:41.166658+0000 mon.a (mon.0) 1794 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-21"}]: dispatch 2024-03-04T15:56:42.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:41 smithi117 bash[21480]: audit 2024-03-04T15:56:41.167328+0000 mon.a (mon.0) 1795 : audit 1 from='client.? 172.21.15.117:0/3257130125' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi117-63923-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:43.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:43 smithi151 bash[23756]: audit 2024-03-04T15:56:42.155795+0000 mon.a (mon.0) 1796 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi117-63915-31"}]': finished 2024-03-04T15:56:43.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:43 smithi151 bash[23756]: audit 2024-03-04T15:56:42.155917+0000 mon.a (mon.0) 1797 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-21"}]': finished 2024-03-04T15:56:43.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:43 smithi151 bash[23756]: audit 2024-03-04T15:56:42.156137+0000 mon.a (mon.0) 1798 : audit 1 from='client.? 172.21.15.117:0/3257130125' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi117-63923-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:43.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:43 smithi151 bash[23756]: cluster 2024-03-04T15:56:42.164381+0000 mon.a (mon.0) 1799 : cluster 0 osdmap e213: 8 total, 8 up, 8 in 2024-03-04T15:56:43.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:43 smithi151 bash[23756]: audit 2024-03-04T15:56:42.167387+0000 mon.c (mon.2) 336 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-21", "mode": "writeback"}]: dispatch 2024-03-04T15:56:43.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:43 smithi151 bash[23756]: audit 2024-03-04T15:56:42.169579+0000 mon.a (mon.0) 1800 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-21", "mode": "writeback"}]: dispatch 2024-03-04T15:56:43.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:43 smithi151 bash[23756]: audit 2024-03-04T15:56:42.212663+0000 mon.c (mon.2) 337 : audit 1 from='client.? 172.21.15.117:0/220667221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:43.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:43 smithi151 bash[23756]: audit 2024-03-04T15:56:42.213245+0000 mon.a (mon.0) 1801 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:43.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:43 smithi151 bash[23756]: audit 2024-03-04T15:56:42.214438+0000 mon.c (mon.2) 338 : audit 1 from='client.? 172.21.15.117:0/220667221' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:43.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:43 smithi151 bash[23756]: audit 2024-03-04T15:56:42.214783+0000 mon.a (mon.0) 1802 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:43.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:43 smithi151 bash[23756]: audit 2024-03-04T15:56:42.215369+0000 mon.c (mon.2) 339 : audit 1 from='client.? 172.21.15.117:0/220667221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi117-63915-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:43.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:43 smithi151 bash[23756]: audit 2024-03-04T15:56:42.215689+0000 mon.a (mon.0) 1803 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi117-63915-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:43.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[21480]: audit 2024-03-04T15:56:42.155795+0000 mon.a (mon.0) 1796 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi117-63915-31"}]': finished 2024-03-04T15:56:43.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[21480]: audit 2024-03-04T15:56:42.155917+0000 mon.a (mon.0) 1797 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-21"}]': finished 2024-03-04T15:56:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[21480]: audit 2024-03-04T15:56:42.156137+0000 mon.a (mon.0) 1798 : audit 1 from='client.? 172.21.15.117:0/3257130125' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi117-63923-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[21480]: cluster 2024-03-04T15:56:42.164381+0000 mon.a (mon.0) 1799 : cluster 0 osdmap e213: 8 total, 8 up, 8 in 2024-03-04T15:56:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[21480]: audit 2024-03-04T15:56:42.167387+0000 mon.c (mon.2) 336 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-21", "mode": "writeback"}]: dispatch 2024-03-04T15:56:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[21480]: audit 2024-03-04T15:56:42.169579+0000 mon.a (mon.0) 1800 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-21", "mode": "writeback"}]: dispatch 2024-03-04T15:56:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[21480]: audit 2024-03-04T15:56:42.212663+0000 mon.c (mon.2) 337 : audit 1 from='client.? 172.21.15.117:0/220667221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[21480]: audit 2024-03-04T15:56:42.213245+0000 mon.a (mon.0) 1801 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[21480]: audit 2024-03-04T15:56:42.214438+0000 mon.c (mon.2) 338 : audit 1 from='client.? 172.21.15.117:0/220667221' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[21480]: audit 2024-03-04T15:56:42.214783+0000 mon.a (mon.0) 1802 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[21480]: audit 2024-03-04T15:56:42.215369+0000 mon.c (mon.2) 339 : audit 1 from='client.? 172.21.15.117:0/220667221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi117-63915-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[21480]: audit 2024-03-04T15:56:42.215689+0000 mon.a (mon.0) 1803 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi117-63915-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:43.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[28843]: audit 2024-03-04T15:56:42.155795+0000 mon.a (mon.0) 1796 : audit 1 from='client.? 172.21.15.117:0/11847296' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi117-63915-31"}]': finished 2024-03-04T15:56:43.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[28843]: audit 2024-03-04T15:56:42.155917+0000 mon.a (mon.0) 1797 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-21"}]': finished 2024-03-04T15:56:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[28843]: audit 2024-03-04T15:56:42.156137+0000 mon.a (mon.0) 1798 : audit 1 from='client.? 172.21.15.117:0/3257130125' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi117-63923-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[28843]: cluster 2024-03-04T15:56:42.164381+0000 mon.a (mon.0) 1799 : cluster 0 osdmap e213: 8 total, 8 up, 8 in 2024-03-04T15:56:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[28843]: audit 2024-03-04T15:56:42.167387+0000 mon.c (mon.2) 336 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-21", "mode": "writeback"}]: dispatch 2024-03-04T15:56:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[28843]: audit 2024-03-04T15:56:42.169579+0000 mon.a (mon.0) 1800 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-21", "mode": "writeback"}]: dispatch 2024-03-04T15:56:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[28843]: audit 2024-03-04T15:56:42.212663+0000 mon.c (mon.2) 337 : audit 1 from='client.? 172.21.15.117:0/220667221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[28843]: audit 2024-03-04T15:56:42.213245+0000 mon.a (mon.0) 1801 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[28843]: audit 2024-03-04T15:56:42.214438+0000 mon.c (mon.2) 338 : audit 1 from='client.? 172.21.15.117:0/220667221' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[28843]: audit 2024-03-04T15:56:42.214783+0000 mon.a (mon.0) 1802 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[28843]: audit 2024-03-04T15:56:42.215369+0000 mon.c (mon.2) 339 : audit 1 from='client.? 172.21.15.117:0/220667221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi117-63915-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:43.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:43 smithi117 bash[28843]: audit 2024-03-04T15:56:42.215689+0000 mon.a (mon.0) 1803 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi117-63915-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:44.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:44 smithi151 bash[23756]: cluster 2024-03-04T15:56:42.861785+0000 mgr.y (mgr.24428) 290 : cluster 0 pgmap v326: 260 pgs: 32 creating+peering, 228 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:56:44.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:44 smithi151 bash[23756]: cluster 2024-03-04T15:56:43.158452+0000 mon.a (mon.0) 1804 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:56:44.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:44 smithi151 bash[23756]: audit 2024-03-04T15:56:43.168107+0000 mon.a (mon.0) 1805 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-21", "mode": "writeback"}]': finished 2024-03-04T15:56:44.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:44 smithi151 bash[23756]: audit 2024-03-04T15:56:43.168386+0000 mon.a (mon.0) 1806 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi117-63915-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:44.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:44 smithi151 bash[23756]: cluster 2024-03-04T15:56:43.182001+0000 mon.a (mon.0) 1807 : cluster 0 osdmap e214: 8 total, 8 up, 8 in 2024-03-04T15:56:44.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:44 smithi151 bash[23756]: audit 2024-03-04T15:56:43.185354+0000 mon.c (mon.2) 340 : audit 1 from='client.? 172.21.15.117:0/220667221' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi117-63915-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:44.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:44 smithi151 bash[23756]: audit 2024-03-04T15:56:43.188402+0000 mon.a (mon.0) 1808 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi117-63915-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:44.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:44 smithi151 bash[23756]: audit 2024-03-04T15:56:43.189887+0000 mon.a (mon.0) 1809 : audit 1 from='client.? 172.21.15.117:0/1333493058' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi117-63923-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:44.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:44 smithi151 bash[23756]: audit 2024-03-04T15:56:43.276393+0000 mon.c (mon.2) 341 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:44.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:44 smithi151 bash[23756]: audit 2024-03-04T15:56:43.277006+0000 mon.a (mon.0) 1810 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:44.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:44 smithi151 bash[23756]: audit 2024-03-04T15:56:43.364497+0000 mon.c (mon.2) 342 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:56:45.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[21480]: cluster 2024-03-04T15:56:42.861785+0000 mgr.y (mgr.24428) 290 : cluster 0 pgmap v326: 260 pgs: 32 creating+peering, 228 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:56:45.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[21480]: cluster 2024-03-04T15:56:43.158452+0000 mon.a (mon.0) 1804 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:56:45.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[21480]: audit 2024-03-04T15:56:43.168107+0000 mon.a (mon.0) 1805 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-21", "mode": "writeback"}]': finished 2024-03-04T15:56:45.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[21480]: audit 2024-03-04T15:56:43.168386+0000 mon.a (mon.0) 1806 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi117-63915-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:45.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[21480]: cluster 2024-03-04T15:56:43.182001+0000 mon.a (mon.0) 1807 : cluster 0 osdmap e214: 8 total, 8 up, 8 in 2024-03-04T15:56:45.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[21480]: audit 2024-03-04T15:56:43.185354+0000 mon.c (mon.2) 340 : audit 1 from='client.? 172.21.15.117:0/220667221' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi117-63915-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:45.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[21480]: audit 2024-03-04T15:56:43.188402+0000 mon.a (mon.0) 1808 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi117-63915-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:45.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[21480]: audit 2024-03-04T15:56:43.189887+0000 mon.a (mon.0) 1809 : audit 1 from='client.? 172.21.15.117:0/1333493058' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi117-63923-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:45.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[21480]: audit 2024-03-04T15:56:43.276393+0000 mon.c (mon.2) 341 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:45.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[21480]: audit 2024-03-04T15:56:43.277006+0000 mon.a (mon.0) 1810 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:45.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[21480]: audit 2024-03-04T15:56:43.364497+0000 mon.c (mon.2) 342 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:56:45.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[28843]: cluster 2024-03-04T15:56:42.861785+0000 mgr.y (mgr.24428) 290 : cluster 0 pgmap v326: 260 pgs: 32 creating+peering, 228 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:56:45.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[28843]: cluster 2024-03-04T15:56:43.158452+0000 mon.a (mon.0) 1804 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:56:45.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[28843]: audit 2024-03-04T15:56:43.168107+0000 mon.a (mon.0) 1805 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-21", "mode": "writeback"}]': finished 2024-03-04T15:56:45.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[28843]: audit 2024-03-04T15:56:43.168386+0000 mon.a (mon.0) 1806 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi117-63915-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:45.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[28843]: cluster 2024-03-04T15:56:43.182001+0000 mon.a (mon.0) 1807 : cluster 0 osdmap e214: 8 total, 8 up, 8 in 2024-03-04T15:56:45.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[28843]: audit 2024-03-04T15:56:43.185354+0000 mon.c (mon.2) 340 : audit 1 from='client.? 172.21.15.117:0/220667221' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi117-63915-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:45.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[28843]: audit 2024-03-04T15:56:43.188402+0000 mon.a (mon.0) 1808 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi117-63915-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:45.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[28843]: audit 2024-03-04T15:56:43.189887+0000 mon.a (mon.0) 1809 : audit 1 from='client.? 172.21.15.117:0/1333493058' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi117-63923-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:45.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[28843]: audit 2024-03-04T15:56:43.276393+0000 mon.c (mon.2) 341 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:45.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[28843]: audit 2024-03-04T15:56:43.277006+0000 mon.a (mon.0) 1810 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:45.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:44 smithi117 bash[28843]: audit 2024-03-04T15:56:43.364497+0000 mon.c (mon.2) 342 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:56:45.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:45 smithi151 bash[23756]: audit 2024-03-04T15:56:44.172108+0000 mon.a (mon.0) 1811 : audit 1 from='client.? 172.21.15.117:0/1333493058' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi117-63923-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:45.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:45 smithi151 bash[23756]: audit 2024-03-04T15:56:44.172311+0000 mon.a (mon.0) 1812 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:56:45.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:45 smithi151 bash[23756]: audit 2024-03-04T15:56:44.186595+0000 mon.c (mon.2) 343 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-21"}]: dispatch 2024-03-04T15:56:45.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:45 smithi151 bash[23756]: cluster 2024-03-04T15:56:44.635974+0000 mon.a (mon.0) 1813 : cluster 0 osdmap e215: 8 total, 8 up, 8 in 2024-03-04T15:56:45.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:45 smithi151 bash[23756]: audit 2024-03-04T15:56:44.638247+0000 mon.a (mon.0) 1814 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-21"}]: dispatch 2024-03-04T15:56:45.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:45 smithi151 bash[23756]: cluster 2024-03-04T15:56:44.862665+0000 mgr.y (mgr.24428) 291 : cluster 0 pgmap v329: 292 pgs: 32 unknown, 32 creating+peering, 228 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:56:45.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:45 smithi151 bash[23756]: cluster 2024-03-04T15:56:45.173478+0000 mon.a (mon.0) 1815 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:56:45.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:45 smithi151 bash[23756]: audit 2024-03-04T15:56:45.176366+0000 mon.a (mon.0) 1816 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi117-63915-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi117-63915-32"}]': finished 2024-03-04T15:56:45.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:45 smithi151 bash[23756]: audit 2024-03-04T15:56:45.176608+0000 mon.a (mon.0) 1817 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-21"}]': finished 2024-03-04T15:56:45.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:45 smithi151 bash[23756]: cluster 2024-03-04T15:56:45.185368+0000 mon.a (mon.0) 1818 : cluster 0 osdmap e216: 8 total, 8 up, 8 in 2024-03-04T15:56:45.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:45 smithi151 bash[23756]: audit 2024-03-04T15:56:45.196018+0000 mon.a (mon.0) 1819 : audit 1 from='client.? 172.21.15.117:0/3788687562' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi117-63923-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:46.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[21480]: audit 2024-03-04T15:56:44.172108+0000 mon.a (mon.0) 1811 : audit 1 from='client.? 172.21.15.117:0/1333493058' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi117-63923-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:46.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[21480]: audit 2024-03-04T15:56:44.172311+0000 mon.a (mon.0) 1812 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:56:46.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[21480]: audit 2024-03-04T15:56:44.186595+0000 mon.c (mon.2) 343 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-21"}]: dispatch 2024-03-04T15:56:46.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[21480]: cluster 2024-03-04T15:56:44.635974+0000 mon.a (mon.0) 1813 : cluster 0 osdmap e215: 8 total, 8 up, 8 in 2024-03-04T15:56:46.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[21480]: audit 2024-03-04T15:56:44.638247+0000 mon.a (mon.0) 1814 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-21"}]: dispatch 2024-03-04T15:56:46.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[21480]: cluster 2024-03-04T15:56:44.862665+0000 mgr.y (mgr.24428) 291 : cluster 0 pgmap v329: 292 pgs: 32 unknown, 32 creating+peering, 228 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:56:46.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[21480]: cluster 2024-03-04T15:56:45.173478+0000 mon.a (mon.0) 1815 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:56:46.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[21480]: audit 2024-03-04T15:56:45.176366+0000 mon.a (mon.0) 1816 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi117-63915-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi117-63915-32"}]': finished 2024-03-04T15:56:46.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[21480]: audit 2024-03-04T15:56:45.176608+0000 mon.a (mon.0) 1817 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-21"}]': finished 2024-03-04T15:56:46.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[21480]: cluster 2024-03-04T15:56:45.185368+0000 mon.a (mon.0) 1818 : cluster 0 osdmap e216: 8 total, 8 up, 8 in 2024-03-04T15:56:46.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[21480]: audit 2024-03-04T15:56:45.196018+0000 mon.a (mon.0) 1819 : audit 1 from='client.? 172.21.15.117:0/3788687562' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi117-63923-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:46.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[28843]: audit 2024-03-04T15:56:44.172108+0000 mon.a (mon.0) 1811 : audit 1 from='client.? 172.21.15.117:0/1333493058' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi117-63923-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:46.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[28843]: audit 2024-03-04T15:56:44.172311+0000 mon.a (mon.0) 1812 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:56:46.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[28843]: audit 2024-03-04T15:56:44.186595+0000 mon.c (mon.2) 343 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-21"}]: dispatch 2024-03-04T15:56:46.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[28843]: cluster 2024-03-04T15:56:44.635974+0000 mon.a (mon.0) 1813 : cluster 0 osdmap e215: 8 total, 8 up, 8 in 2024-03-04T15:56:46.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[28843]: audit 2024-03-04T15:56:44.638247+0000 mon.a (mon.0) 1814 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-21"}]: dispatch 2024-03-04T15:56:46.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[28843]: cluster 2024-03-04T15:56:44.862665+0000 mgr.y (mgr.24428) 291 : cluster 0 pgmap v329: 292 pgs: 32 unknown, 32 creating+peering, 228 active+clean; 583 KiB data, 587 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:56:46.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[28843]: cluster 2024-03-04T15:56:45.173478+0000 mon.a (mon.0) 1815 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:56:46.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[28843]: audit 2024-03-04T15:56:45.176366+0000 mon.a (mon.0) 1816 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi117-63915-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi117-63915-32"}]': finished 2024-03-04T15:56:46.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[28843]: audit 2024-03-04T15:56:45.176608+0000 mon.a (mon.0) 1817 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-21"}]': finished 2024-03-04T15:56:46.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[28843]: cluster 2024-03-04T15:56:45.185368+0000 mon.a (mon.0) 1818 : cluster 0 osdmap e216: 8 total, 8 up, 8 in 2024-03-04T15:56:46.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:45 smithi117 bash[28843]: audit 2024-03-04T15:56:45.196018+0000 mon.a (mon.0) 1819 : audit 1 from='client.? 172.21.15.117:0/3788687562' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi117-63923-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:46.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:56:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:56:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:56:46.932 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:56:46 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:56:47.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:47 smithi117 bash[28843]: audit 2024-03-04T15:56:46.179838+0000 mon.a (mon.0) 1820 : audit 1 from='client.? 172.21.15.117:0/3788687562' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi117-63923-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:47.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:47 smithi117 bash[28843]: cluster 2024-03-04T15:56:46.187975+0000 mon.a (mon.0) 1821 : cluster 0 osdmap e217: 8 total, 8 up, 8 in 2024-03-04T15:56:47.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:47 smithi117 bash[21480]: audit 2024-03-04T15:56:46.179838+0000 mon.a (mon.0) 1820 : audit 1 from='client.? 172.21.15.117:0/3788687562' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi117-63923-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:47.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:47 smithi117 bash[21480]: cluster 2024-03-04T15:56:46.187975+0000 mon.a (mon.0) 1821 : cluster 0 osdmap e217: 8 total, 8 up, 8 in 2024-03-04T15:56:47.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:47 smithi151 bash[23756]: audit 2024-03-04T15:56:46.179838+0000 mon.a (mon.0) 1820 : audit 1 from='client.? 172.21.15.117:0/3788687562' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi117-63923-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:47.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:47 smithi151 bash[23756]: cluster 2024-03-04T15:56:46.187975+0000 mon.a (mon.0) 1821 : cluster 0 osdmap e217: 8 total, 8 up, 8 in 2024-03-04T15:56:48.235 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:48 smithi117 bash[21480]: audit 2024-03-04T15:56:46.594928+0000 mgr.y (mgr.24428) 292 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:56:48.235 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:48 smithi117 bash[21480]: cluster 2024-03-04T15:56:46.865376+0000 mgr.y (mgr.24428) 293 : cluster 0 pgmap v332: 300 pgs: 300 active+clean; 584 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-04T15:56:48.235 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:48 smithi117 bash[21480]: cluster 2024-03-04T15:56:47.179888+0000 mon.a (mon.0) 1822 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:48.235 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:48 smithi117 bash[21480]: cluster 2024-03-04T15:56:47.205448+0000 mon.a (mon.0) 1823 : cluster 0 osdmap e218: 8 total, 8 up, 8 in 2024-03-04T15:56:48.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:48 smithi117 bash[21480]: audit 2024-03-04T15:56:47.208670+0000 mon.c (mon.2) 344 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:48.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:48 smithi117 bash[21480]: audit 2024-03-04T15:56:47.215710+0000 mon.c (mon.2) 345 : audit 1 from='client.? 172.21.15.117:0/220667221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:48.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:48 smithi117 bash[21480]: audit 2024-03-04T15:56:47.219092+0000 mon.a (mon.0) 1824 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:48.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:48 smithi117 bash[21480]: audit 2024-03-04T15:56:47.221448+0000 mon.a (mon.0) 1825 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:48.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:48 smithi117 bash[28843]: audit 2024-03-04T15:56:46.594928+0000 mgr.y (mgr.24428) 292 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:56:48.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:48 smithi117 bash[28843]: cluster 2024-03-04T15:56:46.865376+0000 mgr.y (mgr.24428) 293 : cluster 0 pgmap v332: 300 pgs: 300 active+clean; 584 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-04T15:56:48.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:48 smithi117 bash[28843]: cluster 2024-03-04T15:56:47.179888+0000 mon.a (mon.0) 1822 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:48.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:48 smithi117 bash[28843]: cluster 2024-03-04T15:56:47.205448+0000 mon.a (mon.0) 1823 : cluster 0 osdmap e218: 8 total, 8 up, 8 in 2024-03-04T15:56:48.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:48 smithi117 bash[28843]: audit 2024-03-04T15:56:47.208670+0000 mon.c (mon.2) 344 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:48.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:48 smithi117 bash[28843]: audit 2024-03-04T15:56:47.215710+0000 mon.c (mon.2) 345 : audit 1 from='client.? 172.21.15.117:0/220667221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:48.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:48 smithi117 bash[28843]: audit 2024-03-04T15:56:47.219092+0000 mon.a (mon.0) 1824 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:48.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:48 smithi117 bash[28843]: audit 2024-03-04T15:56:47.221448+0000 mon.a (mon.0) 1825 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:48 smithi151 bash[23756]: audit 2024-03-04T15:56:46.594928+0000 mgr.y (mgr.24428) 292 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:56:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:48 smithi151 bash[23756]: cluster 2024-03-04T15:56:46.865376+0000 mgr.y (mgr.24428) 293 : cluster 0 pgmap v332: 300 pgs: 300 active+clean; 584 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-04T15:56:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:48 smithi151 bash[23756]: cluster 2024-03-04T15:56:47.179888+0000 mon.a (mon.0) 1822 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:48 smithi151 bash[23756]: cluster 2024-03-04T15:56:47.205448+0000 mon.a (mon.0) 1823 : cluster 0 osdmap e218: 8 total, 8 up, 8 in 2024-03-04T15:56:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:48 smithi151 bash[23756]: audit 2024-03-04T15:56:47.208670+0000 mon.c (mon.2) 344 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:48 smithi151 bash[23756]: audit 2024-03-04T15:56:47.215710+0000 mon.c (mon.2) 345 : audit 1 from='client.? 172.21.15.117:0/220667221' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:48 smithi151 bash[23756]: audit 2024-03-04T15:56:47.219092+0000 mon.a (mon.0) 1824 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:48 smithi151 bash[23756]: audit 2024-03-04T15:56:47.221448+0000 mon.a (mon.0) 1825 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:49.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:49 smithi117 bash[21480]: audit 2024-03-04T15:56:48.195807+0000 mon.a (mon.0) 1826 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:49.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:49 smithi117 bash[21480]: audit 2024-03-04T15:56:48.195997+0000 mon.a (mon.0) 1827 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi117-63915-32"}]': finished 2024-03-04T15:56:49.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:49 smithi117 bash[21480]: cluster 2024-03-04T15:56:48.208741+0000 mon.a (mon.0) 1828 : cluster 0 osdmap e219: 8 total, 8 up, 8 in 2024-03-04T15:56:49.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:49 smithi117 bash[21480]: audit 2024-03-04T15:56:48.213157+0000 mon.c (mon.2) 346 : audit 1 from='client.? 172.21.15.117:0/220667221' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:49.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:49 smithi117 bash[21480]: audit 2024-03-04T15:56:48.225046+0000 mon.a (mon.0) 1829 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:49.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:49 smithi117 bash[21480]: audit 2024-03-04T15:56:49.199209+0000 mon.a (mon.0) 1830 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi117-63915-32"}]': finished 2024-03-04T15:56:49.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:49 smithi117 bash[21480]: cluster 2024-03-04T15:56:49.210707+0000 mon.a (mon.0) 1831 : cluster 0 osdmap e220: 8 total, 8 up, 8 in 2024-03-04T15:56:49.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:49 smithi117 bash[28843]: audit 2024-03-04T15:56:48.195807+0000 mon.a (mon.0) 1826 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:49.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:49 smithi117 bash[28843]: audit 2024-03-04T15:56:48.195997+0000 mon.a (mon.0) 1827 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi117-63915-32"}]': finished 2024-03-04T15:56:49.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:49 smithi117 bash[28843]: cluster 2024-03-04T15:56:48.208741+0000 mon.a (mon.0) 1828 : cluster 0 osdmap e219: 8 total, 8 up, 8 in 2024-03-04T15:56:49.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:49 smithi117 bash[28843]: audit 2024-03-04T15:56:48.213157+0000 mon.c (mon.2) 346 : audit 1 from='client.? 172.21.15.117:0/220667221' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:49.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:49 smithi117 bash[28843]: audit 2024-03-04T15:56:48.225046+0000 mon.a (mon.0) 1829 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:49.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:49 smithi117 bash[28843]: audit 2024-03-04T15:56:49.199209+0000 mon.a (mon.0) 1830 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi117-63915-32"}]': finished 2024-03-04T15:56:49.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:49 smithi117 bash[28843]: cluster 2024-03-04T15:56:49.210707+0000 mon.a (mon.0) 1831 : cluster 0 osdmap e220: 8 total, 8 up, 8 in 2024-03-04T15:56:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:49 smithi151 bash[23756]: audit 2024-03-04T15:56:48.195807+0000 mon.a (mon.0) 1826 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:49 smithi151 bash[23756]: audit 2024-03-04T15:56:48.195997+0000 mon.a (mon.0) 1827 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi117-63915-32"}]': finished 2024-03-04T15:56:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:49 smithi151 bash[23756]: cluster 2024-03-04T15:56:48.208741+0000 mon.a (mon.0) 1828 : cluster 0 osdmap e219: 8 total, 8 up, 8 in 2024-03-04T15:56:49.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:49 smithi151 bash[23756]: audit 2024-03-04T15:56:48.213157+0000 mon.c (mon.2) 346 : audit 1 from='client.? 172.21.15.117:0/220667221' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:49.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:49 smithi151 bash[23756]: audit 2024-03-04T15:56:48.225046+0000 mon.a (mon.0) 1829 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi117-63915-32"}]: dispatch 2024-03-04T15:56:49.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:49 smithi151 bash[23756]: audit 2024-03-04T15:56:49.199209+0000 mon.a (mon.0) 1830 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi117-63915-32"}]': finished 2024-03-04T15:56:49.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:49 smithi151 bash[23756]: cluster 2024-03-04T15:56:49.210707+0000 mon.a (mon.0) 1831 : cluster 0 osdmap e220: 8 total, 8 up, 8 in 2024-03-04T15:56:50.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:50 smithi117 bash[21480]: cluster 2024-03-04T15:56:48.866272+0000 mgr.y (mgr.24428) 294 : cluster 0 pgmap v335: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T15:56:50.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:50 smithi117 bash[21480]: audit 2024-03-04T15:56:49.243931+0000 mon.c (mon.2) 347 : audit 1 from='client.? 172.21.15.117:0/1462527519' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:50.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:50 smithi117 bash[21480]: audit 2024-03-04T15:56:49.250757+0000 mon.a (mon.0) 1832 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:50.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:50 smithi117 bash[21480]: audit 2024-03-04T15:56:49.254053+0000 mon.c (mon.2) 348 : audit 1 from='client.? 172.21.15.117:0/1462527519' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:50.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:50 smithi117 bash[21480]: audit 2024-03-04T15:56:49.262392+0000 mon.c (mon.2) 349 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:50.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:50 smithi117 bash[21480]: audit 2024-03-04T15:56:49.693201+0000 mon.a (mon.0) 1833 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:50.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:50 smithi117 bash[21480]: audit 2024-03-04T15:56:49.693836+0000 mon.a (mon.0) 1834 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:50.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:50 smithi117 bash[21480]: audit 2024-03-04T15:56:49.694332+0000 mon.c (mon.2) 350 : audit 1 from='client.? 172.21.15.117:0/1462527519' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi117-63915-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:50.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:50 smithi117 bash[21480]: audit 2024-03-04T15:56:49.694842+0000 mon.a (mon.0) 1835 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi117-63915-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:50.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:50 smithi117 bash[28843]: cluster 2024-03-04T15:56:48.866272+0000 mgr.y (mgr.24428) 294 : cluster 0 pgmap v335: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T15:56:50.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:50 smithi117 bash[28843]: audit 2024-03-04T15:56:49.243931+0000 mon.c (mon.2) 347 : audit 1 from='client.? 172.21.15.117:0/1462527519' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:50.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:50 smithi117 bash[28843]: audit 2024-03-04T15:56:49.250757+0000 mon.a (mon.0) 1832 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:50.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:50 smithi117 bash[28843]: audit 2024-03-04T15:56:49.254053+0000 mon.c (mon.2) 348 : audit 1 from='client.? 172.21.15.117:0/1462527519' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:50.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:50 smithi117 bash[28843]: audit 2024-03-04T15:56:49.262392+0000 mon.c (mon.2) 349 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:50.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:50 smithi117 bash[28843]: audit 2024-03-04T15:56:49.693201+0000 mon.a (mon.0) 1833 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:50.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:50 smithi117 bash[28843]: audit 2024-03-04T15:56:49.693836+0000 mon.a (mon.0) 1834 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:50.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:50 smithi117 bash[28843]: audit 2024-03-04T15:56:49.694332+0000 mon.c (mon.2) 350 : audit 1 from='client.? 172.21.15.117:0/1462527519' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi117-63915-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:50.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:50 smithi117 bash[28843]: audit 2024-03-04T15:56:49.694842+0000 mon.a (mon.0) 1835 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi117-63915-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:50 smithi151 bash[23756]: cluster 2024-03-04T15:56:48.866272+0000 mgr.y (mgr.24428) 294 : cluster 0 pgmap v335: 260 pgs: 32 unknown, 228 active+clean; 584 KiB data, 591 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T15:56:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:50 smithi151 bash[23756]: audit 2024-03-04T15:56:49.243931+0000 mon.c (mon.2) 347 : audit 1 from='client.? 172.21.15.117:0/1462527519' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:50 smithi151 bash[23756]: audit 2024-03-04T15:56:49.250757+0000 mon.a (mon.0) 1832 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:50 smithi151 bash[23756]: audit 2024-03-04T15:56:49.254053+0000 mon.c (mon.2) 348 : audit 1 from='client.? 172.21.15.117:0/1462527519' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:50 smithi151 bash[23756]: audit 2024-03-04T15:56:49.262392+0000 mon.c (mon.2) 349 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:50 smithi151 bash[23756]: audit 2024-03-04T15:56:49.693201+0000 mon.a (mon.0) 1833 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:50 smithi151 bash[23756]: audit 2024-03-04T15:56:49.693836+0000 mon.a (mon.0) 1834 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:50 smithi151 bash[23756]: audit 2024-03-04T15:56:49.694332+0000 mon.c (mon.2) 350 : audit 1 from='client.? 172.21.15.117:0/1462527519' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi117-63915-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:50.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:50 smithi151 bash[23756]: audit 2024-03-04T15:56:49.694842+0000 mon.a (mon.0) 1835 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi117-63915-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:52.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:51 smithi117 bash[21480]: audit 2024-03-04T15:56:50.262614+0000 mon.a (mon.0) 1836 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-23", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:56:52.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:51 smithi117 bash[21480]: audit 2024-03-04T15:56:50.262830+0000 mon.a (mon.0) 1837 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi117-63915-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:52.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:51 smithi117 bash[21480]: cluster 2024-03-04T15:56:50.270583+0000 mon.a (mon.0) 1838 : cluster 0 osdmap e221: 8 total, 8 up, 8 in 2024-03-04T15:56:52.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:51 smithi117 bash[21480]: audit 2024-03-04T15:56:50.272934+0000 mon.c (mon.2) 351 : audit 1 from='client.? 172.21.15.117:0/1462527519' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi117-63915-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:52.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:51 smithi117 bash[21480]: audit 2024-03-04T15:56:50.273007+0000 mon.a (mon.0) 1839 : audit 1 from='client.? 172.21.15.117:0/365069642' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:52.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:51 smithi117 bash[21480]: audit 2024-03-04T15:56:50.275831+0000 mon.c (mon.2) 352 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-23"}]: dispatch 2024-03-04T15:56:52.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:51 smithi117 bash[21480]: audit 2024-03-04T15:56:50.279985+0000 mon.a (mon.0) 1840 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi117-63915-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:52.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:51 smithi117 bash[21480]: audit 2024-03-04T15:56:50.280987+0000 mon.a (mon.0) 1841 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-23"}]: dispatch 2024-03-04T15:56:52.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:51 smithi117 bash[28843]: audit 2024-03-04T15:56:50.262614+0000 mon.a (mon.0) 1836 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-23", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:56:52.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:51 smithi117 bash[28843]: audit 2024-03-04T15:56:50.262830+0000 mon.a (mon.0) 1837 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi117-63915-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:52.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:51 smithi117 bash[28843]: cluster 2024-03-04T15:56:50.270583+0000 mon.a (mon.0) 1838 : cluster 0 osdmap e221: 8 total, 8 up, 8 in 2024-03-04T15:56:52.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:51 smithi117 bash[28843]: audit 2024-03-04T15:56:50.272934+0000 mon.c (mon.2) 351 : audit 1 from='client.? 172.21.15.117:0/1462527519' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi117-63915-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:52.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:51 smithi117 bash[28843]: audit 2024-03-04T15:56:50.273007+0000 mon.a (mon.0) 1839 : audit 1 from='client.? 172.21.15.117:0/365069642' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:52.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:51 smithi117 bash[28843]: audit 2024-03-04T15:56:50.275831+0000 mon.c (mon.2) 352 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-23"}]: dispatch 2024-03-04T15:56:52.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:51 smithi117 bash[28843]: audit 2024-03-04T15:56:50.279985+0000 mon.a (mon.0) 1840 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi117-63915-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:52.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:51 smithi117 bash[28843]: audit 2024-03-04T15:56:50.280987+0000 mon.a (mon.0) 1841 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-23"}]: dispatch 2024-03-04T15:56:52.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:51 smithi151 bash[23756]: audit 2024-03-04T15:56:50.262614+0000 mon.a (mon.0) 1836 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-23", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:56:52.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:51 smithi151 bash[23756]: audit 2024-03-04T15:56:50.262830+0000 mon.a (mon.0) 1837 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi117-63915-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:52.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:51 smithi151 bash[23756]: cluster 2024-03-04T15:56:50.270583+0000 mon.a (mon.0) 1838 : cluster 0 osdmap e221: 8 total, 8 up, 8 in 2024-03-04T15:56:52.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:51 smithi151 bash[23756]: audit 2024-03-04T15:56:50.272934+0000 mon.c (mon.2) 351 : audit 1 from='client.? 172.21.15.117:0/1462527519' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi117-63915-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:52.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:51 smithi151 bash[23756]: audit 2024-03-04T15:56:50.273007+0000 mon.a (mon.0) 1839 : audit 1 from='client.? 172.21.15.117:0/365069642' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:52.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:51 smithi151 bash[23756]: audit 2024-03-04T15:56:50.275831+0000 mon.c (mon.2) 352 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-23"}]: dispatch 2024-03-04T15:56:52.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:51 smithi151 bash[23756]: audit 2024-03-04T15:56:50.279985+0000 mon.a (mon.0) 1840 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi117-63915-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:52.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:51 smithi151 bash[23756]: audit 2024-03-04T15:56:50.280987+0000 mon.a (mon.0) 1841 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-23"}]: dispatch 2024-03-04T15:56:53.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[21480]: cluster 2024-03-04T15:56:50.867124+0000 mgr.y (mgr.24428) 295 : cluster 0 pgmap v338: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:56:53.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[21480]: audit 2024-03-04T15:56:51.266647+0000 mon.a (mon.0) 1842 : audit 1 from='client.? 172.21.15.117:0/365069642' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:53.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[21480]: audit 2024-03-04T15:56:51.266813+0000 mon.a (mon.0) 1843 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-23"}]': finished 2024-03-04T15:56:53.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[21480]: audit 2024-03-04T15:56:51.277373+0000 mon.c (mon.2) 353 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-23", "mode": "writeback"}]: dispatch 2024-03-04T15:56:53.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[21480]: cluster 2024-03-04T15:56:51.724261+0000 mon.a (mon.0) 1844 : cluster 0 osdmap e222: 8 total, 8 up, 8 in 2024-03-04T15:56:53.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[21480]: audit 2024-03-04T15:56:51.742676+0000 mon.a (mon.0) 1845 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-23", "mode": "writeback"}]: dispatch 2024-03-04T15:56:53.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[21480]: cluster 2024-03-04T15:56:52.268113+0000 mon.a (mon.0) 1846 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:56:53.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[21480]: audit 2024-03-04T15:56:52.270997+0000 mon.a (mon.0) 1847 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi117-63915-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi117-63915-33"}]': finished 2024-03-04T15:56:53.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[21480]: audit 2024-03-04T15:56:52.271273+0000 mon.a (mon.0) 1848 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-23", "mode": "writeback"}]': finished 2024-03-04T15:56:53.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[21480]: cluster 2024-03-04T15:56:52.282341+0000 mon.a (mon.0) 1849 : cluster 0 osdmap e223: 8 total, 8 up, 8 in 2024-03-04T15:56:53.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[21480]: audit 2024-03-04T15:56:52.297682+0000 mon.c (mon.2) 354 : audit 1 from='client.? 172.21.15.117:0/3854249032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:53.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[21480]: audit 2024-03-04T15:56:52.298201+0000 mon.a (mon.0) 1850 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:53.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[28843]: cluster 2024-03-04T15:56:50.867124+0000 mgr.y (mgr.24428) 295 : cluster 0 pgmap v338: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:56:53.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[28843]: audit 2024-03-04T15:56:51.266647+0000 mon.a (mon.0) 1842 : audit 1 from='client.? 172.21.15.117:0/365069642' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:53.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[28843]: audit 2024-03-04T15:56:51.266813+0000 mon.a (mon.0) 1843 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-23"}]': finished 2024-03-04T15:56:53.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[28843]: audit 2024-03-04T15:56:51.277373+0000 mon.c (mon.2) 353 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-23", "mode": "writeback"}]: dispatch 2024-03-04T15:56:53.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[28843]: cluster 2024-03-04T15:56:51.724261+0000 mon.a (mon.0) 1844 : cluster 0 osdmap e222: 8 total, 8 up, 8 in 2024-03-04T15:56:53.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[28843]: audit 2024-03-04T15:56:51.742676+0000 mon.a (mon.0) 1845 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-23", "mode": "writeback"}]: dispatch 2024-03-04T15:56:53.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[28843]: cluster 2024-03-04T15:56:52.268113+0000 mon.a (mon.0) 1846 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:56:53.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[28843]: audit 2024-03-04T15:56:52.270997+0000 mon.a (mon.0) 1847 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi117-63915-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi117-63915-33"}]': finished 2024-03-04T15:56:53.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[28843]: audit 2024-03-04T15:56:52.271273+0000 mon.a (mon.0) 1848 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-23", "mode": "writeback"}]': finished 2024-03-04T15:56:53.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[28843]: cluster 2024-03-04T15:56:52.282341+0000 mon.a (mon.0) 1849 : cluster 0 osdmap e223: 8 total, 8 up, 8 in 2024-03-04T15:56:53.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[28843]: audit 2024-03-04T15:56:52.297682+0000 mon.c (mon.2) 354 : audit 1 from='client.? 172.21.15.117:0/3854249032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:53.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:52 smithi117 bash[28843]: audit 2024-03-04T15:56:52.298201+0000 mon.a (mon.0) 1850 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:53.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:52 smithi151 bash[23756]: cluster 2024-03-04T15:56:50.867124+0000 mgr.y (mgr.24428) 295 : cluster 0 pgmap v338: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 591 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:56:53.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:52 smithi151 bash[23756]: audit 2024-03-04T15:56:51.266647+0000 mon.a (mon.0) 1842 : audit 1 from='client.? 172.21.15.117:0/365069642' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:53.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:52 smithi151 bash[23756]: audit 2024-03-04T15:56:51.266813+0000 mon.a (mon.0) 1843 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-23"}]': finished 2024-03-04T15:56:53.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:52 smithi151 bash[23756]: audit 2024-03-04T15:56:51.277373+0000 mon.c (mon.2) 353 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-23", "mode": "writeback"}]: dispatch 2024-03-04T15:56:53.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:52 smithi151 bash[23756]: cluster 2024-03-04T15:56:51.724261+0000 mon.a (mon.0) 1844 : cluster 0 osdmap e222: 8 total, 8 up, 8 in 2024-03-04T15:56:53.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:52 smithi151 bash[23756]: audit 2024-03-04T15:56:51.742676+0000 mon.a (mon.0) 1845 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-23", "mode": "writeback"}]: dispatch 2024-03-04T15:56:53.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:52 smithi151 bash[23756]: cluster 2024-03-04T15:56:52.268113+0000 mon.a (mon.0) 1846 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:56:53.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:52 smithi151 bash[23756]: audit 2024-03-04T15:56:52.270997+0000 mon.a (mon.0) 1847 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi117-63915-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi117-63915-33"}]': finished 2024-03-04T15:56:53.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:52 smithi151 bash[23756]: audit 2024-03-04T15:56:52.271273+0000 mon.a (mon.0) 1848 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-23", "mode": "writeback"}]': finished 2024-03-04T15:56:53.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:52 smithi151 bash[23756]: cluster 2024-03-04T15:56:52.282341+0000 mon.a (mon.0) 1849 : cluster 0 osdmap e223: 8 total, 8 up, 8 in 2024-03-04T15:56:53.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:52 smithi151 bash[23756]: audit 2024-03-04T15:56:52.297682+0000 mon.c (mon.2) 354 : audit 1 from='client.? 172.21.15.117:0/3854249032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:53.185 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:52 smithi151 bash[23756]: audit 2024-03-04T15:56:52.298201+0000 mon.a (mon.0) 1850 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:54.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:53 smithi117 bash[21480]: cluster 2024-03-04T15:56:52.867965+0000 mgr.y (mgr.24428) 296 : cluster 0 pgmap v341: 300 pgs: 40 unknown, 18 creating+peering, 14 creating+activating, 228 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-04T15:56:54.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:53 smithi117 bash[21480]: audit 2024-03-04T15:56:53.273941+0000 mon.a (mon.0) 1851 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:54.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:53 smithi117 bash[21480]: cluster 2024-03-04T15:56:53.285333+0000 mon.a (mon.0) 1852 : cluster 0 osdmap e224: 8 total, 8 up, 8 in 2024-03-04T15:56:54.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:53 smithi117 bash[21480]: audit 2024-03-04T15:56:53.369197+0000 mon.c (mon.2) 355 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:54.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:53 smithi117 bash[28843]: cluster 2024-03-04T15:56:52.867965+0000 mgr.y (mgr.24428) 296 : cluster 0 pgmap v341: 300 pgs: 40 unknown, 18 creating+peering, 14 creating+activating, 228 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-04T15:56:54.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:53 smithi117 bash[28843]: audit 2024-03-04T15:56:53.273941+0000 mon.a (mon.0) 1851 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:54.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:53 smithi117 bash[28843]: cluster 2024-03-04T15:56:53.285333+0000 mon.a (mon.0) 1852 : cluster 0 osdmap e224: 8 total, 8 up, 8 in 2024-03-04T15:56:54.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:53 smithi117 bash[28843]: audit 2024-03-04T15:56:53.369197+0000 mon.c (mon.2) 355 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:53 smithi151 bash[23756]: cluster 2024-03-04T15:56:52.867965+0000 mgr.y (mgr.24428) 296 : cluster 0 pgmap v341: 300 pgs: 40 unknown, 18 creating+peering, 14 creating+activating, 228 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-04T15:56:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:53 smithi151 bash[23756]: audit 2024-03-04T15:56:53.273941+0000 mon.a (mon.0) 1851 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:53 smithi151 bash[23756]: cluster 2024-03-04T15:56:53.285333+0000 mon.a (mon.0) 1852 : cluster 0 osdmap e224: 8 total, 8 up, 8 in 2024-03-04T15:56:54.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:53 smithi151 bash[23756]: audit 2024-03-04T15:56:53.369197+0000 mon.c (mon.2) 355 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:55.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:54 smithi117 bash[21480]: cluster 2024-03-04T15:56:53.807679+0000 mon.a (mon.0) 1853 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:55.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:54 smithi117 bash[21480]: audit 2024-03-04T15:56:53.808095+0000 mon.a (mon.0) 1854 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:55.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:54 smithi117 bash[21480]: audit 2024-03-04T15:56:54.277971+0000 mon.a (mon.0) 1855 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:56:55.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:54 smithi117 bash[21480]: cluster 2024-03-04T15:56:54.287303+0000 mon.a (mon.0) 1856 : cluster 0 osdmap e225: 8 total, 8 up, 8 in 2024-03-04T15:56:55.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:54 smithi117 bash[21480]: audit 2024-03-04T15:56:54.288034+0000 mon.c (mon.2) 356 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-23"}]: dispatch 2024-03-04T15:56:55.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:54 smithi117 bash[21480]: audit 2024-03-04T15:56:54.290192+0000 mon.c (mon.2) 357 : audit 1 from='client.? 172.21.15.117:0/1462527519' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:55.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:54 smithi117 bash[21480]: audit 2024-03-04T15:56:54.290416+0000 mon.a (mon.0) 1857 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-23"}]: dispatch 2024-03-04T15:56:55.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:54 smithi117 bash[21480]: audit 2024-03-04T15:56:54.290965+0000 mon.c (mon.2) 358 : audit 1 from='client.? 172.21.15.117:0/2845500928' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:55.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:54 smithi117 bash[21480]: audit 2024-03-04T15:56:54.290997+0000 mon.a (mon.0) 1858 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:55.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:54 smithi117 bash[21480]: audit 2024-03-04T15:56:54.296526+0000 mon.a (mon.0) 1859 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:55.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:54 smithi117 bash[28843]: cluster 2024-03-04T15:56:53.807679+0000 mon.a (mon.0) 1853 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:55.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:54 smithi117 bash[28843]: audit 2024-03-04T15:56:53.808095+0000 mon.a (mon.0) 1854 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:55.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:54 smithi117 bash[28843]: audit 2024-03-04T15:56:54.277971+0000 mon.a (mon.0) 1855 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:56:55.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:54 smithi117 bash[28843]: cluster 2024-03-04T15:56:54.287303+0000 mon.a (mon.0) 1856 : cluster 0 osdmap e225: 8 total, 8 up, 8 in 2024-03-04T15:56:55.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:54 smithi117 bash[28843]: audit 2024-03-04T15:56:54.288034+0000 mon.c (mon.2) 356 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-23"}]: dispatch 2024-03-04T15:56:55.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:54 smithi117 bash[28843]: audit 2024-03-04T15:56:54.290192+0000 mon.c (mon.2) 357 : audit 1 from='client.? 172.21.15.117:0/1462527519' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:55.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:54 smithi117 bash[28843]: audit 2024-03-04T15:56:54.290416+0000 mon.a (mon.0) 1857 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-23"}]: dispatch 2024-03-04T15:56:55.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:54 smithi117 bash[28843]: audit 2024-03-04T15:56:54.290965+0000 mon.c (mon.2) 358 : audit 1 from='client.? 172.21.15.117:0/2845500928' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:55.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:54 smithi117 bash[28843]: audit 2024-03-04T15:56:54.290997+0000 mon.a (mon.0) 1858 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:55.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:54 smithi117 bash[28843]: audit 2024-03-04T15:56:54.296526+0000 mon.a (mon.0) 1859 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:55.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:54 smithi151 bash[23756]: cluster 2024-03-04T15:56:53.807679+0000 mon.a (mon.0) 1853 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:56:55.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:54 smithi151 bash[23756]: audit 2024-03-04T15:56:53.808095+0000 mon.a (mon.0) 1854 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:56:55.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:54 smithi151 bash[23756]: audit 2024-03-04T15:56:54.277971+0000 mon.a (mon.0) 1855 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:56:55.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:54 smithi151 bash[23756]: cluster 2024-03-04T15:56:54.287303+0000 mon.a (mon.0) 1856 : cluster 0 osdmap e225: 8 total, 8 up, 8 in 2024-03-04T15:56:55.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:54 smithi151 bash[23756]: audit 2024-03-04T15:56:54.288034+0000 mon.c (mon.2) 356 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-23"}]: dispatch 2024-03-04T15:56:55.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:54 smithi151 bash[23756]: audit 2024-03-04T15:56:54.290192+0000 mon.c (mon.2) 357 : audit 1 from='client.? 172.21.15.117:0/1462527519' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:55.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:54 smithi151 bash[23756]: audit 2024-03-04T15:56:54.290416+0000 mon.a (mon.0) 1857 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-23"}]: dispatch 2024-03-04T15:56:55.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:54 smithi151 bash[23756]: audit 2024-03-04T15:56:54.290965+0000 mon.c (mon.2) 358 : audit 1 from='client.? 172.21.15.117:0/2845500928' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:55.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:54 smithi151 bash[23756]: audit 2024-03-04T15:56:54.290997+0000 mon.a (mon.0) 1858 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:55.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:54 smithi151 bash[23756]: audit 2024-03-04T15:56:54.296526+0000 mon.a (mon.0) 1859 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:56.092 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:55 smithi117 bash[21480]: cluster 2024-03-04T15:56:54.868874+0000 mgr.y (mgr.24428) 297 : cluster 0 pgmap v344: 324 pgs: 64 unknown, 18 creating+peering, 14 creating+activating, 228 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-04T15:56:56.093 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:55 smithi117 bash[21480]: cluster 2024-03-04T15:56:55.278528+0000 mon.a (mon.0) 1860 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:56:56.093 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:55 smithi117 bash[21480]: audit 2024-03-04T15:56:55.282754+0000 mon.a (mon.0) 1861 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-23"}]': finished 2024-03-04T15:56:56.093 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:55 smithi117 bash[21480]: audit 2024-03-04T15:56:55.283017+0000 mon.a (mon.0) 1862 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi117-63915-33"}]': finished 2024-03-04T15:56:56.093 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:55 smithi117 bash[21480]: audit 2024-03-04T15:56:55.283144+0000 mon.a (mon.0) 1863 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:56.093 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:55 smithi117 bash[21480]: cluster 2024-03-04T15:56:55.295614+0000 mon.a (mon.0) 1864 : cluster 0 osdmap e226: 8 total, 8 up, 8 in 2024-03-04T15:56:56.093 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:55 smithi117 bash[21480]: audit 2024-03-04T15:56:55.301993+0000 mon.c (mon.2) 359 : audit 1 from='client.? 172.21.15.117:0/1462527519' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:56.093 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:55 smithi117 bash[21480]: audit 2024-03-04T15:56:55.302700+0000 mon.a (mon.0) 1865 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:56.093 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:55 smithi117 bash[28843]: cluster 2024-03-04T15:56:54.868874+0000 mgr.y (mgr.24428) 297 : cluster 0 pgmap v344: 324 pgs: 64 unknown, 18 creating+peering, 14 creating+activating, 228 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-04T15:56:56.093 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:55 smithi117 bash[28843]: cluster 2024-03-04T15:56:55.278528+0000 mon.a (mon.0) 1860 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:56:56.093 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:55 smithi117 bash[28843]: audit 2024-03-04T15:56:55.282754+0000 mon.a (mon.0) 1861 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-23"}]': finished 2024-03-04T15:56:56.094 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:55 smithi117 bash[28843]: audit 2024-03-04T15:56:55.283017+0000 mon.a (mon.0) 1862 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi117-63915-33"}]': finished 2024-03-04T15:56:56.094 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:55 smithi117 bash[28843]: audit 2024-03-04T15:56:55.283144+0000 mon.a (mon.0) 1863 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:56.094 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:55 smithi117 bash[28843]: cluster 2024-03-04T15:56:55.295614+0000 mon.a (mon.0) 1864 : cluster 0 osdmap e226: 8 total, 8 up, 8 in 2024-03-04T15:56:56.094 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:55 smithi117 bash[28843]: audit 2024-03-04T15:56:55.301993+0000 mon.c (mon.2) 359 : audit 1 from='client.? 172.21.15.117:0/1462527519' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:56.094 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:55 smithi117 bash[28843]: audit 2024-03-04T15:56:55.302700+0000 mon.a (mon.0) 1865 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:55 smithi151 bash[23756]: cluster 2024-03-04T15:56:54.868874+0000 mgr.y (mgr.24428) 297 : cluster 0 pgmap v344: 324 pgs: 64 unknown, 18 creating+peering, 14 creating+activating, 228 active+clean; 584 KiB data, 592 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-04T15:56:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:55 smithi151 bash[23756]: cluster 2024-03-04T15:56:55.278528+0000 mon.a (mon.0) 1860 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:56:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:55 smithi151 bash[23756]: audit 2024-03-04T15:56:55.282754+0000 mon.a (mon.0) 1861 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-23"}]': finished 2024-03-04T15:56:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:55 smithi151 bash[23756]: audit 2024-03-04T15:56:55.283017+0000 mon.a (mon.0) 1862 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi117-63915-33"}]': finished 2024-03-04T15:56:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:55 smithi151 bash[23756]: audit 2024-03-04T15:56:55.283144+0000 mon.a (mon.0) 1863 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:55 smithi151 bash[23756]: cluster 2024-03-04T15:56:55.295614+0000 mon.a (mon.0) 1864 : cluster 0 osdmap e226: 8 total, 8 up, 8 in 2024-03-04T15:56:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:55 smithi151 bash[23756]: audit 2024-03-04T15:56:55.301993+0000 mon.c (mon.2) 359 : audit 1 from='client.? 172.21.15.117:0/1462527519' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:56.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:55 smithi151 bash[23756]: audit 2024-03-04T15:56:55.302700+0000 mon.a (mon.0) 1865 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi117-63915-33"}]: dispatch 2024-03-04T15:56:56.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:56:56 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:56:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:56:56.932 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:56:56 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:56:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:57 smithi151 bash[23756]: audit 2024-03-04T15:56:56.287232+0000 mon.a (mon.0) 1866 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi117-63915-33"}]': finished 2024-03-04T15:56:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:57 smithi151 bash[23756]: cluster 2024-03-04T15:56:56.296137+0000 mon.a (mon.0) 1867 : cluster 0 osdmap e227: 8 total, 8 up, 8 in 2024-03-04T15:56:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:57 smithi151 bash[23756]: audit 2024-03-04T15:56:56.303303+0000 mon.a (mon.0) 1868 : audit 1 from='client.? 172.21.15.117:0/2404752113' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:57 smithi151 bash[23756]: audit 2024-03-04T15:56:56.344284+0000 mon.c (mon.2) 360 : audit 1 from='client.? 172.21.15.117:0/3737331369' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:56:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:57 smithi151 bash[23756]: audit 2024-03-04T15:56:56.601186+0000 mgr.y (mgr.24428) 298 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:56:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:57 smithi151 bash[23756]: audit 2024-03-04T15:56:56.719866+0000 mon.a (mon.0) 1869 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:56:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:57 smithi151 bash[23756]: audit 2024-03-04T15:56:56.720785+0000 mon.c (mon.2) 361 : audit 1 from='client.? 172.21.15.117:0/3737331369' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:56:57.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:57 smithi151 bash[23756]: audit 2024-03-04T15:56:56.721205+0000 mon.a (mon.0) 1870 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:56:57.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:57 smithi151 bash[23756]: audit 2024-03-04T15:56:56.721962+0000 mon.c (mon.2) 362 : audit 1 from='client.? 172.21.15.117:0/3737331369' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi117-63915-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:57.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:57 smithi151 bash[23756]: audit 2024-03-04T15:56:56.722517+0000 mon.a (mon.0) 1871 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi117-63915-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:57.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:57 smithi151 bash[23756]: cluster 2024-03-04T15:56:56.871624+0000 mgr.y (mgr.24428) 299 : cluster 0 pgmap v347: 324 pgs: 32 creating+peering, 292 active+clean; 584 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T15:56:57.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[21480]: audit 2024-03-04T15:56:56.287232+0000 mon.a (mon.0) 1866 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi117-63915-33"}]': finished 2024-03-04T15:56:57.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[21480]: cluster 2024-03-04T15:56:56.296137+0000 mon.a (mon.0) 1867 : cluster 0 osdmap e227: 8 total, 8 up, 8 in 2024-03-04T15:56:57.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[21480]: audit 2024-03-04T15:56:56.303303+0000 mon.a (mon.0) 1868 : audit 1 from='client.? 172.21.15.117:0/2404752113' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:57.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[21480]: audit 2024-03-04T15:56:56.344284+0000 mon.c (mon.2) 360 : audit 1 from='client.? 172.21.15.117:0/3737331369' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:56:57.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[21480]: audit 2024-03-04T15:56:56.601186+0000 mgr.y (mgr.24428) 298 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:56:57.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[21480]: audit 2024-03-04T15:56:56.719866+0000 mon.a (mon.0) 1869 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:56:57.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[21480]: audit 2024-03-04T15:56:56.720785+0000 mon.c (mon.2) 361 : audit 1 from='client.? 172.21.15.117:0/3737331369' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:56:57.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[21480]: audit 2024-03-04T15:56:56.721205+0000 mon.a (mon.0) 1870 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:56:57.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[21480]: audit 2024-03-04T15:56:56.721962+0000 mon.c (mon.2) 362 : audit 1 from='client.? 172.21.15.117:0/3737331369' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi117-63915-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:57.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[21480]: audit 2024-03-04T15:56:56.722517+0000 mon.a (mon.0) 1871 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi117-63915-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:57.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[21480]: cluster 2024-03-04T15:56:56.871624+0000 mgr.y (mgr.24428) 299 : cluster 0 pgmap v347: 324 pgs: 32 creating+peering, 292 active+clean; 584 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T15:56:57.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[28843]: audit 2024-03-04T15:56:56.287232+0000 mon.a (mon.0) 1866 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi117-63915-33"}]': finished 2024-03-04T15:56:57.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[28843]: cluster 2024-03-04T15:56:56.296137+0000 mon.a (mon.0) 1867 : cluster 0 osdmap e227: 8 total, 8 up, 8 in 2024-03-04T15:56:57.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[28843]: audit 2024-03-04T15:56:56.303303+0000 mon.a (mon.0) 1868 : audit 1 from='client.? 172.21.15.117:0/2404752113' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:57.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[28843]: audit 2024-03-04T15:56:56.344284+0000 mon.c (mon.2) 360 : audit 1 from='client.? 172.21.15.117:0/3737331369' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:56:57.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[28843]: audit 2024-03-04T15:56:56.601186+0000 mgr.y (mgr.24428) 298 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:56:57.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[28843]: audit 2024-03-04T15:56:56.719866+0000 mon.a (mon.0) 1869 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:56:57.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[28843]: audit 2024-03-04T15:56:56.720785+0000 mon.c (mon.2) 361 : audit 1 from='client.? 172.21.15.117:0/3737331369' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:56:57.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[28843]: audit 2024-03-04T15:56:56.721205+0000 mon.a (mon.0) 1870 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:56:57.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[28843]: audit 2024-03-04T15:56:56.721962+0000 mon.c (mon.2) 362 : audit 1 from='client.? 172.21.15.117:0/3737331369' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi117-63915-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:57.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[28843]: audit 2024-03-04T15:56:56.722517+0000 mon.a (mon.0) 1871 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi117-63915-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:56:57.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:57 smithi117 bash[28843]: cluster 2024-03-04T15:56:56.871624+0000 mgr.y (mgr.24428) 299 : cluster 0 pgmap v347: 324 pgs: 32 creating+peering, 292 active+clean; 584 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T15:56:58.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:58 smithi117 bash[21480]: audit 2024-03-04T15:56:57.291754+0000 mon.a (mon.0) 1872 : audit 1 from='client.? 172.21.15.117:0/2404752113' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:58.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:58 smithi117 bash[21480]: audit 2024-03-04T15:56:57.292017+0000 mon.a (mon.0) 1873 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi117-63915-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:58.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:58 smithi117 bash[21480]: cluster 2024-03-04T15:56:57.304158+0000 mon.a (mon.0) 1874 : cluster 0 osdmap e228: 8 total, 8 up, 8 in 2024-03-04T15:56:58.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:58 smithi117 bash[21480]: audit 2024-03-04T15:56:57.311450+0000 mon.c (mon.2) 363 : audit 1 from='client.? 172.21.15.117:0/3737331369' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi117-63915-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:56:58.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:58 smithi117 bash[21480]: audit 2024-03-04T15:56:57.323581+0000 mon.c (mon.2) 364 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:58.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:58 smithi117 bash[21480]: audit 2024-03-04T15:56:57.330242+0000 mon.a (mon.0) 1875 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi117-63915-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:56:58.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:58 smithi117 bash[21480]: audit 2024-03-04T15:56:57.355295+0000 mon.a (mon.0) 1876 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:58.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:58 smithi117 bash[21480]: audit 2024-03-04T15:56:58.078864+0000 mon.a (mon.0) 1877 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:58.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:58 smithi117 bash[28843]: audit 2024-03-04T15:56:57.291754+0000 mon.a (mon.0) 1872 : audit 1 from='client.? 172.21.15.117:0/2404752113' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:58.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:58 smithi117 bash[28843]: audit 2024-03-04T15:56:57.292017+0000 mon.a (mon.0) 1873 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi117-63915-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:58.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:58 smithi117 bash[28843]: cluster 2024-03-04T15:56:57.304158+0000 mon.a (mon.0) 1874 : cluster 0 osdmap e228: 8 total, 8 up, 8 in 2024-03-04T15:56:58.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:58 smithi117 bash[28843]: audit 2024-03-04T15:56:57.311450+0000 mon.c (mon.2) 363 : audit 1 from='client.? 172.21.15.117:0/3737331369' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi117-63915-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:56:58.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:58 smithi117 bash[28843]: audit 2024-03-04T15:56:57.323581+0000 mon.c (mon.2) 364 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:58.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:58 smithi117 bash[28843]: audit 2024-03-04T15:56:57.330242+0000 mon.a (mon.0) 1875 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi117-63915-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:56:58.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:58 smithi117 bash[28843]: audit 2024-03-04T15:56:57.355295+0000 mon.a (mon.0) 1876 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:58.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:58 smithi117 bash[28843]: audit 2024-03-04T15:56:58.078864+0000 mon.a (mon.0) 1877 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:58.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:58 smithi151 bash[23756]: audit 2024-03-04T15:56:57.291754+0000 mon.a (mon.0) 1872 : audit 1 from='client.? 172.21.15.117:0/2404752113' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:58.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:58 smithi151 bash[23756]: audit 2024-03-04T15:56:57.292017+0000 mon.a (mon.0) 1873 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi117-63915-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:56:58.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:58 smithi151 bash[23756]: cluster 2024-03-04T15:56:57.304158+0000 mon.a (mon.0) 1874 : cluster 0 osdmap e228: 8 total, 8 up, 8 in 2024-03-04T15:56:58.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:58 smithi151 bash[23756]: audit 2024-03-04T15:56:57.311450+0000 mon.c (mon.2) 363 : audit 1 from='client.? 172.21.15.117:0/3737331369' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi117-63915-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:56:58.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:58 smithi151 bash[23756]: audit 2024-03-04T15:56:57.323581+0000 mon.c (mon.2) 364 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:58.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:58 smithi151 bash[23756]: audit 2024-03-04T15:56:57.330242+0000 mon.a (mon.0) 1875 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi117-63915-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:56:58.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:58 smithi151 bash[23756]: audit 2024-03-04T15:56:57.355295+0000 mon.a (mon.0) 1876 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:58.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:58 smithi151 bash[23756]: audit 2024-03-04T15:56:58.078864+0000 mon.a (mon.0) 1877 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:59.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[21480]: cluster 2024-03-04T15:56:58.544585+0000 mon.a (mon.0) 1878 : cluster 0 osdmap e229: 8 total, 8 up, 8 in 2024-03-04T15:56:59.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[21480]: audit 2024-03-04T15:56:58.546047+0000 mon.c (mon.2) 365 : audit 1 from='client.? 172.21.15.117:0/3389123751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:59.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[21480]: audit 2024-03-04T15:56:58.561370+0000 mon.a (mon.0) 1879 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:59.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[21480]: audit 2024-03-04T15:56:58.563789+0000 mon.a (mon.0) 1880 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:56:59.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[21480]: audit 2024-03-04T15:56:58.567847+0000 mon.c (mon.2) 366 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:56:59.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[21480]: cluster 2024-03-04T15:56:58.872610+0000 mgr.y (mgr.24428) 300 : cluster 0 pgmap v350: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T15:56:59.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[21480]: audit 2024-03-04T15:56:59.084494+0000 mon.a (mon.0) 1881 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi117-63915-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi117-63915-34"}]': finished 2024-03-04T15:56:59.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[21480]: audit 2024-03-04T15:56:59.084710+0000 mon.a (mon.0) 1882 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:59.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[21480]: cluster 2024-03-04T15:56:59.095150+0000 mon.a (mon.0) 1883 : cluster 0 osdmap e230: 8 total, 8 up, 8 in 2024-03-04T15:56:59.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[21480]: audit 2024-03-04T15:56:59.136812+0000 mon.c (mon.2) 367 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:59.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[21480]: audit 2024-03-04T15:56:59.137245+0000 mon.a (mon.0) 1884 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:59.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[28843]: cluster 2024-03-04T15:56:58.544585+0000 mon.a (mon.0) 1878 : cluster 0 osdmap e229: 8 total, 8 up, 8 in 2024-03-04T15:56:59.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[28843]: audit 2024-03-04T15:56:58.546047+0000 mon.c (mon.2) 365 : audit 1 from='client.? 172.21.15.117:0/3389123751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:59.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[28843]: audit 2024-03-04T15:56:58.561370+0000 mon.a (mon.0) 1879 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:59.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[28843]: audit 2024-03-04T15:56:58.563789+0000 mon.a (mon.0) 1880 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:56:59.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[28843]: audit 2024-03-04T15:56:58.567847+0000 mon.c (mon.2) 366 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:56:59.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[28843]: cluster 2024-03-04T15:56:58.872610+0000 mgr.y (mgr.24428) 300 : cluster 0 pgmap v350: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T15:56:59.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[28843]: audit 2024-03-04T15:56:59.084494+0000 mon.a (mon.0) 1881 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi117-63915-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi117-63915-34"}]': finished 2024-03-04T15:56:59.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[28843]: audit 2024-03-04T15:56:59.084710+0000 mon.a (mon.0) 1882 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:59.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[28843]: cluster 2024-03-04T15:56:59.095150+0000 mon.a (mon.0) 1883 : cluster 0 osdmap e230: 8 total, 8 up, 8 in 2024-03-04T15:56:59.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[28843]: audit 2024-03-04T15:56:59.136812+0000 mon.c (mon.2) 367 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:59.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:56:59 smithi117 bash[28843]: audit 2024-03-04T15:56:59.137245+0000 mon.a (mon.0) 1884 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:59.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:59 smithi151 bash[23756]: cluster 2024-03-04T15:56:58.544585+0000 mon.a (mon.0) 1878 : cluster 0 osdmap e229: 8 total, 8 up, 8 in 2024-03-04T15:56:59.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:59 smithi151 bash[23756]: audit 2024-03-04T15:56:58.546047+0000 mon.c (mon.2) 365 : audit 1 from='client.? 172.21.15.117:0/3389123751' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:59.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:59 smithi151 bash[23756]: audit 2024-03-04T15:56:58.561370+0000 mon.a (mon.0) 1879 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:56:59.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:59 smithi151 bash[23756]: audit 2024-03-04T15:56:58.563789+0000 mon.a (mon.0) 1880 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:56:59.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:59 smithi151 bash[23756]: audit 2024-03-04T15:56:58.567847+0000 mon.c (mon.2) 366 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:56:59.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:59 smithi151 bash[23756]: cluster 2024-03-04T15:56:58.872610+0000 mgr.y (mgr.24428) 300 : cluster 0 pgmap v350: 388 pgs: 64 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 593 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T15:56:59.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:59 smithi151 bash[23756]: audit 2024-03-04T15:56:59.084494+0000 mon.a (mon.0) 1881 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi117-63915-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi117-63915-34"}]': finished 2024-03-04T15:56:59.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:59 smithi151 bash[23756]: audit 2024-03-04T15:56:59.084710+0000 mon.a (mon.0) 1882 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi117-63923-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:56:59.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:59 smithi151 bash[23756]: cluster 2024-03-04T15:56:59.095150+0000 mon.a (mon.0) 1883 : cluster 0 osdmap e230: 8 total, 8 up, 8 in 2024-03-04T15:56:59.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:59 smithi151 bash[23756]: audit 2024-03-04T15:56:59.136812+0000 mon.c (mon.2) 367 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:56:59.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:56:59 smithi151 bash[23756]: audit 2024-03-04T15:56:59.137245+0000 mon.a (mon.0) 1884 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:01.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:01 smithi151 bash[23756]: audit 2024-03-04T15:57:00.087744+0000 mon.a (mon.0) 1885 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-25", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:57:01.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:01 smithi151 bash[23756]: audit 2024-03-04T15:57:00.095948+0000 mon.c (mon.2) 368 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-25"}]: dispatch 2024-03-04T15:57:01.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:01 smithi151 bash[23756]: cluster 2024-03-04T15:57:00.098342+0000 mon.a (mon.0) 1886 : cluster 0 osdmap e231: 8 total, 8 up, 8 in 2024-03-04T15:57:01.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:01 smithi151 bash[23756]: audit 2024-03-04T15:57:00.103188+0000 mon.a (mon.0) 1887 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-25"}]: dispatch 2024-03-04T15:57:01.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:01 smithi117 bash[21480]: audit 2024-03-04T15:57:00.087744+0000 mon.a (mon.0) 1885 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-25", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:57:01.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:01 smithi117 bash[21480]: audit 2024-03-04T15:57:00.095948+0000 mon.c (mon.2) 368 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-25"}]: dispatch 2024-03-04T15:57:01.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:01 smithi117 bash[21480]: cluster 2024-03-04T15:57:00.098342+0000 mon.a (mon.0) 1886 : cluster 0 osdmap e231: 8 total, 8 up, 8 in 2024-03-04T15:57:01.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:01 smithi117 bash[21480]: audit 2024-03-04T15:57:00.103188+0000 mon.a (mon.0) 1887 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-25"}]: dispatch 2024-03-04T15:57:01.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:01 smithi117 bash[28843]: audit 2024-03-04T15:57:00.087744+0000 mon.a (mon.0) 1885 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-25", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:57:01.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:01 smithi117 bash[28843]: audit 2024-03-04T15:57:00.095948+0000 mon.c (mon.2) 368 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-25"}]: dispatch 2024-03-04T15:57:01.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:01 smithi117 bash[28843]: cluster 2024-03-04T15:57:00.098342+0000 mon.a (mon.0) 1886 : cluster 0 osdmap e231: 8 total, 8 up, 8 in 2024-03-04T15:57:01.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:01 smithi117 bash[28843]: audit 2024-03-04T15:57:00.103188+0000 mon.a (mon.0) 1887 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-25"}]: dispatch 2024-03-04T15:57:02.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:02 smithi151 bash[23756]: cluster 2024-03-04T15:57:00.873587+0000 mgr.y (mgr.24428) 301 : cluster 0 pgmap v353: 364 pgs: 40 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:57:02.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:02 smithi151 bash[23756]: audit 2024-03-04T15:57:01.090619+0000 mon.a (mon.0) 1888 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-25"}]': finished 2024-03-04T15:57:02.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:02 smithi151 bash[23756]: cluster 2024-03-04T15:57:01.098342+0000 mon.a (mon.0) 1889 : cluster 0 osdmap e232: 8 total, 8 up, 8 in 2024-03-04T15:57:02.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:02 smithi151 bash[23756]: audit 2024-03-04T15:57:01.100630+0000 mon.c (mon.2) 369 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-25", "mode": "writeback"}]: dispatch 2024-03-04T15:57:02.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:02 smithi151 bash[23756]: audit 2024-03-04T15:57:01.101897+0000 mon.c (mon.2) 370 : audit 1 from='client.? 172.21.15.117:0/3737331369' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:57:02.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:02 smithi151 bash[23756]: audit 2024-03-04T15:57:01.102815+0000 mon.a (mon.0) 1890 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-25", "mode": "writeback"}]: dispatch 2024-03-04T15:57:02.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:02 smithi151 bash[23756]: audit 2024-03-04T15:57:01.115274+0000 mon.a (mon.0) 1891 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:57:02.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:02 smithi151 bash[23756]: cluster 2024-03-04T15:57:02.090952+0000 mon.a (mon.0) 1892 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:57:02.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:02 smithi151 bash[23756]: audit 2024-03-04T15:57:02.093619+0000 mon.a (mon.0) 1893 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-25", "mode": "writeback"}]': finished 2024-03-04T15:57:02.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:02 smithi151 bash[23756]: audit 2024-03-04T15:57:02.093819+0000 mon.a (mon.0) 1894 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi117-63915-34"}]': finished 2024-03-04T15:57:02.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:02 smithi151 bash[23756]: cluster 2024-03-04T15:57:02.101379+0000 mon.a (mon.0) 1895 : cluster 0 osdmap e233: 8 total, 8 up, 8 in 2024-03-04T15:57:02.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[21480]: cluster 2024-03-04T15:57:00.873587+0000 mgr.y (mgr.24428) 301 : cluster 0 pgmap v353: 364 pgs: 40 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:57:02.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[21480]: audit 2024-03-04T15:57:01.090619+0000 mon.a (mon.0) 1888 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-25"}]': finished 2024-03-04T15:57:02.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[21480]: cluster 2024-03-04T15:57:01.098342+0000 mon.a (mon.0) 1889 : cluster 0 osdmap e232: 8 total, 8 up, 8 in 2024-03-04T15:57:02.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[21480]: audit 2024-03-04T15:57:01.100630+0000 mon.c (mon.2) 369 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-25", "mode": "writeback"}]: dispatch 2024-03-04T15:57:02.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[21480]: audit 2024-03-04T15:57:01.101897+0000 mon.c (mon.2) 370 : audit 1 from='client.? 172.21.15.117:0/3737331369' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:57:02.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[21480]: audit 2024-03-04T15:57:01.102815+0000 mon.a (mon.0) 1890 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-25", "mode": "writeback"}]: dispatch 2024-03-04T15:57:02.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[21480]: audit 2024-03-04T15:57:01.115274+0000 mon.a (mon.0) 1891 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:57:02.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[21480]: cluster 2024-03-04T15:57:02.090952+0000 mon.a (mon.0) 1892 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:57:02.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[21480]: audit 2024-03-04T15:57:02.093619+0000 mon.a (mon.0) 1893 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-25", "mode": "writeback"}]': finished 2024-03-04T15:57:02.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[21480]: audit 2024-03-04T15:57:02.093819+0000 mon.a (mon.0) 1894 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi117-63915-34"}]': finished 2024-03-04T15:57:02.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[21480]: cluster 2024-03-04T15:57:02.101379+0000 mon.a (mon.0) 1895 : cluster 0 osdmap e233: 8 total, 8 up, 8 in 2024-03-04T15:57:02.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[28843]: cluster 2024-03-04T15:57:00.873587+0000 mgr.y (mgr.24428) 301 : cluster 0 pgmap v353: 364 pgs: 40 unknown, 32 creating+peering, 292 active+clean; 584 KiB data, 593 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:57:02.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[28843]: audit 2024-03-04T15:57:01.090619+0000 mon.a (mon.0) 1888 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-25"}]': finished 2024-03-04T15:57:02.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[28843]: cluster 2024-03-04T15:57:01.098342+0000 mon.a (mon.0) 1889 : cluster 0 osdmap e232: 8 total, 8 up, 8 in 2024-03-04T15:57:02.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[28843]: audit 2024-03-04T15:57:01.100630+0000 mon.c (mon.2) 369 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-25", "mode": "writeback"}]: dispatch 2024-03-04T15:57:02.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[28843]: audit 2024-03-04T15:57:01.101897+0000 mon.c (mon.2) 370 : audit 1 from='client.? 172.21.15.117:0/3737331369' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:57:02.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[28843]: audit 2024-03-04T15:57:01.102815+0000 mon.a (mon.0) 1890 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-25", "mode": "writeback"}]: dispatch 2024-03-04T15:57:02.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[28843]: audit 2024-03-04T15:57:01.115274+0000 mon.a (mon.0) 1891 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:57:02.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[28843]: cluster 2024-03-04T15:57:02.090952+0000 mon.a (mon.0) 1892 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:57:02.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[28843]: audit 2024-03-04T15:57:02.093619+0000 mon.a (mon.0) 1893 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-25", "mode": "writeback"}]': finished 2024-03-04T15:57:02.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[28843]: audit 2024-03-04T15:57:02.093819+0000 mon.a (mon.0) 1894 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi117-63915-34"}]': finished 2024-03-04T15:57:02.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:02 smithi117 bash[28843]: cluster 2024-03-04T15:57:02.101379+0000 mon.a (mon.0) 1895 : cluster 0 osdmap e233: 8 total, 8 up, 8 in 2024-03-04T15:57:03.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:03 smithi151 bash[23756]: audit 2024-03-04T15:57:02.107175+0000 mon.c (mon.2) 371 : audit 1 from='client.? 172.21.15.117:0/3737331369' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:57:03.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:03 smithi151 bash[23756]: audit 2024-03-04T15:57:02.122335+0000 mon.a (mon.0) 1896 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:57:03.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:03 smithi151 bash[23756]: audit 2024-03-04T15:57:02.221475+0000 mon.c (mon.2) 372 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:03.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:03 smithi151 bash[23756]: audit 2024-03-04T15:57:02.222168+0000 mon.a (mon.0) 1897 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:03.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:03 smithi151 bash[23756]: cluster 2024-03-04T15:57:03.077937+0000 mon.a (mon.0) 1898 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:03.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:03 smithi117 bash[21480]: audit 2024-03-04T15:57:02.107175+0000 mon.c (mon.2) 371 : audit 1 from='client.? 172.21.15.117:0/3737331369' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:57:03.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:03 smithi117 bash[21480]: audit 2024-03-04T15:57:02.122335+0000 mon.a (mon.0) 1896 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:57:03.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:03 smithi117 bash[21480]: audit 2024-03-04T15:57:02.221475+0000 mon.c (mon.2) 372 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:03.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:03 smithi117 bash[21480]: audit 2024-03-04T15:57:02.222168+0000 mon.a (mon.0) 1897 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:03.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:03 smithi117 bash[21480]: cluster 2024-03-04T15:57:03.077937+0000 mon.a (mon.0) 1898 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:03.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:03 smithi117 bash[28843]: audit 2024-03-04T15:57:02.107175+0000 mon.c (mon.2) 371 : audit 1 from='client.? 172.21.15.117:0/3737331369' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:57:03.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:03 smithi117 bash[28843]: audit 2024-03-04T15:57:02.122335+0000 mon.a (mon.0) 1896 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi117-63915-34"}]: dispatch 2024-03-04T15:57:03.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:03 smithi117 bash[28843]: audit 2024-03-04T15:57:02.221475+0000 mon.c (mon.2) 372 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:03.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:03 smithi117 bash[28843]: audit 2024-03-04T15:57:02.222168+0000 mon.a (mon.0) 1897 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:03.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:03 smithi117 bash[28843]: cluster 2024-03-04T15:57:03.077937+0000 mon.a (mon.0) 1898 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:04 smithi151 bash[23756]: cluster 2024-03-04T15:57:02.874599+0000 mgr.y (mgr.24428) 302 : cluster 0 pgmap v356: 292 pgs: 292 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T15:57:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:04 smithi151 bash[23756]: audit 2024-03-04T15:57:03.122871+0000 mon.a (mon.0) 1899 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi117-63915-34"}]': finished 2024-03-04T15:57:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:04 smithi151 bash[23756]: audit 2024-03-04T15:57:03.123061+0000 mon.a (mon.0) 1900 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:04.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:04 smithi151 bash[23756]: cluster 2024-03-04T15:57:03.131186+0000 mon.a (mon.0) 1901 : cluster 0 osdmap e234: 8 total, 8 up, 8 in 2024-03-04T15:57:04.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:04 smithi151 bash[23756]: audit 2024-03-04T15:57:03.136283+0000 mon.c (mon.2) 373 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-25"}]: dispatch 2024-03-04T15:57:04.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:04 smithi151 bash[23756]: audit 2024-03-04T15:57:03.138064+0000 mon.a (mon.0) 1902 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-25"}]: dispatch 2024-03-04T15:57:04.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:04 smithi151 bash[23756]: audit 2024-03-04T15:57:03.165457+0000 mon.a (mon.0) 1903 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi117-63915-35"}]: dispatch 2024-03-04T15:57:04.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:04 smithi151 bash[23756]: audit 2024-03-04T15:57:03.166524+0000 mon.a (mon.0) 1904 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi117-63915-35"}]: dispatch 2024-03-04T15:57:04.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:04 smithi151 bash[23756]: audit 2024-03-04T15:57:03.166897+0000 mon.a (mon.0) 1905 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi117-63915-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:04.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:04 smithi117 bash[28843]: cluster 2024-03-04T15:57:02.874599+0000 mgr.y (mgr.24428) 302 : cluster 0 pgmap v356: 292 pgs: 292 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T15:57:04.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:04 smithi117 bash[28843]: audit 2024-03-04T15:57:03.122871+0000 mon.a (mon.0) 1899 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi117-63915-34"}]': finished 2024-03-04T15:57:04.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:04 smithi117 bash[28843]: audit 2024-03-04T15:57:03.123061+0000 mon.a (mon.0) 1900 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:04.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:04 smithi117 bash[28843]: cluster 2024-03-04T15:57:03.131186+0000 mon.a (mon.0) 1901 : cluster 0 osdmap e234: 8 total, 8 up, 8 in 2024-03-04T15:57:04.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:04 smithi117 bash[28843]: audit 2024-03-04T15:57:03.136283+0000 mon.c (mon.2) 373 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-25"}]: dispatch 2024-03-04T15:57:04.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:04 smithi117 bash[28843]: audit 2024-03-04T15:57:03.138064+0000 mon.a (mon.0) 1902 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-25"}]: dispatch 2024-03-04T15:57:04.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:04 smithi117 bash[28843]: audit 2024-03-04T15:57:03.165457+0000 mon.a (mon.0) 1903 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi117-63915-35"}]: dispatch 2024-03-04T15:57:04.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:04 smithi117 bash[28843]: audit 2024-03-04T15:57:03.166524+0000 mon.a (mon.0) 1904 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi117-63915-35"}]: dispatch 2024-03-04T15:57:04.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:04 smithi117 bash[28843]: audit 2024-03-04T15:57:03.166897+0000 mon.a (mon.0) 1905 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi117-63915-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:04.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:04 smithi117 bash[21480]: cluster 2024-03-04T15:57:02.874599+0000 mgr.y (mgr.24428) 302 : cluster 0 pgmap v356: 292 pgs: 292 active+clean; 584 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T15:57:04.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:04 smithi117 bash[21480]: audit 2024-03-04T15:57:03.122871+0000 mon.a (mon.0) 1899 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi117-63915-34"}]': finished 2024-03-04T15:57:04.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:04 smithi117 bash[21480]: audit 2024-03-04T15:57:03.123061+0000 mon.a (mon.0) 1900 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:04.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:04 smithi117 bash[21480]: cluster 2024-03-04T15:57:03.131186+0000 mon.a (mon.0) 1901 : cluster 0 osdmap e234: 8 total, 8 up, 8 in 2024-03-04T15:57:04.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:04 smithi117 bash[21480]: audit 2024-03-04T15:57:03.136283+0000 mon.c (mon.2) 373 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-25"}]: dispatch 2024-03-04T15:57:04.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:04 smithi117 bash[21480]: audit 2024-03-04T15:57:03.138064+0000 mon.a (mon.0) 1902 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-25"}]: dispatch 2024-03-04T15:57:04.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:04 smithi117 bash[21480]: audit 2024-03-04T15:57:03.165457+0000 mon.a (mon.0) 1903 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi117-63915-35"}]: dispatch 2024-03-04T15:57:04.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:04 smithi117 bash[21480]: audit 2024-03-04T15:57:03.166524+0000 mon.a (mon.0) 1904 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi117-63915-35"}]: dispatch 2024-03-04T15:57:04.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:04 smithi117 bash[21480]: audit 2024-03-04T15:57:03.166897+0000 mon.a (mon.0) 1905 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi117-63915-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:06.318 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:57:06 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:57:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:57:06.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:06 smithi117 bash[21480]: cluster 2024-03-04T15:57:04.123235+0000 mon.a (mon.0) 1906 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:57:06.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:06 smithi117 bash[21480]: audit 2024-03-04T15:57:04.125630+0000 mon.a (mon.0) 1907 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-25"}]': finished 2024-03-04T15:57:06.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:06 smithi117 bash[21480]: audit 2024-03-04T15:57:04.125787+0000 mon.a (mon.0) 1908 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi117-63915-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:06.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:06 smithi117 bash[21480]: cluster 2024-03-04T15:57:04.133138+0000 mon.a (mon.0) 1909 : cluster 0 osdmap e235: 8 total, 8 up, 8 in 2024-03-04T15:57:06.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:06 smithi117 bash[21480]: audit 2024-03-04T15:57:04.136173+0000 mon.a (mon.0) 1910 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi117-63915-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi117-63915-35"}]: dispatch 2024-03-04T15:57:06.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:06 smithi117 bash[28843]: cluster 2024-03-04T15:57:04.123235+0000 mon.a (mon.0) 1906 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:57:06.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:06 smithi117 bash[28843]: audit 2024-03-04T15:57:04.125630+0000 mon.a (mon.0) 1907 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-25"}]': finished 2024-03-04T15:57:06.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:06 smithi117 bash[28843]: audit 2024-03-04T15:57:04.125787+0000 mon.a (mon.0) 1908 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi117-63915-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:06.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:06 smithi117 bash[28843]: cluster 2024-03-04T15:57:04.133138+0000 mon.a (mon.0) 1909 : cluster 0 osdmap e235: 8 total, 8 up, 8 in 2024-03-04T15:57:06.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:06 smithi117 bash[28843]: audit 2024-03-04T15:57:04.136173+0000 mon.a (mon.0) 1910 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi117-63915-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi117-63915-35"}]: dispatch 2024-03-04T15:57:06.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:06 smithi151 bash[23756]: cluster 2024-03-04T15:57:04.123235+0000 mon.a (mon.0) 1906 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:57:06.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:06 smithi151 bash[23756]: audit 2024-03-04T15:57:04.125630+0000 mon.a (mon.0) 1907 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-25"}]': finished 2024-03-04T15:57:06.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:06 smithi151 bash[23756]: audit 2024-03-04T15:57:04.125787+0000 mon.a (mon.0) 1908 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi117-63915-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:06.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:06 smithi151 bash[23756]: cluster 2024-03-04T15:57:04.133138+0000 mon.a (mon.0) 1909 : cluster 0 osdmap e235: 8 total, 8 up, 8 in 2024-03-04T15:57:06.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:06 smithi151 bash[23756]: audit 2024-03-04T15:57:04.136173+0000 mon.a (mon.0) 1910 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi117-63915-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi117-63915-35"}]: dispatch 2024-03-04T15:57:06.931 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:57:06 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:57:07.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:07 smithi117 bash[21480]: cluster 2024-03-04T15:57:04.875409+0000 mgr.y (mgr.24428) 303 : cluster 0 pgmap v359: 228 pgs: 228 active+clean; 583 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:57:07.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:07 smithi117 bash[21480]: cluster 2024-03-04T15:57:05.591771+0000 mon.a (mon.0) 1911 : cluster 0 osdmap e236: 8 total, 8 up, 8 in 2024-03-04T15:57:07.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:07 smithi117 bash[21480]: audit 2024-03-04T15:57:05.594568+0000 mon.a (mon.0) 1912 : audit 1 from='client.? 172.21.15.117:0/1288012769' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi117-63923-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:07.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:07 smithi117 bash[21480]: audit 2024-03-04T15:57:06.135067+0000 mon.a (mon.0) 1913 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi117-63915-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi117-63915-35"}]': finished 2024-03-04T15:57:07.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:07 smithi117 bash[21480]: audit 2024-03-04T15:57:06.135265+0000 mon.a (mon.0) 1914 : audit 1 from='client.? 172.21.15.117:0/1288012769' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi117-63923-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:07.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:07 smithi117 bash[21480]: cluster 2024-03-04T15:57:06.139884+0000 mon.a (mon.0) 1915 : cluster 0 osdmap e237: 8 total, 8 up, 8 in 2024-03-04T15:57:07.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:07 smithi117 bash[21480]: audit 2024-03-04T15:57:06.154203+0000 mon.c (mon.2) 374 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:07.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:07 smithi117 bash[21480]: audit 2024-03-04T15:57:06.174141+0000 mon.a (mon.0) 1916 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:07.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:07 smithi117 bash[28843]: cluster 2024-03-04T15:57:04.875409+0000 mgr.y (mgr.24428) 303 : cluster 0 pgmap v359: 228 pgs: 228 active+clean; 583 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:57:07.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:07 smithi117 bash[28843]: cluster 2024-03-04T15:57:05.591771+0000 mon.a (mon.0) 1911 : cluster 0 osdmap e236: 8 total, 8 up, 8 in 2024-03-04T15:57:07.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:07 smithi117 bash[28843]: audit 2024-03-04T15:57:05.594568+0000 mon.a (mon.0) 1912 : audit 1 from='client.? 172.21.15.117:0/1288012769' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi117-63923-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:07.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:07 smithi117 bash[28843]: audit 2024-03-04T15:57:06.135067+0000 mon.a (mon.0) 1913 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi117-63915-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi117-63915-35"}]': finished 2024-03-04T15:57:07.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:07 smithi117 bash[28843]: audit 2024-03-04T15:57:06.135265+0000 mon.a (mon.0) 1914 : audit 1 from='client.? 172.21.15.117:0/1288012769' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi117-63923-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:07.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:07 smithi117 bash[28843]: cluster 2024-03-04T15:57:06.139884+0000 mon.a (mon.0) 1915 : cluster 0 osdmap e237: 8 total, 8 up, 8 in 2024-03-04T15:57:07.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:07 smithi117 bash[28843]: audit 2024-03-04T15:57:06.154203+0000 mon.c (mon.2) 374 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:07.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:07 smithi117 bash[28843]: audit 2024-03-04T15:57:06.174141+0000 mon.a (mon.0) 1916 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:07.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:07 smithi151 bash[23756]: cluster 2024-03-04T15:57:04.875409+0000 mgr.y (mgr.24428) 303 : cluster 0 pgmap v359: 228 pgs: 228 active+clean; 583 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:57:07.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:07 smithi151 bash[23756]: cluster 2024-03-04T15:57:05.591771+0000 mon.a (mon.0) 1911 : cluster 0 osdmap e236: 8 total, 8 up, 8 in 2024-03-04T15:57:07.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:07 smithi151 bash[23756]: audit 2024-03-04T15:57:05.594568+0000 mon.a (mon.0) 1912 : audit 1 from='client.? 172.21.15.117:0/1288012769' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi117-63923-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:07.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:07 smithi151 bash[23756]: audit 2024-03-04T15:57:06.135067+0000 mon.a (mon.0) 1913 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi117-63915-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi117-63915-35"}]': finished 2024-03-04T15:57:07.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:07 smithi151 bash[23756]: audit 2024-03-04T15:57:06.135265+0000 mon.a (mon.0) 1914 : audit 1 from='client.? 172.21.15.117:0/1288012769' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi117-63923-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:07.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:07 smithi151 bash[23756]: cluster 2024-03-04T15:57:06.139884+0000 mon.a (mon.0) 1915 : cluster 0 osdmap e237: 8 total, 8 up, 8 in 2024-03-04T15:57:07.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:07 smithi151 bash[23756]: audit 2024-03-04T15:57:06.154203+0000 mon.c (mon.2) 374 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:07.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:07 smithi151 bash[23756]: audit 2024-03-04T15:57:06.174141+0000 mon.a (mon.0) 1916 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:08.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:08 smithi151 bash[23756]: audit 2024-03-04T15:57:06.612641+0000 mgr.y (mgr.24428) 304 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:57:08.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:08 smithi151 bash[23756]: cluster 2024-03-04T15:57:06.877815+0000 mgr.y (mgr.24428) 305 : cluster 0 pgmap v362: 268 pgs: 40 creating+peering, 228 active+clean; 583 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:57:08.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:08 smithi151 bash[23756]: audit 2024-03-04T15:57:07.148773+0000 mon.a (mon.0) 1917 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:08.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:08 smithi151 bash[23756]: cluster 2024-03-04T15:57:07.157921+0000 mon.a (mon.0) 1918 : cluster 0 osdmap e238: 8 total, 8 up, 8 in 2024-03-04T15:57:08.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:08 smithi151 bash[23756]: cluster 2024-03-04T15:57:08.079404+0000 mon.a (mon.0) 1919 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:08.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:08 smithi117 bash[21480]: audit 2024-03-04T15:57:06.612641+0000 mgr.y (mgr.24428) 304 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:57:08.728 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:08 smithi117 bash[21480]: cluster 2024-03-04T15:57:06.877815+0000 mgr.y (mgr.24428) 305 : cluster 0 pgmap v362: 268 pgs: 40 creating+peering, 228 active+clean; 583 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:57:08.728 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:08 smithi117 bash[21480]: audit 2024-03-04T15:57:07.148773+0000 mon.a (mon.0) 1917 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:08.728 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:08 smithi117 bash[21480]: cluster 2024-03-04T15:57:07.157921+0000 mon.a (mon.0) 1918 : cluster 0 osdmap e238: 8 total, 8 up, 8 in 2024-03-04T15:57:08.728 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:08 smithi117 bash[21480]: cluster 2024-03-04T15:57:08.079404+0000 mon.a (mon.0) 1919 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:08.729 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:08 smithi117 bash[28843]: audit 2024-03-04T15:57:06.612641+0000 mgr.y (mgr.24428) 304 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:57:08.729 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:08 smithi117 bash[28843]: cluster 2024-03-04T15:57:06.877815+0000 mgr.y (mgr.24428) 305 : cluster 0 pgmap v362: 268 pgs: 40 creating+peering, 228 active+clean; 583 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:57:08.729 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:08 smithi117 bash[28843]: audit 2024-03-04T15:57:07.148773+0000 mon.a (mon.0) 1917 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:08.729 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:08 smithi117 bash[28843]: cluster 2024-03-04T15:57:07.157921+0000 mon.a (mon.0) 1918 : cluster 0 osdmap e238: 8 total, 8 up, 8 in 2024-03-04T15:57:08.729 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:08 smithi117 bash[28843]: cluster 2024-03-04T15:57:08.079404+0000 mon.a (mon.0) 1919 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:09.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:09 smithi151 bash[23756]: cluster 2024-03-04T15:57:08.163414+0000 mon.a (mon.0) 1920 : cluster 0 osdmap e239: 8 total, 8 up, 8 in 2024-03-04T15:57:09.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:09 smithi151 bash[23756]: audit 2024-03-04T15:57:08.166877+0000 mon.a (mon.0) 1921 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi117-63915-35"}]: dispatch 2024-03-04T15:57:09.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:09 smithi151 bash[23756]: audit 2024-03-04T15:57:08.167160+0000 mon.a (mon.0) 1922 : audit 1 from='client.? 172.21.15.117:0/2989802050' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi117-63923-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:09.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:09 smithi151 bash[23756]: audit 2024-03-04T15:57:08.208855+0000 mon.c (mon.2) 375 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:09.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:09 smithi151 bash[23756]: audit 2024-03-04T15:57:08.209376+0000 mon.a (mon.0) 1923 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:09.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:09 smithi151 bash[23756]: audit 2024-03-04T15:57:09.155675+0000 mon.a (mon.0) 1924 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi117-63915-35"}]': finished 2024-03-04T15:57:09.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:09 smithi151 bash[23756]: audit 2024-03-04T15:57:09.155819+0000 mon.a (mon.0) 1925 : audit 1 from='client.? 172.21.15.117:0/2989802050' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi117-63923-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:09.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:09 smithi151 bash[23756]: audit 2024-03-04T15:57:09.155985+0000 mon.a (mon.0) 1926 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-27", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:57:09.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:09 smithi151 bash[23756]: cluster 2024-03-04T15:57:09.164300+0000 mon.a (mon.0) 1927 : cluster 0 osdmap e240: 8 total, 8 up, 8 in 2024-03-04T15:57:09.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:09 smithi151 bash[23756]: audit 2024-03-04T15:57:09.165699+0000 mon.c (mon.2) 376 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-27"}]: dispatch 2024-03-04T15:57:09.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:09 smithi151 bash[23756]: audit 2024-03-04T15:57:09.168541+0000 mon.a (mon.0) 1928 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi117-63915-35"}]: dispatch 2024-03-04T15:57:09.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[21480]: cluster 2024-03-04T15:57:08.163414+0000 mon.a (mon.0) 1920 : cluster 0 osdmap e239: 8 total, 8 up, 8 in 2024-03-04T15:57:09.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[21480]: audit 2024-03-04T15:57:08.166877+0000 mon.a (mon.0) 1921 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi117-63915-35"}]: dispatch 2024-03-04T15:57:09.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[21480]: audit 2024-03-04T15:57:08.167160+0000 mon.a (mon.0) 1922 : audit 1 from='client.? 172.21.15.117:0/2989802050' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi117-63923-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:09.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[21480]: audit 2024-03-04T15:57:08.208855+0000 mon.c (mon.2) 375 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:09.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[21480]: audit 2024-03-04T15:57:08.209376+0000 mon.a (mon.0) 1923 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:09.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[21480]: audit 2024-03-04T15:57:09.155675+0000 mon.a (mon.0) 1924 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi117-63915-35"}]': finished 2024-03-04T15:57:09.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[21480]: audit 2024-03-04T15:57:09.155819+0000 mon.a (mon.0) 1925 : audit 1 from='client.? 172.21.15.117:0/2989802050' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi117-63923-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:09.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[21480]: audit 2024-03-04T15:57:09.155985+0000 mon.a (mon.0) 1926 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-27", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:57:09.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[21480]: cluster 2024-03-04T15:57:09.164300+0000 mon.a (mon.0) 1927 : cluster 0 osdmap e240: 8 total, 8 up, 8 in 2024-03-04T15:57:09.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[21480]: audit 2024-03-04T15:57:09.165699+0000 mon.c (mon.2) 376 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-27"}]: dispatch 2024-03-04T15:57:09.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[21480]: audit 2024-03-04T15:57:09.168541+0000 mon.a (mon.0) 1928 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi117-63915-35"}]: dispatch 2024-03-04T15:57:09.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[28843]: cluster 2024-03-04T15:57:08.163414+0000 mon.a (mon.0) 1920 : cluster 0 osdmap e239: 8 total, 8 up, 8 in 2024-03-04T15:57:09.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[28843]: audit 2024-03-04T15:57:08.166877+0000 mon.a (mon.0) 1921 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi117-63915-35"}]: dispatch 2024-03-04T15:57:09.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[28843]: audit 2024-03-04T15:57:08.167160+0000 mon.a (mon.0) 1922 : audit 1 from='client.? 172.21.15.117:0/2989802050' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi117-63923-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:09.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[28843]: audit 2024-03-04T15:57:08.208855+0000 mon.c (mon.2) 375 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:09.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[28843]: audit 2024-03-04T15:57:08.209376+0000 mon.a (mon.0) 1923 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:09.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[28843]: audit 2024-03-04T15:57:09.155675+0000 mon.a (mon.0) 1924 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi117-63915-35"}]': finished 2024-03-04T15:57:09.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[28843]: audit 2024-03-04T15:57:09.155819+0000 mon.a (mon.0) 1925 : audit 1 from='client.? 172.21.15.117:0/2989802050' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi117-63923-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:09.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[28843]: audit 2024-03-04T15:57:09.155985+0000 mon.a (mon.0) 1926 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-27", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:57:09.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[28843]: cluster 2024-03-04T15:57:09.164300+0000 mon.a (mon.0) 1927 : cluster 0 osdmap e240: 8 total, 8 up, 8 in 2024-03-04T15:57:09.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[28843]: audit 2024-03-04T15:57:09.165699+0000 mon.c (mon.2) 376 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-27"}]: dispatch 2024-03-04T15:57:09.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:09 smithi117 bash[28843]: audit 2024-03-04T15:57:09.168541+0000 mon.a (mon.0) 1928 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi117-63915-35"}]: dispatch 2024-03-04T15:57:10.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:10 smithi117 bash[21480]: cluster 2024-03-04T15:57:08.878698+0000 mgr.y (mgr.24428) 306 : cluster 0 pgmap v365: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 583 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:57:10.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:10 smithi117 bash[21480]: audit 2024-03-04T15:57:09.175135+0000 mon.a (mon.0) 1929 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-27"}]: dispatch 2024-03-04T15:57:10.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:10 smithi117 bash[21480]: audit 2024-03-04T15:57:10.159290+0000 mon.a (mon.0) 1930 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi117-63915-35"}]': finished 2024-03-04T15:57:10.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:10 smithi117 bash[21480]: audit 2024-03-04T15:57:10.159593+0000 mon.a (mon.0) 1931 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-27"}]': finished 2024-03-04T15:57:10.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:10 smithi117 bash[21480]: cluster 2024-03-04T15:57:10.166870+0000 mon.a (mon.0) 1932 : cluster 0 osdmap e241: 8 total, 8 up, 8 in 2024-03-04T15:57:10.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:10 smithi117 bash[21480]: audit 2024-03-04T15:57:10.170035+0000 mon.c (mon.2) 377 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-27", "mode": "writeback"}]: dispatch 2024-03-04T15:57:10.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:10 smithi117 bash[21480]: audit 2024-03-04T15:57:10.178038+0000 mon.a (mon.0) 1933 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-27", "mode": "writeback"}]: dispatch 2024-03-04T15:57:10.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:10 smithi117 bash[28843]: cluster 2024-03-04T15:57:08.878698+0000 mgr.y (mgr.24428) 306 : cluster 0 pgmap v365: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 583 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:57:10.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:10 smithi117 bash[28843]: audit 2024-03-04T15:57:09.175135+0000 mon.a (mon.0) 1929 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-27"}]: dispatch 2024-03-04T15:57:10.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:10 smithi117 bash[28843]: audit 2024-03-04T15:57:10.159290+0000 mon.a (mon.0) 1930 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi117-63915-35"}]': finished 2024-03-04T15:57:10.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:10 smithi117 bash[28843]: audit 2024-03-04T15:57:10.159593+0000 mon.a (mon.0) 1931 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-27"}]': finished 2024-03-04T15:57:10.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:10 smithi117 bash[28843]: cluster 2024-03-04T15:57:10.166870+0000 mon.a (mon.0) 1932 : cluster 0 osdmap e241: 8 total, 8 up, 8 in 2024-03-04T15:57:10.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:10 smithi117 bash[28843]: audit 2024-03-04T15:57:10.170035+0000 mon.c (mon.2) 377 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-27", "mode": "writeback"}]: dispatch 2024-03-04T15:57:10.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:10 smithi117 bash[28843]: audit 2024-03-04T15:57:10.178038+0000 mon.a (mon.0) 1933 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-27", "mode": "writeback"}]: dispatch 2024-03-04T15:57:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:10 smithi151 bash[23756]: cluster 2024-03-04T15:57:08.878698+0000 mgr.y (mgr.24428) 306 : cluster 0 pgmap v365: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 583 KiB data, 602 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:57:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:10 smithi151 bash[23756]: audit 2024-03-04T15:57:09.175135+0000 mon.a (mon.0) 1929 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-27"}]: dispatch 2024-03-04T15:57:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:10 smithi151 bash[23756]: audit 2024-03-04T15:57:10.159290+0000 mon.a (mon.0) 1930 : audit 1 from='client.? 172.21.15.117:0/3583603746' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi117-63915-35"}]': finished 2024-03-04T15:57:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:10 smithi151 bash[23756]: audit 2024-03-04T15:57:10.159593+0000 mon.a (mon.0) 1931 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-27"}]': finished 2024-03-04T15:57:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:10 smithi151 bash[23756]: cluster 2024-03-04T15:57:10.166870+0000 mon.a (mon.0) 1932 : cluster 0 osdmap e241: 8 total, 8 up, 8 in 2024-03-04T15:57:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:10 smithi151 bash[23756]: audit 2024-03-04T15:57:10.170035+0000 mon.c (mon.2) 377 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-27", "mode": "writeback"}]: dispatch 2024-03-04T15:57:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:10 smithi151 bash[23756]: audit 2024-03-04T15:57:10.178038+0000 mon.a (mon.0) 1933 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-27", "mode": "writeback"}]: dispatch 2024-03-04T15:57:11.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[21480]: audit 2024-03-04T15:57:10.204937+0000 mon.a (mon.0) 1934 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-46"}]: dispatch 2024-03-04T15:57:11.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[21480]: audit 2024-03-04T15:57:10.206326+0000 mon.a (mon.0) 1935 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-46"}]: dispatch 2024-03-04T15:57:11.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[21480]: audit 2024-03-04T15:57:10.206749+0000 mon.a (mon.0) 1936 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi117-63923-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:11.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[21480]: audit 2024-03-04T15:57:10.210319+0000 mon.a (mon.0) 1937 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi117-63915-36"}]: dispatch 2024-03-04T15:57:11.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[21480]: audit 2024-03-04T15:57:10.211258+0000 mon.a (mon.0) 1938 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi117-63915-36"}]: dispatch 2024-03-04T15:57:11.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[21480]: audit 2024-03-04T15:57:10.211644+0000 mon.a (mon.0) 1939 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi117-63915-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:11.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[21480]: cluster 2024-03-04T15:57:11.159674+0000 mon.a (mon.0) 1940 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:57:11.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[21480]: audit 2024-03-04T15:57:11.162087+0000 mon.a (mon.0) 1941 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-27", "mode": "writeback"}]': finished 2024-03-04T15:57:11.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[21480]: audit 2024-03-04T15:57:11.162242+0000 mon.a (mon.0) 1942 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi117-63923-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:11.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[21480]: audit 2024-03-04T15:57:11.162349+0000 mon.a (mon.0) 1943 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi117-63915-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:11.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[21480]: cluster 2024-03-04T15:57:11.168173+0000 mon.a (mon.0) 1944 : cluster 0 osdmap e242: 8 total, 8 up, 8 in 2024-03-04T15:57:11.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[21480]: audit 2024-03-04T15:57:11.169829+0000 mon.a (mon.0) 1945 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi117-63915-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi117-63915-36"}]: dispatch 2024-03-04T15:57:11.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[21480]: audit 2024-03-04T15:57:11.170486+0000 mon.a (mon.0) 1946 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi117-63923-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi117-63923-46"}]: dispatch 2024-03-04T15:57:11.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[28843]: audit 2024-03-04T15:57:10.204937+0000 mon.a (mon.0) 1934 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-46"}]: dispatch 2024-03-04T15:57:11.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[28843]: audit 2024-03-04T15:57:10.206326+0000 mon.a (mon.0) 1935 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-46"}]: dispatch 2024-03-04T15:57:11.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[28843]: audit 2024-03-04T15:57:10.206749+0000 mon.a (mon.0) 1936 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi117-63923-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:11.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[28843]: audit 2024-03-04T15:57:10.210319+0000 mon.a (mon.0) 1937 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi117-63915-36"}]: dispatch 2024-03-04T15:57:11.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[28843]: audit 2024-03-04T15:57:10.211258+0000 mon.a (mon.0) 1938 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi117-63915-36"}]: dispatch 2024-03-04T15:57:11.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[28843]: audit 2024-03-04T15:57:10.211644+0000 mon.a (mon.0) 1939 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi117-63915-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:11.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[28843]: cluster 2024-03-04T15:57:11.159674+0000 mon.a (mon.0) 1940 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:57:11.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[28843]: audit 2024-03-04T15:57:11.162087+0000 mon.a (mon.0) 1941 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-27", "mode": "writeback"}]': finished 2024-03-04T15:57:11.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[28843]: audit 2024-03-04T15:57:11.162242+0000 mon.a (mon.0) 1942 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi117-63923-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:11.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[28843]: audit 2024-03-04T15:57:11.162349+0000 mon.a (mon.0) 1943 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi117-63915-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:11.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[28843]: cluster 2024-03-04T15:57:11.168173+0000 mon.a (mon.0) 1944 : cluster 0 osdmap e242: 8 total, 8 up, 8 in 2024-03-04T15:57:11.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[28843]: audit 2024-03-04T15:57:11.169829+0000 mon.a (mon.0) 1945 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi117-63915-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi117-63915-36"}]: dispatch 2024-03-04T15:57:11.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:11 smithi117 bash[28843]: audit 2024-03-04T15:57:11.170486+0000 mon.a (mon.0) 1946 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi117-63923-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi117-63923-46"}]: dispatch 2024-03-04T15:57:11.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:11 smithi151 bash[23756]: audit 2024-03-04T15:57:10.204937+0000 mon.a (mon.0) 1934 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-46"}]: dispatch 2024-03-04T15:57:11.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:11 smithi151 bash[23756]: audit 2024-03-04T15:57:10.206326+0000 mon.a (mon.0) 1935 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-46"}]: dispatch 2024-03-04T15:57:11.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:11 smithi151 bash[23756]: audit 2024-03-04T15:57:10.206749+0000 mon.a (mon.0) 1936 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi117-63923-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:11.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:11 smithi151 bash[23756]: audit 2024-03-04T15:57:10.210319+0000 mon.a (mon.0) 1937 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi117-63915-36"}]: dispatch 2024-03-04T15:57:11.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:11 smithi151 bash[23756]: audit 2024-03-04T15:57:10.211258+0000 mon.a (mon.0) 1938 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi117-63915-36"}]: dispatch 2024-03-04T15:57:11.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:11 smithi151 bash[23756]: audit 2024-03-04T15:57:10.211644+0000 mon.a (mon.0) 1939 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi117-63915-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:11.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:11 smithi151 bash[23756]: cluster 2024-03-04T15:57:11.159674+0000 mon.a (mon.0) 1940 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:57:11.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:11 smithi151 bash[23756]: audit 2024-03-04T15:57:11.162087+0000 mon.a (mon.0) 1941 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-27", "mode": "writeback"}]': finished 2024-03-04T15:57:11.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:11 smithi151 bash[23756]: audit 2024-03-04T15:57:11.162242+0000 mon.a (mon.0) 1942 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi117-63923-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:11.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:11 smithi151 bash[23756]: audit 2024-03-04T15:57:11.162349+0000 mon.a (mon.0) 1943 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi117-63915-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:11.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:11 smithi151 bash[23756]: cluster 2024-03-04T15:57:11.168173+0000 mon.a (mon.0) 1944 : cluster 0 osdmap e242: 8 total, 8 up, 8 in 2024-03-04T15:57:11.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:11 smithi151 bash[23756]: audit 2024-03-04T15:57:11.169829+0000 mon.a (mon.0) 1945 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi117-63915-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi117-63915-36"}]: dispatch 2024-03-04T15:57:11.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:11 smithi151 bash[23756]: audit 2024-03-04T15:57:11.170486+0000 mon.a (mon.0) 1946 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi117-63923-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi117-63923-46"}]: dispatch 2024-03-04T15:57:13.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:13 smithi151 bash[23756]: cluster 2024-03-04T15:57:10.879524+0000 mgr.y (mgr.24428) 307 : cluster 0 pgmap v368: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:57:13.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:13 smithi117 bash[21480]: cluster 2024-03-04T15:57:10.879524+0000 mgr.y (mgr.24428) 307 : cluster 0 pgmap v368: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:57:13.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:13 smithi117 bash[28843]: cluster 2024-03-04T15:57:10.879524+0000 mgr.y (mgr.24428) 307 : cluster 0 pgmap v368: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 602 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:57:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:14 smithi151 bash[23756]: cluster 2024-03-04T15:57:12.880444+0000 mgr.y (mgr.24428) 308 : cluster 0 pgmap v371: 228 pgs: 228 active+clean; 584 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-04T15:57:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:14 smithi151 bash[23756]: cluster 2024-03-04T15:57:13.067210+0000 mon.a (mon.0) 1947 : cluster 0 osdmap e243: 8 total, 8 up, 8 in 2024-03-04T15:57:14.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:14 smithi151 bash[23756]: cluster 2024-03-04T15:57:13.081653+0000 mon.a (mon.0) 1948 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:14.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:14 smithi151 bash[23756]: audit 2024-03-04T15:57:13.092105+0000 mon.c (mon.2) 378 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:14.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:14 smithi151 bash[23756]: audit 2024-03-04T15:57:13.092471+0000 mon.a (mon.0) 1949 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:14.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:14 smithi151 bash[23756]: audit 2024-03-04T15:57:13.169034+0000 mon.a (mon.0) 1950 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi117-63915-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi117-63915-36"}]': finished 2024-03-04T15:57:14.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:14 smithi151 bash[23756]: audit 2024-03-04T15:57:13.169263+0000 mon.a (mon.0) 1951 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi117-63923-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi117-63923-46"}]': finished 2024-03-04T15:57:14.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:14 smithi151 bash[23756]: audit 2024-03-04T15:57:13.169445+0000 mon.a (mon.0) 1952 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:14.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:14 smithi151 bash[23756]: cluster 2024-03-04T15:57:13.174860+0000 mon.a (mon.0) 1953 : cluster 0 osdmap e244: 8 total, 8 up, 8 in 2024-03-04T15:57:14.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:14 smithi151 bash[23756]: audit 2024-03-04T15:57:13.178001+0000 mon.c (mon.2) 379 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-27"}]: dispatch 2024-03-04T15:57:14.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:14 smithi151 bash[23756]: audit 2024-03-04T15:57:13.192820+0000 mon.a (mon.0) 1954 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-27"}]: dispatch 2024-03-04T15:57:14.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:14 smithi151 bash[23756]: audit 2024-03-04T15:57:13.610594+0000 mon.a (mon.0) 1955 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:57:14.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:14 smithi151 bash[23756]: audit 2024-03-04T15:57:13.615425+0000 mon.c (mon.2) 380 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:57:14.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[28843]: cluster 2024-03-04T15:57:12.880444+0000 mgr.y (mgr.24428) 308 : cluster 0 pgmap v371: 228 pgs: 228 active+clean; 584 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-04T15:57:14.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[28843]: cluster 2024-03-04T15:57:13.067210+0000 mon.a (mon.0) 1947 : cluster 0 osdmap e243: 8 total, 8 up, 8 in 2024-03-04T15:57:14.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[28843]: cluster 2024-03-04T15:57:13.081653+0000 mon.a (mon.0) 1948 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:14.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[28843]: audit 2024-03-04T15:57:13.092105+0000 mon.c (mon.2) 378 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:14.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[28843]: audit 2024-03-04T15:57:13.092471+0000 mon.a (mon.0) 1949 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:14.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[28843]: audit 2024-03-04T15:57:13.169034+0000 mon.a (mon.0) 1950 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi117-63915-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi117-63915-36"}]': finished 2024-03-04T15:57:14.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[28843]: audit 2024-03-04T15:57:13.169263+0000 mon.a (mon.0) 1951 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi117-63923-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi117-63923-46"}]': finished 2024-03-04T15:57:14.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[28843]: audit 2024-03-04T15:57:13.169445+0000 mon.a (mon.0) 1952 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:14.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[28843]: cluster 2024-03-04T15:57:13.174860+0000 mon.a (mon.0) 1953 : cluster 0 osdmap e244: 8 total, 8 up, 8 in 2024-03-04T15:57:14.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[28843]: audit 2024-03-04T15:57:13.178001+0000 mon.c (mon.2) 379 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-27"}]: dispatch 2024-03-04T15:57:14.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[28843]: audit 2024-03-04T15:57:13.192820+0000 mon.a (mon.0) 1954 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-27"}]: dispatch 2024-03-04T15:57:14.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[28843]: audit 2024-03-04T15:57:13.610594+0000 mon.a (mon.0) 1955 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:57:14.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[28843]: audit 2024-03-04T15:57:13.615425+0000 mon.c (mon.2) 380 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:57:14.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[21480]: cluster 2024-03-04T15:57:12.880444+0000 mgr.y (mgr.24428) 308 : cluster 0 pgmap v371: 228 pgs: 228 active+clean; 584 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-04T15:57:14.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[21480]: cluster 2024-03-04T15:57:13.067210+0000 mon.a (mon.0) 1947 : cluster 0 osdmap e243: 8 total, 8 up, 8 in 2024-03-04T15:57:14.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[21480]: cluster 2024-03-04T15:57:13.081653+0000 mon.a (mon.0) 1948 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:14.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[21480]: audit 2024-03-04T15:57:13.092105+0000 mon.c (mon.2) 378 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:14.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[21480]: audit 2024-03-04T15:57:13.092471+0000 mon.a (mon.0) 1949 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:14.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[21480]: audit 2024-03-04T15:57:13.169034+0000 mon.a (mon.0) 1950 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi117-63915-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi117-63915-36"}]': finished 2024-03-04T15:57:14.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[21480]: audit 2024-03-04T15:57:13.169263+0000 mon.a (mon.0) 1951 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi117-63923-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi117-63923-46"}]': finished 2024-03-04T15:57:14.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[21480]: audit 2024-03-04T15:57:13.169445+0000 mon.a (mon.0) 1952 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:14.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[21480]: cluster 2024-03-04T15:57:13.174860+0000 mon.a (mon.0) 1953 : cluster 0 osdmap e244: 8 total, 8 up, 8 in 2024-03-04T15:57:14.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[21480]: audit 2024-03-04T15:57:13.178001+0000 mon.c (mon.2) 379 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-27"}]: dispatch 2024-03-04T15:57:14.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[21480]: audit 2024-03-04T15:57:13.192820+0000 mon.a (mon.0) 1954 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-27"}]: dispatch 2024-03-04T15:57:14.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[21480]: audit 2024-03-04T15:57:13.610594+0000 mon.a (mon.0) 1955 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:57:14.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:14 smithi117 bash[21480]: audit 2024-03-04T15:57:13.615425+0000 mon.c (mon.2) 380 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:57:15.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:15 smithi151 bash[23756]: cluster 2024-03-04T15:57:14.169612+0000 mon.a (mon.0) 1956 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:57:15.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:15 smithi151 bash[23756]: audit 2024-03-04T15:57:14.173125+0000 mon.a (mon.0) 1957 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-27"}]': finished 2024-03-04T15:57:15.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:15 smithi151 bash[23756]: cluster 2024-03-04T15:57:14.181920+0000 mon.a (mon.0) 1958 : cluster 0 osdmap e245: 8 total, 8 up, 8 in 2024-03-04T15:57:15.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:15 smithi117 bash[21480]: cluster 2024-03-04T15:57:14.169612+0000 mon.a (mon.0) 1956 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:57:15.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:15 smithi117 bash[21480]: audit 2024-03-04T15:57:14.173125+0000 mon.a (mon.0) 1957 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-27"}]': finished 2024-03-04T15:57:15.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:15 smithi117 bash[21480]: cluster 2024-03-04T15:57:14.181920+0000 mon.a (mon.0) 1958 : cluster 0 osdmap e245: 8 total, 8 up, 8 in 2024-03-04T15:57:15.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:15 smithi117 bash[28843]: cluster 2024-03-04T15:57:14.169612+0000 mon.a (mon.0) 1956 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:57:15.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:15 smithi117 bash[28843]: audit 2024-03-04T15:57:14.173125+0000 mon.a (mon.0) 1957 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-27"}]': finished 2024-03-04T15:57:15.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:15 smithi117 bash[28843]: cluster 2024-03-04T15:57:14.181920+0000 mon.a (mon.0) 1958 : cluster 0 osdmap e245: 8 total, 8 up, 8 in 2024-03-04T15:57:16.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:57:16 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:57:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:57:16.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:16 smithi117 bash[21480]: cluster 2024-03-04T15:57:14.881378+0000 mgr.y (mgr.24428) 309 : cluster 0 pgmap v374: 244 pgs: 16 unknown, 228 active+clean; 584 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-04T15:57:16.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:16 smithi117 bash[21480]: cluster 2024-03-04T15:57:15.192186+0000 mon.a (mon.0) 1959 : cluster 0 osdmap e246: 8 total, 8 up, 8 in 2024-03-04T15:57:16.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:16 smithi117 bash[21480]: audit 2024-03-04T15:57:15.194034+0000 mon.a (mon.0) 1960 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi117-63915-36"}]: dispatch 2024-03-04T15:57:16.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:16 smithi117 bash[21480]: audit 2024-03-04T15:57:15.194826+0000 mon.a (mon.0) 1961 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-46"}]: dispatch 2024-03-04T15:57:16.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:16 smithi117 bash[28843]: cluster 2024-03-04T15:57:14.881378+0000 mgr.y (mgr.24428) 309 : cluster 0 pgmap v374: 244 pgs: 16 unknown, 228 active+clean; 584 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-04T15:57:16.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:16 smithi117 bash[28843]: cluster 2024-03-04T15:57:15.192186+0000 mon.a (mon.0) 1959 : cluster 0 osdmap e246: 8 total, 8 up, 8 in 2024-03-04T15:57:16.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:16 smithi117 bash[28843]: audit 2024-03-04T15:57:15.194034+0000 mon.a (mon.0) 1960 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi117-63915-36"}]: dispatch 2024-03-04T15:57:16.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:16 smithi117 bash[28843]: audit 2024-03-04T15:57:15.194826+0000 mon.a (mon.0) 1961 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-46"}]: dispatch 2024-03-04T15:57:16.622 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:16 smithi151 bash[23756]: cluster 2024-03-04T15:57:14.881378+0000 mgr.y (mgr.24428) 309 : cluster 0 pgmap v374: 244 pgs: 16 unknown, 228 active+clean; 584 KiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-04T15:57:16.622 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:16 smithi151 bash[23756]: cluster 2024-03-04T15:57:15.192186+0000 mon.a (mon.0) 1959 : cluster 0 osdmap e246: 8 total, 8 up, 8 in 2024-03-04T15:57:16.622 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:16 smithi151 bash[23756]: audit 2024-03-04T15:57:15.194034+0000 mon.a (mon.0) 1960 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi117-63915-36"}]: dispatch 2024-03-04T15:57:16.622 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:16 smithi151 bash[23756]: audit 2024-03-04T15:57:15.194826+0000 mon.a (mon.0) 1961 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-46"}]: dispatch 2024-03-04T15:57:16.932 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:57:16 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:57:17.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[21480]: audit 2024-03-04T15:57:16.187718+0000 mon.a (mon.0) 1962 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi117-63915-36"}]': finished 2024-03-04T15:57:17.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[21480]: audit 2024-03-04T15:57:16.187899+0000 mon.a (mon.0) 1963 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-46"}]': finished 2024-03-04T15:57:17.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[21480]: cluster 2024-03-04T15:57:16.199672+0000 mon.a (mon.0) 1964 : cluster 0 osdmap e247: 8 total, 8 up, 8 in 2024-03-04T15:57:17.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[21480]: audit 2024-03-04T15:57:16.203759+0000 mon.a (mon.0) 1965 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi117-63915-36"}]: dispatch 2024-03-04T15:57:17.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[21480]: audit 2024-03-04T15:57:16.204964+0000 mon.a (mon.0) 1966 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-46"}]: dispatch 2024-03-04T15:57:17.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[21480]: audit 2024-03-04T15:57:16.206224+0000 mon.c (mon.2) 381 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:17.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[21480]: audit 2024-03-04T15:57:16.227346+0000 mon.a (mon.0) 1967 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:17.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[21480]: audit 2024-03-04T15:57:17.191467+0000 mon.a (mon.0) 1968 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi117-63915-36"}]': finished 2024-03-04T15:57:17.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[21480]: audit 2024-03-04T15:57:17.191661+0000 mon.a (mon.0) 1969 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-46"}]': finished 2024-03-04T15:57:17.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[21480]: audit 2024-03-04T15:57:17.191826+0000 mon.a (mon.0) 1970 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:17.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[21480]: cluster 2024-03-04T15:57:17.201193+0000 mon.a (mon.0) 1971 : cluster 0 osdmap e248: 8 total, 8 up, 8 in 2024-03-04T15:57:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[28843]: audit 2024-03-04T15:57:16.187718+0000 mon.a (mon.0) 1962 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi117-63915-36"}]': finished 2024-03-04T15:57:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[28843]: audit 2024-03-04T15:57:16.187899+0000 mon.a (mon.0) 1963 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-46"}]': finished 2024-03-04T15:57:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[28843]: cluster 2024-03-04T15:57:16.199672+0000 mon.a (mon.0) 1964 : cluster 0 osdmap e247: 8 total, 8 up, 8 in 2024-03-04T15:57:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[28843]: audit 2024-03-04T15:57:16.203759+0000 mon.a (mon.0) 1965 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi117-63915-36"}]: dispatch 2024-03-04T15:57:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[28843]: audit 2024-03-04T15:57:16.204964+0000 mon.a (mon.0) 1966 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-46"}]: dispatch 2024-03-04T15:57:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[28843]: audit 2024-03-04T15:57:16.206224+0000 mon.c (mon.2) 381 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[28843]: audit 2024-03-04T15:57:16.227346+0000 mon.a (mon.0) 1967 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[28843]: audit 2024-03-04T15:57:17.191467+0000 mon.a (mon.0) 1968 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi117-63915-36"}]': finished 2024-03-04T15:57:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[28843]: audit 2024-03-04T15:57:17.191661+0000 mon.a (mon.0) 1969 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-46"}]': finished 2024-03-04T15:57:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[28843]: audit 2024-03-04T15:57:17.191826+0000 mon.a (mon.0) 1970 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:17.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:17 smithi117 bash[28843]: cluster 2024-03-04T15:57:17.201193+0000 mon.a (mon.0) 1971 : cluster 0 osdmap e248: 8 total, 8 up, 8 in 2024-03-04T15:57:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:17 smithi151 bash[23756]: audit 2024-03-04T15:57:16.187718+0000 mon.a (mon.0) 1962 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi117-63915-36"}]': finished 2024-03-04T15:57:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:17 smithi151 bash[23756]: audit 2024-03-04T15:57:16.187899+0000 mon.a (mon.0) 1963 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-46"}]': finished 2024-03-04T15:57:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:17 smithi151 bash[23756]: cluster 2024-03-04T15:57:16.199672+0000 mon.a (mon.0) 1964 : cluster 0 osdmap e247: 8 total, 8 up, 8 in 2024-03-04T15:57:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:17 smithi151 bash[23756]: audit 2024-03-04T15:57:16.203759+0000 mon.a (mon.0) 1965 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi117-63915-36"}]: dispatch 2024-03-04T15:57:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:17 smithi151 bash[23756]: audit 2024-03-04T15:57:16.204964+0000 mon.a (mon.0) 1966 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-46"}]: dispatch 2024-03-04T15:57:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:17 smithi151 bash[23756]: audit 2024-03-04T15:57:16.206224+0000 mon.c (mon.2) 381 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:17 smithi151 bash[23756]: audit 2024-03-04T15:57:16.227346+0000 mon.a (mon.0) 1967 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:17.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:17 smithi151 bash[23756]: audit 2024-03-04T15:57:17.191467+0000 mon.a (mon.0) 1968 : audit 1 from='client.? 172.21.15.117:0/2684699416' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi117-63915-36"}]': finished 2024-03-04T15:57:17.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:17 smithi151 bash[23756]: audit 2024-03-04T15:57:17.191661+0000 mon.a (mon.0) 1969 : audit 1 from='client.? 172.21.15.117:0/263498935' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-46"}]': finished 2024-03-04T15:57:17.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:17 smithi151 bash[23756]: audit 2024-03-04T15:57:17.191826+0000 mon.a (mon.0) 1970 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:17.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:17 smithi151 bash[23756]: cluster 2024-03-04T15:57:17.201193+0000 mon.a (mon.0) 1971 : cluster 0 osdmap e248: 8 total, 8 up, 8 in 2024-03-04T15:57:18.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[21480]: audit 2024-03-04T15:57:16.624149+0000 mgr.y (mgr.24428) 310 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:57:18.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[21480]: cluster 2024-03-04T15:57:16.883556+0000 mgr.y (mgr.24428) 311 : cluster 0 pgmap v377: 228 pgs: 32 creating+peering, 196 active+clean; 584 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-04T15:57:18.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[21480]: audit 2024-03-04T15:57:17.264258+0000 mon.c (mon.2) 382 : audit 1 from='client.? 172.21.15.117:0/2345402729' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:18.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[21480]: audit 2024-03-04T15:57:17.264644+0000 mon.a (mon.0) 1972 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:18.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[21480]: audit 2024-03-04T15:57:17.265009+0000 mon.a (mon.0) 1973 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi117-63915-37"}]: dispatch 2024-03-04T15:57:18.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[21480]: audit 2024-03-04T15:57:17.265848+0000 mon.a (mon.0) 1974 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi117-63915-37"}]: dispatch 2024-03-04T15:57:18.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[21480]: audit 2024-03-04T15:57:17.266059+0000 mon.c (mon.2) 383 : audit 1 from='client.? 172.21.15.117:0/2345402729' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:18.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[21480]: audit 2024-03-04T15:57:17.266382+0000 mon.a (mon.0) 1975 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi117-63915-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:18.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[21480]: audit 2024-03-04T15:57:17.266720+0000 mon.a (mon.0) 1976 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:18.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[21480]: audit 2024-03-04T15:57:17.267152+0000 mon.c (mon.2) 384 : audit 1 from='client.? 172.21.15.117:0/2345402729' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi117-63923-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:18.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[21480]: audit 2024-03-04T15:57:17.267417+0000 mon.a (mon.0) 1977 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi117-63923-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:18.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[21480]: cluster 2024-03-04T15:57:18.083043+0000 mon.a (mon.0) 1978 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:18.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[21480]: audit 2024-03-04T15:57:18.087197+0000 mon.a (mon.0) 1979 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi117-63915-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:18.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[21480]: audit 2024-03-04T15:57:18.087399+0000 mon.a (mon.0) 1980 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi117-63923-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:18.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[21480]: cluster 2024-03-04T15:57:18.097920+0000 mon.a (mon.0) 1981 : cluster 0 osdmap e249: 8 total, 8 up, 8 in 2024-03-04T15:57:18.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[28843]: audit 2024-03-04T15:57:16.624149+0000 mgr.y (mgr.24428) 310 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:57:18.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[28843]: cluster 2024-03-04T15:57:16.883556+0000 mgr.y (mgr.24428) 311 : cluster 0 pgmap v377: 228 pgs: 32 creating+peering, 196 active+clean; 584 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-04T15:57:18.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[28843]: audit 2024-03-04T15:57:17.264258+0000 mon.c (mon.2) 382 : audit 1 from='client.? 172.21.15.117:0/2345402729' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:18.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[28843]: audit 2024-03-04T15:57:17.264644+0000 mon.a (mon.0) 1972 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:18.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[28843]: audit 2024-03-04T15:57:17.265009+0000 mon.a (mon.0) 1973 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi117-63915-37"}]: dispatch 2024-03-04T15:57:18.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[28843]: audit 2024-03-04T15:57:17.265848+0000 mon.a (mon.0) 1974 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi117-63915-37"}]: dispatch 2024-03-04T15:57:18.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[28843]: audit 2024-03-04T15:57:17.266059+0000 mon.c (mon.2) 383 : audit 1 from='client.? 172.21.15.117:0/2345402729' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:18.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[28843]: audit 2024-03-04T15:57:17.266382+0000 mon.a (mon.0) 1975 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi117-63915-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:18.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[28843]: audit 2024-03-04T15:57:17.266720+0000 mon.a (mon.0) 1976 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:18.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[28843]: audit 2024-03-04T15:57:17.267152+0000 mon.c (mon.2) 384 : audit 1 from='client.? 172.21.15.117:0/2345402729' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi117-63923-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:18.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[28843]: audit 2024-03-04T15:57:17.267417+0000 mon.a (mon.0) 1977 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi117-63923-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:18.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[28843]: cluster 2024-03-04T15:57:18.083043+0000 mon.a (mon.0) 1978 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:18.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[28843]: audit 2024-03-04T15:57:18.087197+0000 mon.a (mon.0) 1979 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi117-63915-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:18.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[28843]: audit 2024-03-04T15:57:18.087399+0000 mon.a (mon.0) 1980 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi117-63923-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:18.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[28843]: cluster 2024-03-04T15:57:18.097920+0000 mon.a (mon.0) 1981 : cluster 0 osdmap e249: 8 total, 8 up, 8 in 2024-03-04T15:57:18.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[21480]: audit 2024-03-04T15:57:18.099542+0000 mon.a (mon.0) 1982 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi117-63915-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi117-63915-37"}]: dispatch 2024-03-04T15:57:18.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[21480]: audit 2024-03-04T15:57:18.100810+0000 mon.c (mon.2) 385 : audit 1 from='client.? 172.21.15.117:0/2345402729' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi117-63923-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:18.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[21480]: audit 2024-03-04T15:57:18.101825+0000 mon.a (mon.0) 1983 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi117-63923-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:18.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[28843]: audit 2024-03-04T15:57:18.099542+0000 mon.a (mon.0) 1982 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi117-63915-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi117-63915-37"}]: dispatch 2024-03-04T15:57:18.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[28843]: audit 2024-03-04T15:57:18.100810+0000 mon.c (mon.2) 385 : audit 1 from='client.? 172.21.15.117:0/2345402729' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi117-63923-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:18.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:18 smithi117 bash[28843]: audit 2024-03-04T15:57:18.101825+0000 mon.a (mon.0) 1983 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi117-63923-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:18.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:18 smithi151 bash[23756]: audit 2024-03-04T15:57:16.624149+0000 mgr.y (mgr.24428) 310 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:57:18.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:18 smithi151 bash[23756]: cluster 2024-03-04T15:57:16.883556+0000 mgr.y (mgr.24428) 311 : cluster 0 pgmap v377: 228 pgs: 32 creating+peering, 196 active+clean; 584 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-04T15:57:18.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:18 smithi151 bash[23756]: audit 2024-03-04T15:57:17.264258+0000 mon.c (mon.2) 382 : audit 1 from='client.? 172.21.15.117:0/2345402729' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:18.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:18 smithi151 bash[23756]: audit 2024-03-04T15:57:17.264644+0000 mon.a (mon.0) 1972 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:18.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:18 smithi151 bash[23756]: audit 2024-03-04T15:57:17.265009+0000 mon.a (mon.0) 1973 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi117-63915-37"}]: dispatch 2024-03-04T15:57:18.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:18 smithi151 bash[23756]: audit 2024-03-04T15:57:17.265848+0000 mon.a (mon.0) 1974 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi117-63915-37"}]: dispatch 2024-03-04T15:57:18.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:18 smithi151 bash[23756]: audit 2024-03-04T15:57:17.266059+0000 mon.c (mon.2) 383 : audit 1 from='client.? 172.21.15.117:0/2345402729' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:18.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:18 smithi151 bash[23756]: audit 2024-03-04T15:57:17.266382+0000 mon.a (mon.0) 1975 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi117-63915-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:18.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:18 smithi151 bash[23756]: audit 2024-03-04T15:57:17.266720+0000 mon.a (mon.0) 1976 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:18.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:18 smithi151 bash[23756]: audit 2024-03-04T15:57:17.267152+0000 mon.c (mon.2) 384 : audit 1 from='client.? 172.21.15.117:0/2345402729' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi117-63923-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:18.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:18 smithi151 bash[23756]: audit 2024-03-04T15:57:17.267417+0000 mon.a (mon.0) 1977 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi117-63923-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:18.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:18 smithi151 bash[23756]: cluster 2024-03-04T15:57:18.083043+0000 mon.a (mon.0) 1978 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:18.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:18 smithi151 bash[23756]: audit 2024-03-04T15:57:18.087197+0000 mon.a (mon.0) 1979 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi117-63915-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:18.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:18 smithi151 bash[23756]: audit 2024-03-04T15:57:18.087399+0000 mon.a (mon.0) 1980 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi117-63923-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:18.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:18 smithi151 bash[23756]: cluster 2024-03-04T15:57:18.097920+0000 mon.a (mon.0) 1981 : cluster 0 osdmap e249: 8 total, 8 up, 8 in 2024-03-04T15:57:18.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:18 smithi151 bash[23756]: audit 2024-03-04T15:57:18.099542+0000 mon.a (mon.0) 1982 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi117-63915-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi117-63915-37"}]: dispatch 2024-03-04T15:57:18.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:18 smithi151 bash[23756]: audit 2024-03-04T15:57:18.100810+0000 mon.c (mon.2) 385 : audit 1 from='client.? 172.21.15.117:0/2345402729' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi117-63923-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:18.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:18 smithi151 bash[23756]: audit 2024-03-04T15:57:18.101825+0000 mon.a (mon.0) 1983 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi117-63923-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:20.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:20 smithi151 bash[23756]: cluster 2024-03-04T15:57:18.884393+0000 mgr.y (mgr.24428) 312 : cluster 0 pgmap v380: 228 pgs: 32 creating+peering, 196 active+clean; 584 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-04T15:57:20.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:20 smithi151 bash[23756]: cluster 2024-03-04T15:57:20.000270+0000 mon.a (mon.0) 1984 : cluster 0 osdmap e250: 8 total, 8 up, 8 in 2024-03-04T15:57:20.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:20 smithi151 bash[23756]: audit 2024-03-04T15:57:20.008865+0000 mon.c (mon.2) 386 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:20.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:20 smithi151 bash[23756]: audit 2024-03-04T15:57:20.009514+0000 mon.a (mon.0) 1985 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:20.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:20 smithi117 bash[21480]: cluster 2024-03-04T15:57:18.884393+0000 mgr.y (mgr.24428) 312 : cluster 0 pgmap v380: 228 pgs: 32 creating+peering, 196 active+clean; 584 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-04T15:57:20.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:20 smithi117 bash[21480]: cluster 2024-03-04T15:57:20.000270+0000 mon.a (mon.0) 1984 : cluster 0 osdmap e250: 8 total, 8 up, 8 in 2024-03-04T15:57:20.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:20 smithi117 bash[21480]: audit 2024-03-04T15:57:20.008865+0000 mon.c (mon.2) 386 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:20.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:20 smithi117 bash[21480]: audit 2024-03-04T15:57:20.009514+0000 mon.a (mon.0) 1985 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:20.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:20 smithi117 bash[28843]: cluster 2024-03-04T15:57:18.884393+0000 mgr.y (mgr.24428) 312 : cluster 0 pgmap v380: 228 pgs: 32 creating+peering, 196 active+clean; 584 KiB data, 616 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-04T15:57:20.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:20 smithi117 bash[28843]: cluster 2024-03-04T15:57:20.000270+0000 mon.a (mon.0) 1984 : cluster 0 osdmap e250: 8 total, 8 up, 8 in 2024-03-04T15:57:20.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:20 smithi117 bash[28843]: audit 2024-03-04T15:57:20.008865+0000 mon.c (mon.2) 386 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:20.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:20 smithi117 bash[28843]: audit 2024-03-04T15:57:20.009514+0000 mon.a (mon.0) 1985 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:21.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:21 smithi151 bash[23756]: audit 2024-03-04T15:57:20.101037+0000 mon.a (mon.0) 1986 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi117-63915-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi117-63915-37"}]': finished 2024-03-04T15:57:21.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:21 smithi151 bash[23756]: audit 2024-03-04T15:57:20.101361+0000 mon.a (mon.0) 1987 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi117-63923-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi117-63923-47"}]': finished 2024-03-04T15:57:21.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:21 smithi151 bash[23756]: audit 2024-03-04T15:57:20.101586+0000 mon.a (mon.0) 1988 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-29", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:57:21.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:21 smithi151 bash[23756]: cluster 2024-03-04T15:57:20.108453+0000 mon.a (mon.0) 1989 : cluster 0 osdmap e251: 8 total, 8 up, 8 in 2024-03-04T15:57:21.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:21 smithi151 bash[23756]: audit 2024-03-04T15:57:20.114203+0000 mon.c (mon.2) 387 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-29"}]: dispatch 2024-03-04T15:57:21.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:21 smithi151 bash[23756]: audit 2024-03-04T15:57:20.123824+0000 mon.a (mon.0) 1990 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-29"}]: dispatch 2024-03-04T15:57:21.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:21 smithi117 bash[21480]: audit 2024-03-04T15:57:20.101037+0000 mon.a (mon.0) 1986 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi117-63915-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi117-63915-37"}]': finished 2024-03-04T15:57:21.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:21 smithi117 bash[21480]: audit 2024-03-04T15:57:20.101361+0000 mon.a (mon.0) 1987 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi117-63923-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi117-63923-47"}]': finished 2024-03-04T15:57:21.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:21 smithi117 bash[21480]: audit 2024-03-04T15:57:20.101586+0000 mon.a (mon.0) 1988 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-29", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:57:21.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:21 smithi117 bash[21480]: cluster 2024-03-04T15:57:20.108453+0000 mon.a (mon.0) 1989 : cluster 0 osdmap e251: 8 total, 8 up, 8 in 2024-03-04T15:57:21.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:21 smithi117 bash[21480]: audit 2024-03-04T15:57:20.114203+0000 mon.c (mon.2) 387 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-29"}]: dispatch 2024-03-04T15:57:21.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:21 smithi117 bash[21480]: audit 2024-03-04T15:57:20.123824+0000 mon.a (mon.0) 1990 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-29"}]: dispatch 2024-03-04T15:57:21.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:21 smithi117 bash[28843]: audit 2024-03-04T15:57:20.101037+0000 mon.a (mon.0) 1986 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi117-63915-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi117-63915-37"}]': finished 2024-03-04T15:57:21.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:21 smithi117 bash[28843]: audit 2024-03-04T15:57:20.101361+0000 mon.a (mon.0) 1987 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi117-63923-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi117-63923-47"}]': finished 2024-03-04T15:57:21.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:21 smithi117 bash[28843]: audit 2024-03-04T15:57:20.101586+0000 mon.a (mon.0) 1988 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-29", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:57:21.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:21 smithi117 bash[28843]: cluster 2024-03-04T15:57:20.108453+0000 mon.a (mon.0) 1989 : cluster 0 osdmap e251: 8 total, 8 up, 8 in 2024-03-04T15:57:21.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:21 smithi117 bash[28843]: audit 2024-03-04T15:57:20.114203+0000 mon.c (mon.2) 387 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-29"}]: dispatch 2024-03-04T15:57:21.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:21 smithi117 bash[28843]: audit 2024-03-04T15:57:20.123824+0000 mon.a (mon.0) 1990 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-29"}]: dispatch 2024-03-04T15:57:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:22 smithi151 bash[23756]: cluster 2024-03-04T15:57:20.885348+0000 mgr.y (mgr.24428) 313 : cluster 0 pgmap v383: 244 pgs: 16 unknown, 32 creating+peering, 196 active+clean; 584 KiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:57:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:22 smithi151 bash[23756]: audit 2024-03-04T15:57:21.104187+0000 mon.a (mon.0) 1991 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-29"}]': finished 2024-03-04T15:57:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:22 smithi151 bash[23756]: cluster 2024-03-04T15:57:21.112454+0000 mon.a (mon.0) 1992 : cluster 0 osdmap e252: 8 total, 8 up, 8 in 2024-03-04T15:57:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:22 smithi151 bash[23756]: audit 2024-03-04T15:57:21.123658+0000 mon.c (mon.2) 388 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-29", "mode": "writeback"}]: dispatch 2024-03-04T15:57:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:22 smithi151 bash[23756]: audit 2024-03-04T15:57:21.147272+0000 mon.a (mon.0) 1993 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-29", "mode": "writeback"}]: dispatch 2024-03-04T15:57:22.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:22 smithi117 bash[28843]: cluster 2024-03-04T15:57:20.885348+0000 mgr.y (mgr.24428) 313 : cluster 0 pgmap v383: 244 pgs: 16 unknown, 32 creating+peering, 196 active+clean; 584 KiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:57:22.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:22 smithi117 bash[28843]: audit 2024-03-04T15:57:21.104187+0000 mon.a (mon.0) 1991 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-29"}]': finished 2024-03-04T15:57:22.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:22 smithi117 bash[28843]: cluster 2024-03-04T15:57:21.112454+0000 mon.a (mon.0) 1992 : cluster 0 osdmap e252: 8 total, 8 up, 8 in 2024-03-04T15:57:22.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:22 smithi117 bash[28843]: audit 2024-03-04T15:57:21.123658+0000 mon.c (mon.2) 388 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-29", "mode": "writeback"}]: dispatch 2024-03-04T15:57:22.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:22 smithi117 bash[28843]: audit 2024-03-04T15:57:21.147272+0000 mon.a (mon.0) 1993 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-29", "mode": "writeback"}]: dispatch 2024-03-04T15:57:22.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:22 smithi117 bash[21480]: cluster 2024-03-04T15:57:20.885348+0000 mgr.y (mgr.24428) 313 : cluster 0 pgmap v383: 244 pgs: 16 unknown, 32 creating+peering, 196 active+clean; 584 KiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:57:22.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:22 smithi117 bash[21480]: audit 2024-03-04T15:57:21.104187+0000 mon.a (mon.0) 1991 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-29"}]': finished 2024-03-04T15:57:22.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:22 smithi117 bash[21480]: cluster 2024-03-04T15:57:21.112454+0000 mon.a (mon.0) 1992 : cluster 0 osdmap e252: 8 total, 8 up, 8 in 2024-03-04T15:57:22.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:22 smithi117 bash[21480]: audit 2024-03-04T15:57:21.123658+0000 mon.c (mon.2) 388 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-29", "mode": "writeback"}]: dispatch 2024-03-04T15:57:22.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:22 smithi117 bash[21480]: audit 2024-03-04T15:57:21.147272+0000 mon.a (mon.0) 1993 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-29", "mode": "writeback"}]: dispatch 2024-03-04T15:57:23.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:23 smithi151 bash[23756]: cluster 2024-03-04T15:57:22.127113+0000 mon.a (mon.0) 1994 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:57:23.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:23 smithi151 bash[23756]: audit 2024-03-04T15:57:22.129583+0000 mon.a (mon.0) 1995 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-29", "mode": "writeback"}]': finished 2024-03-04T15:57:23.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:23 smithi151 bash[23756]: cluster 2024-03-04T15:57:22.138707+0000 mon.a (mon.0) 1996 : cluster 0 osdmap e253: 8 total, 8 up, 8 in 2024-03-04T15:57:23.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:23 smithi151 bash[23756]: audit 2024-03-04T15:57:22.142896+0000 mon.a (mon.0) 1997 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi117-63915-37"}]: dispatch 2024-03-04T15:57:23.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:23 smithi151 bash[23756]: audit 2024-03-04T15:57:22.157629+0000 mon.c (mon.2) 389 : audit 1 from='client.? 172.21.15.117:0/2345402729' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:23.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:23 smithi151 bash[23756]: audit 2024-03-04T15:57:22.158173+0000 mon.a (mon.0) 1998 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:23.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:23 smithi151 bash[23756]: cluster 2024-03-04T15:57:23.086221+0000 mon.a (mon.0) 1999 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:23.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:23 smithi151 bash[23756]: audit 2024-03-04T15:57:23.135524+0000 mon.a (mon.0) 2000 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi117-63915-37"}]': finished 2024-03-04T15:57:23.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:23 smithi117 bash[21480]: cluster 2024-03-04T15:57:22.127113+0000 mon.a (mon.0) 1994 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:57:23.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:23 smithi117 bash[21480]: audit 2024-03-04T15:57:22.129583+0000 mon.a (mon.0) 1995 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-29", "mode": "writeback"}]': finished 2024-03-04T15:57:23.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:23 smithi117 bash[21480]: cluster 2024-03-04T15:57:22.138707+0000 mon.a (mon.0) 1996 : cluster 0 osdmap e253: 8 total, 8 up, 8 in 2024-03-04T15:57:23.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:23 smithi117 bash[21480]: audit 2024-03-04T15:57:22.142896+0000 mon.a (mon.0) 1997 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi117-63915-37"}]: dispatch 2024-03-04T15:57:23.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:23 smithi117 bash[21480]: audit 2024-03-04T15:57:22.157629+0000 mon.c (mon.2) 389 : audit 1 from='client.? 172.21.15.117:0/2345402729' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:23.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:23 smithi117 bash[21480]: audit 2024-03-04T15:57:22.158173+0000 mon.a (mon.0) 1998 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:23.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:23 smithi117 bash[21480]: cluster 2024-03-04T15:57:23.086221+0000 mon.a (mon.0) 1999 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:23.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:23 smithi117 bash[21480]: audit 2024-03-04T15:57:23.135524+0000 mon.a (mon.0) 2000 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi117-63915-37"}]': finished 2024-03-04T15:57:23.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:23 smithi117 bash[28843]: cluster 2024-03-04T15:57:22.127113+0000 mon.a (mon.0) 1994 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:57:23.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:23 smithi117 bash[28843]: audit 2024-03-04T15:57:22.129583+0000 mon.a (mon.0) 1995 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-29", "mode": "writeback"}]': finished 2024-03-04T15:57:23.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:23 smithi117 bash[28843]: cluster 2024-03-04T15:57:22.138707+0000 mon.a (mon.0) 1996 : cluster 0 osdmap e253: 8 total, 8 up, 8 in 2024-03-04T15:57:23.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:23 smithi117 bash[28843]: audit 2024-03-04T15:57:22.142896+0000 mon.a (mon.0) 1997 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi117-63915-37"}]: dispatch 2024-03-04T15:57:23.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:23 smithi117 bash[28843]: audit 2024-03-04T15:57:22.157629+0000 mon.c (mon.2) 389 : audit 1 from='client.? 172.21.15.117:0/2345402729' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:23.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:23 smithi117 bash[28843]: audit 2024-03-04T15:57:22.158173+0000 mon.a (mon.0) 1998 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:23.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:23 smithi117 bash[28843]: cluster 2024-03-04T15:57:23.086221+0000 mon.a (mon.0) 1999 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:23.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:23 smithi117 bash[28843]: audit 2024-03-04T15:57:23.135524+0000 mon.a (mon.0) 2000 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi117-63915-37"}]': finished 2024-03-04T15:57:24.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:24 smithi151 bash[23756]: cluster 2024-03-04T15:57:22.886168+0000 mgr.y (mgr.24428) 314 : cluster 0 pgmap v386: 228 pgs: 228 active+clean; 4.6 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-04T15:57:24.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:24 smithi151 bash[23756]: audit 2024-03-04T15:57:23.135690+0000 mon.a (mon.0) 2001 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-47"}]': finished 2024-03-04T15:57:24.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:24 smithi151 bash[23756]: cluster 2024-03-04T15:57:23.149829+0000 mon.a (mon.0) 2002 : cluster 0 osdmap e254: 8 total, 8 up, 8 in 2024-03-04T15:57:24.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:24 smithi151 bash[23756]: audit 2024-03-04T15:57:23.152420+0000 mon.a (mon.0) 2003 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi117-63915-37"}]: dispatch 2024-03-04T15:57:24.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:24 smithi151 bash[23756]: audit 2024-03-04T15:57:23.156290+0000 mon.c (mon.2) 390 : audit 1 from='client.? 172.21.15.117:0/2345402729' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:24.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:24 smithi151 bash[23756]: audit 2024-03-04T15:57:23.165451+0000 mon.a (mon.0) 2004 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:24.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:24 smithi151 bash[23756]: audit 2024-03-04T15:57:24.141061+0000 mon.a (mon.0) 2005 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi117-63915-37"}]': finished 2024-03-04T15:57:24.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:24 smithi151 bash[23756]: audit 2024-03-04T15:57:24.141298+0000 mon.a (mon.0) 2006 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-47"}]': finished 2024-03-04T15:57:24.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:24 smithi151 bash[23756]: cluster 2024-03-04T15:57:24.148945+0000 mon.a (mon.0) 2007 : cluster 0 osdmap e255: 8 total, 8 up, 8 in 2024-03-04T15:57:24.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:24 smithi117 bash[21480]: cluster 2024-03-04T15:57:22.886168+0000 mgr.y (mgr.24428) 314 : cluster 0 pgmap v386: 228 pgs: 228 active+clean; 4.6 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-04T15:57:24.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:24 smithi117 bash[21480]: audit 2024-03-04T15:57:23.135690+0000 mon.a (mon.0) 2001 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-47"}]': finished 2024-03-04T15:57:24.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:24 smithi117 bash[21480]: cluster 2024-03-04T15:57:23.149829+0000 mon.a (mon.0) 2002 : cluster 0 osdmap e254: 8 total, 8 up, 8 in 2024-03-04T15:57:24.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:24 smithi117 bash[21480]: audit 2024-03-04T15:57:23.152420+0000 mon.a (mon.0) 2003 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi117-63915-37"}]: dispatch 2024-03-04T15:57:24.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:24 smithi117 bash[21480]: audit 2024-03-04T15:57:23.156290+0000 mon.c (mon.2) 390 : audit 1 from='client.? 172.21.15.117:0/2345402729' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:24.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:24 smithi117 bash[21480]: audit 2024-03-04T15:57:23.165451+0000 mon.a (mon.0) 2004 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:24.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:24 smithi117 bash[21480]: audit 2024-03-04T15:57:24.141061+0000 mon.a (mon.0) 2005 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi117-63915-37"}]': finished 2024-03-04T15:57:24.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:24 smithi117 bash[21480]: audit 2024-03-04T15:57:24.141298+0000 mon.a (mon.0) 2006 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-47"}]': finished 2024-03-04T15:57:24.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:24 smithi117 bash[21480]: cluster 2024-03-04T15:57:24.148945+0000 mon.a (mon.0) 2007 : cluster 0 osdmap e255: 8 total, 8 up, 8 in 2024-03-04T15:57:24.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:24 smithi117 bash[28843]: cluster 2024-03-04T15:57:22.886168+0000 mgr.y (mgr.24428) 314 : cluster 0 pgmap v386: 228 pgs: 228 active+clean; 4.6 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-04T15:57:24.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:24 smithi117 bash[28843]: audit 2024-03-04T15:57:23.135690+0000 mon.a (mon.0) 2001 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi117-63923-47"}]': finished 2024-03-04T15:57:24.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:24 smithi117 bash[28843]: cluster 2024-03-04T15:57:23.149829+0000 mon.a (mon.0) 2002 : cluster 0 osdmap e254: 8 total, 8 up, 8 in 2024-03-04T15:57:24.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:24 smithi117 bash[28843]: audit 2024-03-04T15:57:23.152420+0000 mon.a (mon.0) 2003 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi117-63915-37"}]: dispatch 2024-03-04T15:57:24.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:24 smithi117 bash[28843]: audit 2024-03-04T15:57:23.156290+0000 mon.c (mon.2) 390 : audit 1 from='client.? 172.21.15.117:0/2345402729' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:24.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:24 smithi117 bash[28843]: audit 2024-03-04T15:57:23.165451+0000 mon.a (mon.0) 2004 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-47"}]: dispatch 2024-03-04T15:57:24.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:24 smithi117 bash[28843]: audit 2024-03-04T15:57:24.141061+0000 mon.a (mon.0) 2005 : audit 1 from='client.? 172.21.15.117:0/812761474' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi117-63915-37"}]': finished 2024-03-04T15:57:24.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:24 smithi117 bash[28843]: audit 2024-03-04T15:57:24.141298+0000 mon.a (mon.0) 2006 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi117-63923-47"}]': finished 2024-03-04T15:57:24.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:24 smithi117 bash[28843]: cluster 2024-03-04T15:57:24.148945+0000 mon.a (mon.0) 2007 : cluster 0 osdmap e255: 8 total, 8 up, 8 in 2024-03-04T15:57:25.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:25 smithi151 bash[23756]: audit 2024-03-04T15:57:24.201920+0000 mon.c (mon.2) 391 : audit 1 from='client.? 172.21.15.117:0/2062509334' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:25.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:25 smithi151 bash[23756]: audit 2024-03-04T15:57:24.202349+0000 mon.a (mon.0) 2008 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:25.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:25 smithi151 bash[23756]: audit 2024-03-04T15:57:24.203832+0000 mon.c (mon.2) 392 : audit 1 from='client.? 172.21.15.117:0/2062509334' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:25.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:25 smithi151 bash[23756]: audit 2024-03-04T15:57:24.204187+0000 mon.a (mon.0) 2009 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:25.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:25 smithi151 bash[23756]: audit 2024-03-04T15:57:24.204409+0000 mon.c (mon.2) 393 : audit 1 from='client.? 172.21.15.117:0/2356338718' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:25.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:25 smithi151 bash[23756]: audit 2024-03-04T15:57:24.204791+0000 mon.a (mon.0) 2010 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:25.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:25 smithi151 bash[23756]: audit 2024-03-04T15:57:24.204981+0000 mon.c (mon.2) 394 : audit 1 from='client.? 172.21.15.117:0/2062509334' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi117-63915-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:25.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:25 smithi151 bash[23756]: audit 2024-03-04T15:57:24.205425+0000 mon.a (mon.0) 2011 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi117-63915-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:25.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:25 smithi151 bash[23756]: audit 2024-03-04T15:57:24.205782+0000 mon.c (mon.2) 395 : audit 1 from='client.? 172.21.15.117:0/2356338718' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:25.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:25 smithi151 bash[23756]: audit 2024-03-04T15:57:24.206066+0000 mon.a (mon.0) 2012 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:25.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:25 smithi151 bash[23756]: audit 2024-03-04T15:57:24.206611+0000 mon.c (mon.2) 396 : audit 1 from='client.? 172.21.15.117:0/2356338718' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi117-63923-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:25.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:25 smithi151 bash[23756]: audit 2024-03-04T15:57:24.206893+0000 mon.a (mon.0) 2013 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi117-63923-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:25.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:25 smithi151 bash[23756]: audit 2024-03-04T15:57:25.143643+0000 mon.a (mon.0) 2014 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi117-63915-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:25.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:25 smithi151 bash[23756]: audit 2024-03-04T15:57:25.143931+0000 mon.a (mon.0) 2015 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi117-63923-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:25.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:25 smithi151 bash[23756]: cluster 2024-03-04T15:57:25.152650+0000 mon.a (mon.0) 2016 : cluster 0 osdmap e256: 8 total, 8 up, 8 in 2024-03-04T15:57:25.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:25 smithi151 bash[23756]: audit 2024-03-04T15:57:25.157805+0000 mon.c (mon.2) 397 : audit 1 from='client.? 172.21.15.117:0/2062509334' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi117-63915-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:25.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:25 smithi151 bash[23756]: audit 2024-03-04T15:57:25.158848+0000 mon.c (mon.2) 398 : audit 1 from='client.? 172.21.15.117:0/2356338718' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi117-63923-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:25.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:25 smithi151 bash[23756]: audit 2024-03-04T15:57:25.159588+0000 mon.a (mon.0) 2017 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi117-63915-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:25.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:25 smithi151 bash[23756]: audit 2024-03-04T15:57:25.161328+0000 mon.a (mon.0) 2018 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi117-63923-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:25.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[21480]: audit 2024-03-04T15:57:24.201920+0000 mon.c (mon.2) 391 : audit 1 from='client.? 172.21.15.117:0/2062509334' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:25.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[21480]: audit 2024-03-04T15:57:24.202349+0000 mon.a (mon.0) 2008 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:25.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[21480]: audit 2024-03-04T15:57:24.203832+0000 mon.c (mon.2) 392 : audit 1 from='client.? 172.21.15.117:0/2062509334' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:25.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[21480]: audit 2024-03-04T15:57:24.204187+0000 mon.a (mon.0) 2009 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:25.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[21480]: audit 2024-03-04T15:57:24.204409+0000 mon.c (mon.2) 393 : audit 1 from='client.? 172.21.15.117:0/2356338718' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:25.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[21480]: audit 2024-03-04T15:57:24.204791+0000 mon.a (mon.0) 2010 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:25.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[21480]: audit 2024-03-04T15:57:24.204981+0000 mon.c (mon.2) 394 : audit 1 from='client.? 172.21.15.117:0/2062509334' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi117-63915-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:25.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[21480]: audit 2024-03-04T15:57:24.205425+0000 mon.a (mon.0) 2011 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi117-63915-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:25.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[21480]: audit 2024-03-04T15:57:24.205782+0000 mon.c (mon.2) 395 : audit 1 from='client.? 172.21.15.117:0/2356338718' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:25.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[21480]: audit 2024-03-04T15:57:24.206066+0000 mon.a (mon.0) 2012 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:25.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[21480]: audit 2024-03-04T15:57:24.206611+0000 mon.c (mon.2) 396 : audit 1 from='client.? 172.21.15.117:0/2356338718' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi117-63923-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:25.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[21480]: audit 2024-03-04T15:57:24.206893+0000 mon.a (mon.0) 2013 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi117-63923-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:25.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[21480]: audit 2024-03-04T15:57:25.143643+0000 mon.a (mon.0) 2014 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi117-63915-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:25.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[21480]: audit 2024-03-04T15:57:25.143931+0000 mon.a (mon.0) 2015 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi117-63923-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:25.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[28843]: audit 2024-03-04T15:57:24.201920+0000 mon.c (mon.2) 391 : audit 1 from='client.? 172.21.15.117:0/2062509334' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:25.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[28843]: audit 2024-03-04T15:57:24.202349+0000 mon.a (mon.0) 2008 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:25.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[28843]: audit 2024-03-04T15:57:24.203832+0000 mon.c (mon.2) 392 : audit 1 from='client.? 172.21.15.117:0/2062509334' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:25.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[28843]: audit 2024-03-04T15:57:24.204187+0000 mon.a (mon.0) 2009 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:25.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[28843]: audit 2024-03-04T15:57:24.204409+0000 mon.c (mon.2) 393 : audit 1 from='client.? 172.21.15.117:0/2356338718' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:25.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[28843]: audit 2024-03-04T15:57:24.204791+0000 mon.a (mon.0) 2010 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:25.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[28843]: audit 2024-03-04T15:57:24.204981+0000 mon.c (mon.2) 394 : audit 1 from='client.? 172.21.15.117:0/2062509334' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi117-63915-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:25.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[28843]: audit 2024-03-04T15:57:24.205425+0000 mon.a (mon.0) 2011 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi117-63915-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:25.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[28843]: audit 2024-03-04T15:57:24.205782+0000 mon.c (mon.2) 395 : audit 1 from='client.? 172.21.15.117:0/2356338718' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:25.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[28843]: audit 2024-03-04T15:57:24.206066+0000 mon.a (mon.0) 2012 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:25.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[28843]: audit 2024-03-04T15:57:24.206611+0000 mon.c (mon.2) 396 : audit 1 from='client.? 172.21.15.117:0/2356338718' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi117-63923-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:25.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[28843]: audit 2024-03-04T15:57:24.206893+0000 mon.a (mon.0) 2013 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi117-63923-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:25.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[28843]: audit 2024-03-04T15:57:25.143643+0000 mon.a (mon.0) 2014 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi117-63915-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:25.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[28843]: audit 2024-03-04T15:57:25.143931+0000 mon.a (mon.0) 2015 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi117-63923-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:25.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[21480]: cluster 2024-03-04T15:57:25.152650+0000 mon.a (mon.0) 2016 : cluster 0 osdmap e256: 8 total, 8 up, 8 in 2024-03-04T15:57:25.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[21480]: audit 2024-03-04T15:57:25.157805+0000 mon.c (mon.2) 397 : audit 1 from='client.? 172.21.15.117:0/2062509334' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi117-63915-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:25.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[21480]: audit 2024-03-04T15:57:25.158848+0000 mon.c (mon.2) 398 : audit 1 from='client.? 172.21.15.117:0/2356338718' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi117-63923-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:25.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[21480]: audit 2024-03-04T15:57:25.159588+0000 mon.a (mon.0) 2017 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi117-63915-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:25.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[21480]: audit 2024-03-04T15:57:25.161328+0000 mon.a (mon.0) 2018 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi117-63923-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:25.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[28843]: cluster 2024-03-04T15:57:25.152650+0000 mon.a (mon.0) 2016 : cluster 0 osdmap e256: 8 total, 8 up, 8 in 2024-03-04T15:57:25.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[28843]: audit 2024-03-04T15:57:25.157805+0000 mon.c (mon.2) 397 : audit 1 from='client.? 172.21.15.117:0/2062509334' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi117-63915-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:25.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[28843]: audit 2024-03-04T15:57:25.158848+0000 mon.c (mon.2) 398 : audit 1 from='client.? 172.21.15.117:0/2356338718' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi117-63923-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:25.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[28843]: audit 2024-03-04T15:57:25.159588+0000 mon.a (mon.0) 2017 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi117-63915-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:25.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:25 smithi117 bash[28843]: audit 2024-03-04T15:57:25.161328+0000 mon.a (mon.0) 2018 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi117-63923-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:26.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:57:26 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:57:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:57:26.931 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:57:26 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:57:27.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:27 smithi117 bash[21480]: cluster 2024-03-04T15:57:24.887022+0000 mgr.y (mgr.24428) 315 : cluster 0 pgmap v389: 228 pgs: 228 active+clean; 4.6 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-04T15:57:27.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:27 smithi117 bash[21480]: audit 2024-03-04T15:57:25.264222+0000 mon.c (mon.2) 399 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:27.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:27 smithi117 bash[21480]: audit 2024-03-04T15:57:25.264863+0000 mon.a (mon.0) 2019 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:27.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:27 smithi117 bash[21480]: audit 2024-03-04T15:57:26.147216+0000 mon.a (mon.0) 2020 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:27.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:27 smithi117 bash[28843]: cluster 2024-03-04T15:57:24.887022+0000 mgr.y (mgr.24428) 315 : cluster 0 pgmap v389: 228 pgs: 228 active+clean; 4.6 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-04T15:57:27.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:27 smithi117 bash[28843]: audit 2024-03-04T15:57:25.264222+0000 mon.c (mon.2) 399 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:27.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:27 smithi117 bash[28843]: audit 2024-03-04T15:57:25.264863+0000 mon.a (mon.0) 2019 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:27.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:27 smithi117 bash[28843]: audit 2024-03-04T15:57:26.147216+0000 mon.a (mon.0) 2020 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:27.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:27 smithi151 bash[23756]: cluster 2024-03-04T15:57:24.887022+0000 mgr.y (mgr.24428) 315 : cluster 0 pgmap v389: 228 pgs: 228 active+clean; 4.6 MiB data, 633 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1024 KiB/s wr, 1 op/s 2024-03-04T15:57:27.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:27 smithi151 bash[23756]: audit 2024-03-04T15:57:25.264222+0000 mon.c (mon.2) 399 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:27.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:27 smithi151 bash[23756]: audit 2024-03-04T15:57:25.264863+0000 mon.a (mon.0) 2019 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:27.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:27 smithi151 bash[23756]: audit 2024-03-04T15:57:26.147216+0000 mon.a (mon.0) 2020 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:28 smithi117 bash[28843]: audit 2024-03-04T15:57:26.158630+0000 mon.c (mon.2) 400 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-29"}]: dispatch 2024-03-04T15:57:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:28 smithi117 bash[28843]: audit 2024-03-04T15:57:26.634957+0000 mgr.y (mgr.24428) 316 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:57:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:28 smithi117 bash[28843]: cluster 2024-03-04T15:57:26.889180+0000 mgr.y (mgr.24428) 317 : cluster 0 pgmap v392: 228 pgs: 228 active+clean; 4.6 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-03-04T15:57:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:28 smithi117 bash[28843]: cluster 2024-03-04T15:57:27.043931+0000 mon.a (mon.0) 2021 : cluster 0 osdmap e257: 8 total, 8 up, 8 in 2024-03-04T15:57:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:28 smithi117 bash[28843]: audit 2024-03-04T15:57:27.045864+0000 mon.a (mon.0) 2022 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-29"}]: dispatch 2024-03-04T15:57:28.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:28 smithi117 bash[28843]: cluster 2024-03-04T15:57:27.148753+0000 mon.a (mon.0) 2023 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:57:28.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:28 smithi117 bash[28843]: audit 2024-03-04T15:57:27.156449+0000 mon.a (mon.0) 2024 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi117-63915-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi117-63915-38"}]': finished 2024-03-04T15:57:28.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:28 smithi117 bash[28843]: audit 2024-03-04T15:57:27.156736+0000 mon.a (mon.0) 2025 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi117-63923-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi117-63923-48"}]': finished 2024-03-04T15:57:28.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:28 smithi117 bash[28843]: audit 2024-03-04T15:57:27.156962+0000 mon.a (mon.0) 2026 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-29"}]': finished 2024-03-04T15:57:28.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:28 smithi117 bash[28843]: cluster 2024-03-04T15:57:27.164997+0000 mon.a (mon.0) 2027 : cluster 0 osdmap e258: 8 total, 8 up, 8 in 2024-03-04T15:57:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:28 smithi117 bash[21480]: audit 2024-03-04T15:57:26.158630+0000 mon.c (mon.2) 400 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-29"}]: dispatch 2024-03-04T15:57:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:28 smithi117 bash[21480]: audit 2024-03-04T15:57:26.634957+0000 mgr.y (mgr.24428) 316 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:57:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:28 smithi117 bash[21480]: cluster 2024-03-04T15:57:26.889180+0000 mgr.y (mgr.24428) 317 : cluster 0 pgmap v392: 228 pgs: 228 active+clean; 4.6 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-03-04T15:57:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:28 smithi117 bash[21480]: cluster 2024-03-04T15:57:27.043931+0000 mon.a (mon.0) 2021 : cluster 0 osdmap e257: 8 total, 8 up, 8 in 2024-03-04T15:57:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:28 smithi117 bash[21480]: audit 2024-03-04T15:57:27.045864+0000 mon.a (mon.0) 2022 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-29"}]: dispatch 2024-03-04T15:57:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:28 smithi117 bash[21480]: cluster 2024-03-04T15:57:27.148753+0000 mon.a (mon.0) 2023 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:57:28.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:28 smithi117 bash[21480]: audit 2024-03-04T15:57:27.156449+0000 mon.a (mon.0) 2024 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi117-63915-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi117-63915-38"}]': finished 2024-03-04T15:57:28.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:28 smithi117 bash[21480]: audit 2024-03-04T15:57:27.156736+0000 mon.a (mon.0) 2025 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi117-63923-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi117-63923-48"}]': finished 2024-03-04T15:57:28.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:28 smithi117 bash[21480]: audit 2024-03-04T15:57:27.156962+0000 mon.a (mon.0) 2026 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-29"}]': finished 2024-03-04T15:57:28.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:28 smithi117 bash[21480]: cluster 2024-03-04T15:57:27.164997+0000 mon.a (mon.0) 2027 : cluster 0 osdmap e258: 8 total, 8 up, 8 in 2024-03-04T15:57:28.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:28 smithi151 bash[23756]: audit 2024-03-04T15:57:26.158630+0000 mon.c (mon.2) 400 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-29"}]: dispatch 2024-03-04T15:57:28.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:28 smithi151 bash[23756]: audit 2024-03-04T15:57:26.634957+0000 mgr.y (mgr.24428) 316 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:57:28.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:28 smithi151 bash[23756]: cluster 2024-03-04T15:57:26.889180+0000 mgr.y (mgr.24428) 317 : cluster 0 pgmap v392: 228 pgs: 228 active+clean; 4.6 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 14 KiB/s rd, 1024 KiB/s wr, 27 op/s 2024-03-04T15:57:28.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:28 smithi151 bash[23756]: cluster 2024-03-04T15:57:27.043931+0000 mon.a (mon.0) 2021 : cluster 0 osdmap e257: 8 total, 8 up, 8 in 2024-03-04T15:57:28.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:28 smithi151 bash[23756]: audit 2024-03-04T15:57:27.045864+0000 mon.a (mon.0) 2022 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-29"}]: dispatch 2024-03-04T15:57:28.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:28 smithi151 bash[23756]: cluster 2024-03-04T15:57:27.148753+0000 mon.a (mon.0) 2023 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:57:28.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:28 smithi151 bash[23756]: audit 2024-03-04T15:57:27.156449+0000 mon.a (mon.0) 2024 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi117-63915-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi117-63915-38"}]': finished 2024-03-04T15:57:28.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:28 smithi151 bash[23756]: audit 2024-03-04T15:57:27.156736+0000 mon.a (mon.0) 2025 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi117-63923-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi117-63923-48"}]': finished 2024-03-04T15:57:28.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:28 smithi151 bash[23756]: audit 2024-03-04T15:57:27.156962+0000 mon.a (mon.0) 2026 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-29"}]': finished 2024-03-04T15:57:28.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:28 smithi151 bash[23756]: cluster 2024-03-04T15:57:27.164997+0000 mon.a (mon.0) 2027 : cluster 0 osdmap e258: 8 total, 8 up, 8 in 2024-03-04T15:57:29.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:29 smithi117 bash[28843]: cluster 2024-03-04T15:57:28.087450+0000 mon.a (mon.0) 2028 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:29.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:29 smithi117 bash[28843]: cluster 2024-03-04T15:57:28.101622+0000 mon.a (mon.0) 2029 : cluster 0 osdmap e259: 8 total, 8 up, 8 in 2024-03-04T15:57:29.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:29 smithi117 bash[28843]: audit 2024-03-04T15:57:28.658570+0000 mon.a (mon.0) 2030 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:57:29.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:29 smithi117 bash[28843]: audit 2024-03-04T15:57:28.661390+0000 mon.c (mon.2) 401 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:57:29.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:29 smithi117 bash[21480]: cluster 2024-03-04T15:57:28.087450+0000 mon.a (mon.0) 2028 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:29.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:29 smithi117 bash[21480]: cluster 2024-03-04T15:57:28.101622+0000 mon.a (mon.0) 2029 : cluster 0 osdmap e259: 8 total, 8 up, 8 in 2024-03-04T15:57:29.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:29 smithi117 bash[21480]: audit 2024-03-04T15:57:28.658570+0000 mon.a (mon.0) 2030 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:57:29.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:29 smithi117 bash[21480]: audit 2024-03-04T15:57:28.661390+0000 mon.c (mon.2) 401 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:57:29.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:29 smithi151 bash[23756]: cluster 2024-03-04T15:57:28.087450+0000 mon.a (mon.0) 2028 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:29.434 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:29 smithi151 bash[23756]: cluster 2024-03-04T15:57:28.101622+0000 mon.a (mon.0) 2029 : cluster 0 osdmap e259: 8 total, 8 up, 8 in 2024-03-04T15:57:29.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:29 smithi151 bash[23756]: audit 2024-03-04T15:57:28.658570+0000 mon.a (mon.0) 2030 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:57:29.435 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:29 smithi151 bash[23756]: audit 2024-03-04T15:57:28.661390+0000 mon.c (mon.2) 401 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:57:30.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:30 smithi151 bash[23756]: cluster 2024-03-04T15:57:28.890021+0000 mgr.y (mgr.24428) 318 : cluster 0 pgmap v395: 212 pgs: 16 unknown, 196 active+clean; 4.6 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 KiB/s wr, 1 op/s 2024-03-04T15:57:30.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:30 smithi151 bash[23756]: cluster 2024-03-04T15:57:29.103398+0000 mon.a (mon.0) 2031 : cluster 0 osdmap e260: 8 total, 8 up, 8 in 2024-03-04T15:57:30.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:30 smithi151 bash[23756]: audit 2024-03-04T15:57:29.105961+0000 mon.c (mon.2) 402 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:30.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:30 smithi151 bash[23756]: audit 2024-03-04T15:57:29.106672+0000 mon.c (mon.2) 403 : audit 1 from='client.? 172.21.15.117:0/2356338718' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:30.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:30 smithi151 bash[23756]: audit 2024-03-04T15:57:29.115233+0000 mon.c (mon.2) 404 : audit 1 from='client.? 172.21.15.117:0/2062509334' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:30.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:30 smithi151 bash[23756]: audit 2024-03-04T15:57:29.118398+0000 mon.a (mon.0) 2032 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:30.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:30 smithi151 bash[23756]: audit 2024-03-04T15:57:29.119480+0000 mon.a (mon.0) 2033 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:30.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:30 smithi151 bash[23756]: audit 2024-03-04T15:57:29.120369+0000 mon.a (mon.0) 2034 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:30.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:30 smithi117 bash[21480]: cluster 2024-03-04T15:57:28.890021+0000 mgr.y (mgr.24428) 318 : cluster 0 pgmap v395: 212 pgs: 16 unknown, 196 active+clean; 4.6 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 KiB/s wr, 1 op/s 2024-03-04T15:57:30.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:30 smithi117 bash[21480]: cluster 2024-03-04T15:57:29.103398+0000 mon.a (mon.0) 2031 : cluster 0 osdmap e260: 8 total, 8 up, 8 in 2024-03-04T15:57:30.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:30 smithi117 bash[21480]: audit 2024-03-04T15:57:29.105961+0000 mon.c (mon.2) 402 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:30.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:30 smithi117 bash[21480]: audit 2024-03-04T15:57:29.106672+0000 mon.c (mon.2) 403 : audit 1 from='client.? 172.21.15.117:0/2356338718' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:30.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:30 smithi117 bash[21480]: audit 2024-03-04T15:57:29.115233+0000 mon.c (mon.2) 404 : audit 1 from='client.? 172.21.15.117:0/2062509334' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:30.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:30 smithi117 bash[21480]: audit 2024-03-04T15:57:29.118398+0000 mon.a (mon.0) 2032 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:30.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:30 smithi117 bash[21480]: audit 2024-03-04T15:57:29.119480+0000 mon.a (mon.0) 2033 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:30.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:30 smithi117 bash[21480]: audit 2024-03-04T15:57:29.120369+0000 mon.a (mon.0) 2034 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:30 smithi117 bash[28843]: cluster 2024-03-04T15:57:28.890021+0000 mgr.y (mgr.24428) 318 : cluster 0 pgmap v395: 212 pgs: 16 unknown, 196 active+clean; 4.6 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 KiB/s wr, 1 op/s 2024-03-04T15:57:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:30 smithi117 bash[28843]: cluster 2024-03-04T15:57:29.103398+0000 mon.a (mon.0) 2031 : cluster 0 osdmap e260: 8 total, 8 up, 8 in 2024-03-04T15:57:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:30 smithi117 bash[28843]: audit 2024-03-04T15:57:29.105961+0000 mon.c (mon.2) 402 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:30 smithi117 bash[28843]: audit 2024-03-04T15:57:29.106672+0000 mon.c (mon.2) 403 : audit 1 from='client.? 172.21.15.117:0/2356338718' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:30 smithi117 bash[28843]: audit 2024-03-04T15:57:29.115233+0000 mon.c (mon.2) 404 : audit 1 from='client.? 172.21.15.117:0/2062509334' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:30 smithi117 bash[28843]: audit 2024-03-04T15:57:29.118398+0000 mon.a (mon.0) 2032 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:30 smithi117 bash[28843]: audit 2024-03-04T15:57:29.119480+0000 mon.a (mon.0) 2033 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:30.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:30 smithi117 bash[28843]: audit 2024-03-04T15:57:29.120369+0000 mon.a (mon.0) 2034 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:31.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:31 smithi151 bash[23756]: audit 2024-03-04T15:57:30.098288+0000 mon.a (mon.0) 2035 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:31.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:31 smithi151 bash[23756]: audit 2024-03-04T15:57:30.098485+0000 mon.a (mon.0) 2036 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi117-63923-48"}]': finished 2024-03-04T15:57:31.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:31 smithi151 bash[23756]: audit 2024-03-04T15:57:30.098599+0000 mon.a (mon.0) 2037 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi117-63915-38"}]': finished 2024-03-04T15:57:31.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:31 smithi151 bash[23756]: cluster 2024-03-04T15:57:30.107400+0000 mon.a (mon.0) 2038 : cluster 0 osdmap e261: 8 total, 8 up, 8 in 2024-03-04T15:57:31.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:31 smithi151 bash[23756]: audit 2024-03-04T15:57:30.112055+0000 mon.c (mon.2) 405 : audit 1 from='client.? 172.21.15.117:0/2062509334' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:31.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:31 smithi151 bash[23756]: audit 2024-03-04T15:57:30.112624+0000 mon.c (mon.2) 406 : audit 1 from='client.? 172.21.15.117:0/2356338718' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:31.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:31 smithi151 bash[23756]: audit 2024-03-04T15:57:30.114044+0000 mon.a (mon.0) 2039 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:31.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:31 smithi151 bash[23756]: audit 2024-03-04T15:57:30.114397+0000 mon.c (mon.2) 407 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:31.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:31 smithi151 bash[23756]: audit 2024-03-04T15:57:30.114769+0000 mon.a (mon.0) 2040 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:31.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:31 smithi151 bash[23756]: audit 2024-03-04T15:57:30.131688+0000 mon.a (mon.0) 2041 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:31.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:31 smithi151 bash[23756]: audit 2024-03-04T15:57:31.102015+0000 mon.a (mon.0) 2042 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi117-63915-38"}]': finished 2024-03-04T15:57:31.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:31 smithi151 bash[23756]: audit 2024-03-04T15:57:31.102299+0000 mon.a (mon.0) 2043 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi117-63923-48"}]': finished 2024-03-04T15:57:31.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:31 smithi151 bash[23756]: audit 2024-03-04T15:57:31.102521+0000 mon.a (mon.0) 2044 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-31", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:57:31.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:31 smithi151 bash[23756]: cluster 2024-03-04T15:57:31.110225+0000 mon.a (mon.0) 2045 : cluster 0 osdmap e262: 8 total, 8 up, 8 in 2024-03-04T15:57:31.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[21480]: audit 2024-03-04T15:57:30.098288+0000 mon.a (mon.0) 2035 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:31.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[21480]: audit 2024-03-04T15:57:30.098485+0000 mon.a (mon.0) 2036 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi117-63923-48"}]': finished 2024-03-04T15:57:31.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[21480]: audit 2024-03-04T15:57:30.098599+0000 mon.a (mon.0) 2037 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi117-63915-38"}]': finished 2024-03-04T15:57:31.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[21480]: cluster 2024-03-04T15:57:30.107400+0000 mon.a (mon.0) 2038 : cluster 0 osdmap e261: 8 total, 8 up, 8 in 2024-03-04T15:57:31.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[21480]: audit 2024-03-04T15:57:30.112055+0000 mon.c (mon.2) 405 : audit 1 from='client.? 172.21.15.117:0/2062509334' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:31.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[21480]: audit 2024-03-04T15:57:30.112624+0000 mon.c (mon.2) 406 : audit 1 from='client.? 172.21.15.117:0/2356338718' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:31.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[21480]: audit 2024-03-04T15:57:30.114044+0000 mon.a (mon.0) 2039 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:31.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[21480]: audit 2024-03-04T15:57:30.114397+0000 mon.c (mon.2) 407 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:31.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[21480]: audit 2024-03-04T15:57:30.114769+0000 mon.a (mon.0) 2040 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:31.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[21480]: audit 2024-03-04T15:57:30.131688+0000 mon.a (mon.0) 2041 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:31.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[21480]: audit 2024-03-04T15:57:31.102015+0000 mon.a (mon.0) 2042 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi117-63915-38"}]': finished 2024-03-04T15:57:31.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[21480]: audit 2024-03-04T15:57:31.102299+0000 mon.a (mon.0) 2043 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi117-63923-48"}]': finished 2024-03-04T15:57:31.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[21480]: audit 2024-03-04T15:57:31.102521+0000 mon.a (mon.0) 2044 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-31", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:57:31.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[21480]: cluster 2024-03-04T15:57:31.110225+0000 mon.a (mon.0) 2045 : cluster 0 osdmap e262: 8 total, 8 up, 8 in 2024-03-04T15:57:31.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[28843]: audit 2024-03-04T15:57:30.098288+0000 mon.a (mon.0) 2035 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:31.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[28843]: audit 2024-03-04T15:57:30.098485+0000 mon.a (mon.0) 2036 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi117-63923-48"}]': finished 2024-03-04T15:57:31.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[28843]: audit 2024-03-04T15:57:30.098599+0000 mon.a (mon.0) 2037 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi117-63915-38"}]': finished 2024-03-04T15:57:31.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[28843]: cluster 2024-03-04T15:57:30.107400+0000 mon.a (mon.0) 2038 : cluster 0 osdmap e261: 8 total, 8 up, 8 in 2024-03-04T15:57:31.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[28843]: audit 2024-03-04T15:57:30.112055+0000 mon.c (mon.2) 405 : audit 1 from='client.? 172.21.15.117:0/2062509334' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:31.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[28843]: audit 2024-03-04T15:57:30.112624+0000 mon.c (mon.2) 406 : audit 1 from='client.? 172.21.15.117:0/2356338718' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:31.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[28843]: audit 2024-03-04T15:57:30.114044+0000 mon.a (mon.0) 2039 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi117-63915-38"}]: dispatch 2024-03-04T15:57:31.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[28843]: audit 2024-03-04T15:57:30.114397+0000 mon.c (mon.2) 407 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:31.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[28843]: audit 2024-03-04T15:57:30.114769+0000 mon.a (mon.0) 2040 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi117-63923-48"}]: dispatch 2024-03-04T15:57:31.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[28843]: audit 2024-03-04T15:57:30.131688+0000 mon.a (mon.0) 2041 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:31.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[28843]: audit 2024-03-04T15:57:31.102015+0000 mon.a (mon.0) 2042 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi117-63915-38"}]': finished 2024-03-04T15:57:31.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[28843]: audit 2024-03-04T15:57:31.102299+0000 mon.a (mon.0) 2043 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi117-63923-48"}]': finished 2024-03-04T15:57:31.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[28843]: audit 2024-03-04T15:57:31.102521+0000 mon.a (mon.0) 2044 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-31", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:57:31.575 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:31 smithi117 bash[28843]: cluster 2024-03-04T15:57:31.110225+0000 mon.a (mon.0) 2045 : cluster 0 osdmap e262: 8 total, 8 up, 8 in 2024-03-04T15:57:32.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:32 smithi151 bash[23756]: cluster 2024-03-04T15:57:30.890853+0000 mgr.y (mgr.24428) 319 : cluster 0 pgmap v398: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:57:32.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:32 smithi151 bash[23756]: audit 2024-03-04T15:57:31.120220+0000 mon.c (mon.2) 408 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-31"}]: dispatch 2024-03-04T15:57:32.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:32 smithi151 bash[23756]: audit 2024-03-04T15:57:31.132799+0000 mon.a (mon.0) 2046 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-31"}]: dispatch 2024-03-04T15:57:32.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:32 smithi151 bash[23756]: audit 2024-03-04T15:57:31.169185+0000 mon.c (mon.2) 409 : audit 1 from='client.? 172.21.15.117:0/4101380182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:32.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:32 smithi151 bash[23756]: audit 2024-03-04T15:57:31.169754+0000 mon.a (mon.0) 2047 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:32.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:32 smithi151 bash[23756]: audit 2024-03-04T15:57:31.170289+0000 mon.a (mon.0) 2048 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi117-63923-49"}]: dispatch 2024-03-04T15:57:32.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:32 smithi151 bash[23756]: audit 2024-03-04T15:57:31.170944+0000 mon.c (mon.2) 410 : audit 1 from='client.? 172.21.15.117:0/4101380182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:32.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:32 smithi151 bash[23756]: audit 2024-03-04T15:57:31.171201+0000 mon.a (mon.0) 2049 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi117-63923-49"}]: dispatch 2024-03-04T15:57:32.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:32 smithi151 bash[23756]: audit 2024-03-04T15:57:31.171605+0000 mon.a (mon.0) 2050 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:32.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:32 smithi151 bash[23756]: audit 2024-03-04T15:57:31.171811+0000 mon.a (mon.0) 2051 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi117-63923-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:32.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:32 smithi151 bash[23756]: audit 2024-03-04T15:57:31.172247+0000 mon.c (mon.2) 411 : audit 1 from='client.? 172.21.15.117:0/4101380182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi117-63915-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:32.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:32 smithi151 bash[23756]: audit 2024-03-04T15:57:31.172534+0000 mon.a (mon.0) 2052 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi117-63915-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:32.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[21480]: cluster 2024-03-04T15:57:30.890853+0000 mgr.y (mgr.24428) 319 : cluster 0 pgmap v398: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:57:32.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[21480]: audit 2024-03-04T15:57:31.120220+0000 mon.c (mon.2) 408 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-31"}]: dispatch 2024-03-04T15:57:32.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[21480]: audit 2024-03-04T15:57:31.132799+0000 mon.a (mon.0) 2046 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-31"}]: dispatch 2024-03-04T15:57:32.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[21480]: audit 2024-03-04T15:57:31.169185+0000 mon.c (mon.2) 409 : audit 1 from='client.? 172.21.15.117:0/4101380182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:32.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[21480]: audit 2024-03-04T15:57:31.169754+0000 mon.a (mon.0) 2047 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:32.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[21480]: audit 2024-03-04T15:57:31.170289+0000 mon.a (mon.0) 2048 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi117-63923-49"}]: dispatch 2024-03-04T15:57:32.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[21480]: audit 2024-03-04T15:57:31.170944+0000 mon.c (mon.2) 410 : audit 1 from='client.? 172.21.15.117:0/4101380182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:32.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[21480]: audit 2024-03-04T15:57:31.171201+0000 mon.a (mon.0) 2049 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi117-63923-49"}]: dispatch 2024-03-04T15:57:32.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[21480]: audit 2024-03-04T15:57:31.171605+0000 mon.a (mon.0) 2050 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:32.571 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[21480]: audit 2024-03-04T15:57:31.171811+0000 mon.a (mon.0) 2051 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi117-63923-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:32.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[21480]: audit 2024-03-04T15:57:31.172247+0000 mon.c (mon.2) 411 : audit 1 from='client.? 172.21.15.117:0/4101380182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi117-63915-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:32.572 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[21480]: audit 2024-03-04T15:57:31.172534+0000 mon.a (mon.0) 2052 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi117-63915-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:32.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[28843]: cluster 2024-03-04T15:57:30.890853+0000 mgr.y (mgr.24428) 319 : cluster 0 pgmap v398: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:57:32.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[28843]: audit 2024-03-04T15:57:31.120220+0000 mon.c (mon.2) 408 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-31"}]: dispatch 2024-03-04T15:57:32.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[28843]: audit 2024-03-04T15:57:31.132799+0000 mon.a (mon.0) 2046 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-31"}]: dispatch 2024-03-04T15:57:32.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[28843]: audit 2024-03-04T15:57:31.169185+0000 mon.c (mon.2) 409 : audit 1 from='client.? 172.21.15.117:0/4101380182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:32.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[28843]: audit 2024-03-04T15:57:31.169754+0000 mon.a (mon.0) 2047 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:32.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[28843]: audit 2024-03-04T15:57:31.170289+0000 mon.a (mon.0) 2048 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi117-63923-49"}]: dispatch 2024-03-04T15:57:32.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[28843]: audit 2024-03-04T15:57:31.170944+0000 mon.c (mon.2) 410 : audit 1 from='client.? 172.21.15.117:0/4101380182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:32.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[28843]: audit 2024-03-04T15:57:31.171201+0000 mon.a (mon.0) 2049 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi117-63923-49"}]: dispatch 2024-03-04T15:57:32.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[28843]: audit 2024-03-04T15:57:31.171605+0000 mon.a (mon.0) 2050 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:32.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[28843]: audit 2024-03-04T15:57:31.171811+0000 mon.a (mon.0) 2051 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi117-63923-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:32.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[28843]: audit 2024-03-04T15:57:31.172247+0000 mon.c (mon.2) 411 : audit 1 from='client.? 172.21.15.117:0/4101380182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi117-63915-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:32.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:32 smithi117 bash[28843]: audit 2024-03-04T15:57:31.172534+0000 mon.a (mon.0) 2052 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi117-63915-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:34.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[21480]: audit 2024-03-04T15:57:32.124553+0000 mon.a (mon.0) 2053 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-31"}]': finished 2024-03-04T15:57:34.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[21480]: audit 2024-03-04T15:57:32.124785+0000 mon.a (mon.0) 2054 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi117-63923-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:34.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[21480]: audit 2024-03-04T15:57:32.124902+0000 mon.a (mon.0) 2055 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi117-63915-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:34.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[21480]: cluster 2024-03-04T15:57:32.135922+0000 mon.a (mon.0) 2056 : cluster 0 osdmap e263: 8 total, 8 up, 8 in 2024-03-04T15:57:34.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[21480]: audit 2024-03-04T15:57:32.137427+0000 mon.c (mon.2) 412 : audit 1 from='client.? 172.21.15.117:0/4101380182' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi117-63915-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:34.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[21480]: audit 2024-03-04T15:57:32.138313+0000 mon.c (mon.2) 413 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-31", "mode": "writeback"}]: dispatch 2024-03-04T15:57:34.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[21480]: audit 2024-03-04T15:57:32.139576+0000 mon.a (mon.0) 2057 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi117-63923-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi117-63923-49"}]: dispatch 2024-03-04T15:57:34.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[21480]: audit 2024-03-04T15:57:32.140240+0000 mon.a (mon.0) 2058 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi117-63915-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:34.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[21480]: audit 2024-03-04T15:57:32.147387+0000 mon.a (mon.0) 2059 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-31", "mode": "writeback"}]: dispatch 2024-03-04T15:57:34.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[21480]: cluster 2024-03-04T15:57:33.125776+0000 mon.a (mon.0) 2060 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:57:34.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[21480]: audit 2024-03-04T15:57:33.128232+0000 mon.a (mon.0) 2061 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-31", "mode": "writeback"}]': finished 2024-03-04T15:57:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[28843]: audit 2024-03-04T15:57:32.124553+0000 mon.a (mon.0) 2053 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-31"}]': finished 2024-03-04T15:57:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[28843]: audit 2024-03-04T15:57:32.124785+0000 mon.a (mon.0) 2054 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi117-63923-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[28843]: audit 2024-03-04T15:57:32.124902+0000 mon.a (mon.0) 2055 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi117-63915-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[28843]: cluster 2024-03-04T15:57:32.135922+0000 mon.a (mon.0) 2056 : cluster 0 osdmap e263: 8 total, 8 up, 8 in 2024-03-04T15:57:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[28843]: audit 2024-03-04T15:57:32.137427+0000 mon.c (mon.2) 412 : audit 1 from='client.? 172.21.15.117:0/4101380182' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi117-63915-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[28843]: audit 2024-03-04T15:57:32.138313+0000 mon.c (mon.2) 413 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-31", "mode": "writeback"}]: dispatch 2024-03-04T15:57:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[28843]: audit 2024-03-04T15:57:32.139576+0000 mon.a (mon.0) 2057 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi117-63923-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi117-63923-49"}]: dispatch 2024-03-04T15:57:34.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[28843]: audit 2024-03-04T15:57:32.140240+0000 mon.a (mon.0) 2058 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi117-63915-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:34.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[28843]: audit 2024-03-04T15:57:32.147387+0000 mon.a (mon.0) 2059 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-31", "mode": "writeback"}]: dispatch 2024-03-04T15:57:34.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[28843]: cluster 2024-03-04T15:57:33.125776+0000 mon.a (mon.0) 2060 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:57:34.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:34 smithi117 bash[28843]: audit 2024-03-04T15:57:33.128232+0000 mon.a (mon.0) 2061 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-31", "mode": "writeback"}]': finished 2024-03-04T15:57:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:34 smithi151 bash[23756]: audit 2024-03-04T15:57:32.124553+0000 mon.a (mon.0) 2053 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-31"}]': finished 2024-03-04T15:57:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:34 smithi151 bash[23756]: audit 2024-03-04T15:57:32.124785+0000 mon.a (mon.0) 2054 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi117-63923-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:34 smithi151 bash[23756]: audit 2024-03-04T15:57:32.124902+0000 mon.a (mon.0) 2055 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi117-63915-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:34 smithi151 bash[23756]: cluster 2024-03-04T15:57:32.135922+0000 mon.a (mon.0) 2056 : cluster 0 osdmap e263: 8 total, 8 up, 8 in 2024-03-04T15:57:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:34 smithi151 bash[23756]: audit 2024-03-04T15:57:32.137427+0000 mon.c (mon.2) 412 : audit 1 from='client.? 172.21.15.117:0/4101380182' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi117-63915-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:34.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:34 smithi151 bash[23756]: audit 2024-03-04T15:57:32.138313+0000 mon.c (mon.2) 413 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-31", "mode": "writeback"}]: dispatch 2024-03-04T15:57:34.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:34 smithi151 bash[23756]: audit 2024-03-04T15:57:32.139576+0000 mon.a (mon.0) 2057 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi117-63923-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi117-63923-49"}]: dispatch 2024-03-04T15:57:34.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:34 smithi151 bash[23756]: audit 2024-03-04T15:57:32.140240+0000 mon.a (mon.0) 2058 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi117-63915-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:34.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:34 smithi151 bash[23756]: audit 2024-03-04T15:57:32.147387+0000 mon.a (mon.0) 2059 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-31", "mode": "writeback"}]: dispatch 2024-03-04T15:57:34.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:34 smithi151 bash[23756]: cluster 2024-03-04T15:57:33.125776+0000 mon.a (mon.0) 2060 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:57:34.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:34 smithi151 bash[23756]: audit 2024-03-04T15:57:33.128232+0000 mon.a (mon.0) 2061 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-31", "mode": "writeback"}]': finished 2024-03-04T15:57:35.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:35 smithi117 bash[21480]: cluster 2024-03-04T15:57:32.891664+0000 mgr.y (mgr.24428) 320 : cluster 0 pgmap v401: 228 pgs: 228 active+clean; 4.6 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:57:35.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:35 smithi117 bash[21480]: audit 2024-03-04T15:57:33.752104+0000 mon.c (mon.2) 414 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:35.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:35 smithi117 bash[21480]: cluster 2024-03-04T15:57:34.008255+0000 mon.a (mon.0) 2062 : cluster 0 osdmap e264: 8 total, 8 up, 8 in 2024-03-04T15:57:35.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:35 smithi117 bash[21480]: audit 2024-03-04T15:57:34.023032+0000 mon.a (mon.0) 2063 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:35.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:35 smithi117 bash[21480]: audit 2024-03-04T15:57:34.131402+0000 mon.a (mon.0) 2064 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi117-63923-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi117-63923-49"}]': finished 2024-03-04T15:57:35.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:35 smithi117 bash[21480]: audit 2024-03-04T15:57:34.131577+0000 mon.a (mon.0) 2065 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi117-63915-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi117-63915-39"}]': finished 2024-03-04T15:57:35.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:35 smithi117 bash[21480]: audit 2024-03-04T15:57:34.131785+0000 mon.a (mon.0) 2066 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:35.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:35 smithi117 bash[21480]: cluster 2024-03-04T15:57:34.136486+0000 mon.a (mon.0) 2067 : cluster 0 osdmap e265: 8 total, 8 up, 8 in 2024-03-04T15:57:35.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:35 smithi117 bash[21480]: audit 2024-03-04T15:57:34.144414+0000 mon.c (mon.2) 415 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-31"}]: dispatch 2024-03-04T15:57:35.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:35 smithi117 bash[21480]: audit 2024-03-04T15:57:34.146581+0000 mon.a (mon.0) 2068 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-31"}]: dispatch 2024-03-04T15:57:35.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:35 smithi117 bash[28843]: cluster 2024-03-04T15:57:32.891664+0000 mgr.y (mgr.24428) 320 : cluster 0 pgmap v401: 228 pgs: 228 active+clean; 4.6 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:57:35.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:35 smithi117 bash[28843]: audit 2024-03-04T15:57:33.752104+0000 mon.c (mon.2) 414 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:35.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:35 smithi117 bash[28843]: cluster 2024-03-04T15:57:34.008255+0000 mon.a (mon.0) 2062 : cluster 0 osdmap e264: 8 total, 8 up, 8 in 2024-03-04T15:57:35.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:35 smithi117 bash[28843]: audit 2024-03-04T15:57:34.023032+0000 mon.a (mon.0) 2063 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:35.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:35 smithi117 bash[28843]: audit 2024-03-04T15:57:34.131402+0000 mon.a (mon.0) 2064 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi117-63923-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi117-63923-49"}]': finished 2024-03-04T15:57:35.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:35 smithi117 bash[28843]: audit 2024-03-04T15:57:34.131577+0000 mon.a (mon.0) 2065 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi117-63915-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi117-63915-39"}]': finished 2024-03-04T15:57:35.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:35 smithi117 bash[28843]: audit 2024-03-04T15:57:34.131785+0000 mon.a (mon.0) 2066 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:35.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:35 smithi117 bash[28843]: cluster 2024-03-04T15:57:34.136486+0000 mon.a (mon.0) 2067 : cluster 0 osdmap e265: 8 total, 8 up, 8 in 2024-03-04T15:57:35.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:35 smithi117 bash[28843]: audit 2024-03-04T15:57:34.144414+0000 mon.c (mon.2) 415 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-31"}]: dispatch 2024-03-04T15:57:35.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:35 smithi117 bash[28843]: audit 2024-03-04T15:57:34.146581+0000 mon.a (mon.0) 2068 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-31"}]: dispatch 2024-03-04T15:57:35.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:35 smithi151 bash[23756]: cluster 2024-03-04T15:57:32.891664+0000 mgr.y (mgr.24428) 320 : cluster 0 pgmap v401: 228 pgs: 228 active+clean; 4.6 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:57:35.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:35 smithi151 bash[23756]: audit 2024-03-04T15:57:33.752104+0000 mon.c (mon.2) 414 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:35.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:35 smithi151 bash[23756]: cluster 2024-03-04T15:57:34.008255+0000 mon.a (mon.0) 2062 : cluster 0 osdmap e264: 8 total, 8 up, 8 in 2024-03-04T15:57:35.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:35 smithi151 bash[23756]: audit 2024-03-04T15:57:34.023032+0000 mon.a (mon.0) 2063 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:35.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:35 smithi151 bash[23756]: audit 2024-03-04T15:57:34.131402+0000 mon.a (mon.0) 2064 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi117-63923-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi117-63923-49"}]': finished 2024-03-04T15:57:35.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:35 smithi151 bash[23756]: audit 2024-03-04T15:57:34.131577+0000 mon.a (mon.0) 2065 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi117-63915-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi117-63915-39"}]': finished 2024-03-04T15:57:35.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:35 smithi151 bash[23756]: audit 2024-03-04T15:57:34.131785+0000 mon.a (mon.0) 2066 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:35.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:35 smithi151 bash[23756]: cluster 2024-03-04T15:57:34.136486+0000 mon.a (mon.0) 2067 : cluster 0 osdmap e265: 8 total, 8 up, 8 in 2024-03-04T15:57:35.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:35 smithi151 bash[23756]: audit 2024-03-04T15:57:34.144414+0000 mon.c (mon.2) 415 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-31"}]: dispatch 2024-03-04T15:57:35.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:35 smithi151 bash[23756]: audit 2024-03-04T15:57:34.146581+0000 mon.a (mon.0) 2068 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-31"}]: dispatch 2024-03-04T15:57:36.317 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:57:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:57:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:57:36.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:36 smithi117 bash[21480]: cluster 2024-03-04T15:57:34.892508+0000 mgr.y (mgr.24428) 321 : cluster 0 pgmap v404: 244 pgs: 16 unknown, 228 active+clean; 4.6 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:57:36.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:36 smithi117 bash[21480]: cluster 2024-03-04T15:57:35.131385+0000 mon.a (mon.0) 2069 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:36.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:36 smithi117 bash[21480]: cluster 2024-03-04T15:57:35.132796+0000 mon.a (mon.0) 2070 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:57:36.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:36 smithi117 bash[21480]: audit 2024-03-04T15:57:35.142707+0000 mon.a (mon.0) 2071 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-31"}]': finished 2024-03-04T15:57:36.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:36 smithi117 bash[21480]: cluster 2024-03-04T15:57:35.150703+0000 mon.a (mon.0) 2072 : cluster 0 osdmap e266: 8 total, 8 up, 8 in 2024-03-04T15:57:36.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:36 smithi117 bash[28843]: cluster 2024-03-04T15:57:34.892508+0000 mgr.y (mgr.24428) 321 : cluster 0 pgmap v404: 244 pgs: 16 unknown, 228 active+clean; 4.6 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:57:36.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:36 smithi117 bash[28843]: cluster 2024-03-04T15:57:35.131385+0000 mon.a (mon.0) 2069 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:36.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:36 smithi117 bash[28843]: cluster 2024-03-04T15:57:35.132796+0000 mon.a (mon.0) 2070 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:57:36.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:36 smithi117 bash[28843]: audit 2024-03-04T15:57:35.142707+0000 mon.a (mon.0) 2071 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-31"}]': finished 2024-03-04T15:57:36.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:36 smithi117 bash[28843]: cluster 2024-03-04T15:57:35.150703+0000 mon.a (mon.0) 2072 : cluster 0 osdmap e266: 8 total, 8 up, 8 in 2024-03-04T15:57:36.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:36 smithi151 bash[23756]: cluster 2024-03-04T15:57:34.892508+0000 mgr.y (mgr.24428) 321 : cluster 0 pgmap v404: 244 pgs: 16 unknown, 228 active+clean; 4.6 MiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:57:36.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:36 smithi151 bash[23756]: cluster 2024-03-04T15:57:35.131385+0000 mon.a (mon.0) 2069 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:36.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:36 smithi151 bash[23756]: cluster 2024-03-04T15:57:35.132796+0000 mon.a (mon.0) 2070 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:57:36.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:36 smithi151 bash[23756]: audit 2024-03-04T15:57:35.142707+0000 mon.a (mon.0) 2071 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-31"}]': finished 2024-03-04T15:57:36.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:36 smithi151 bash[23756]: cluster 2024-03-04T15:57:35.150703+0000 mon.a (mon.0) 2072 : cluster 0 osdmap e266: 8 total, 8 up, 8 in 2024-03-04T15:57:36.931 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:57:36 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:57:37.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:37 smithi151 bash[23756]: cluster 2024-03-04T15:57:36.153356+0000 mon.a (mon.0) 2073 : cluster 0 osdmap e267: 8 total, 8 up, 8 in 2024-03-04T15:57:37.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:37 smithi151 bash[23756]: audit 2024-03-04T15:57:36.155389+0000 mon.a (mon.0) 2074 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi117-63923-49"}]: dispatch 2024-03-04T15:57:37.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:37 smithi151 bash[23756]: audit 2024-03-04T15:57:36.163662+0000 mon.c (mon.2) 416 : audit 1 from='client.? 172.21.15.117:0/4101380182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:37.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:37 smithi151 bash[23756]: audit 2024-03-04T15:57:36.610817+0000 mon.a (mon.0) 2075 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:37.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:37 smithi117 bash[21480]: cluster 2024-03-04T15:57:36.153356+0000 mon.a (mon.0) 2073 : cluster 0 osdmap e267: 8 total, 8 up, 8 in 2024-03-04T15:57:37.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:37 smithi117 bash[21480]: audit 2024-03-04T15:57:36.155389+0000 mon.a (mon.0) 2074 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi117-63923-49"}]: dispatch 2024-03-04T15:57:37.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:37 smithi117 bash[21480]: audit 2024-03-04T15:57:36.163662+0000 mon.c (mon.2) 416 : audit 1 from='client.? 172.21.15.117:0/4101380182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:37.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:37 smithi117 bash[21480]: audit 2024-03-04T15:57:36.610817+0000 mon.a (mon.0) 2075 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:37.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:37 smithi117 bash[28843]: cluster 2024-03-04T15:57:36.153356+0000 mon.a (mon.0) 2073 : cluster 0 osdmap e267: 8 total, 8 up, 8 in 2024-03-04T15:57:37.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:37 smithi117 bash[28843]: audit 2024-03-04T15:57:36.155389+0000 mon.a (mon.0) 2074 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi117-63923-49"}]: dispatch 2024-03-04T15:57:37.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:37 smithi117 bash[28843]: audit 2024-03-04T15:57:36.163662+0000 mon.c (mon.2) 416 : audit 1 from='client.? 172.21.15.117:0/4101380182' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:37.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:37 smithi117 bash[28843]: audit 2024-03-04T15:57:36.610817+0000 mon.a (mon.0) 2075 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:38.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:38 smithi151 bash[23756]: audit 2024-03-04T15:57:36.645801+0000 mgr.y (mgr.24428) 322 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:57:38.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:38 smithi151 bash[23756]: cluster 2024-03-04T15:57:36.894527+0000 mgr.y (mgr.24428) 323 : cluster 0 pgmap v407: 196 pgs: 196 active+clean; 4.6 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:57:38.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:38 smithi151 bash[23756]: audit 2024-03-04T15:57:37.160045+0000 mon.a (mon.0) 2076 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi117-63923-49"}]': finished 2024-03-04T15:57:38.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:38 smithi151 bash[23756]: audit 2024-03-04T15:57:37.160234+0000 mon.a (mon.0) 2077 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi117-63915-39"}]': finished 2024-03-04T15:57:38.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:38 smithi151 bash[23756]: cluster 2024-03-04T15:57:37.168465+0000 mon.a (mon.0) 2078 : cluster 0 osdmap e268: 8 total, 8 up, 8 in 2024-03-04T15:57:38.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:38 smithi151 bash[23756]: audit 2024-03-04T15:57:37.169423+0000 mon.a (mon.0) 2079 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi117-63923-49"}]: dispatch 2024-03-04T15:57:38.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:38 smithi151 bash[23756]: audit 2024-03-04T15:57:37.173500+0000 mon.c (mon.2) 417 : audit 1 from='client.? 172.21.15.117:0/4101380182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:38.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:38 smithi151 bash[23756]: audit 2024-03-04T15:57:37.174393+0000 mon.c (mon.2) 418 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:38.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:38 smithi151 bash[23756]: audit 2024-03-04T15:57:37.174671+0000 mon.a (mon.0) 2080 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:38.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:38 smithi151 bash[23756]: audit 2024-03-04T15:57:37.175889+0000 mon.a (mon.0) 2081 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:38.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:38 smithi117 bash[21480]: audit 2024-03-04T15:57:36.645801+0000 mgr.y (mgr.24428) 322 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:57:38.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:38 smithi117 bash[21480]: cluster 2024-03-04T15:57:36.894527+0000 mgr.y (mgr.24428) 323 : cluster 0 pgmap v407: 196 pgs: 196 active+clean; 4.6 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:57:38.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:38 smithi117 bash[21480]: audit 2024-03-04T15:57:37.160045+0000 mon.a (mon.0) 2076 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi117-63923-49"}]': finished 2024-03-04T15:57:38.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:38 smithi117 bash[21480]: audit 2024-03-04T15:57:37.160234+0000 mon.a (mon.0) 2077 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi117-63915-39"}]': finished 2024-03-04T15:57:38.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:38 smithi117 bash[21480]: cluster 2024-03-04T15:57:37.168465+0000 mon.a (mon.0) 2078 : cluster 0 osdmap e268: 8 total, 8 up, 8 in 2024-03-04T15:57:38.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:38 smithi117 bash[21480]: audit 2024-03-04T15:57:37.169423+0000 mon.a (mon.0) 2079 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi117-63923-49"}]: dispatch 2024-03-04T15:57:38.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:38 smithi117 bash[21480]: audit 2024-03-04T15:57:37.173500+0000 mon.c (mon.2) 417 : audit 1 from='client.? 172.21.15.117:0/4101380182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:38.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:38 smithi117 bash[21480]: audit 2024-03-04T15:57:37.174393+0000 mon.c (mon.2) 418 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:38.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:38 smithi117 bash[21480]: audit 2024-03-04T15:57:37.174671+0000 mon.a (mon.0) 2080 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:38.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:38 smithi117 bash[21480]: audit 2024-03-04T15:57:37.175889+0000 mon.a (mon.0) 2081 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:38.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:38 smithi117 bash[28843]: audit 2024-03-04T15:57:36.645801+0000 mgr.y (mgr.24428) 322 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:57:38.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:38 smithi117 bash[28843]: cluster 2024-03-04T15:57:36.894527+0000 mgr.y (mgr.24428) 323 : cluster 0 pgmap v407: 196 pgs: 196 active+clean; 4.6 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:57:38.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:38 smithi117 bash[28843]: audit 2024-03-04T15:57:37.160045+0000 mon.a (mon.0) 2076 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi117-63923-49"}]': finished 2024-03-04T15:57:38.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:38 smithi117 bash[28843]: audit 2024-03-04T15:57:37.160234+0000 mon.a (mon.0) 2077 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi117-63915-39"}]': finished 2024-03-04T15:57:38.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:38 smithi117 bash[28843]: cluster 2024-03-04T15:57:37.168465+0000 mon.a (mon.0) 2078 : cluster 0 osdmap e268: 8 total, 8 up, 8 in 2024-03-04T15:57:38.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:38 smithi117 bash[28843]: audit 2024-03-04T15:57:37.169423+0000 mon.a (mon.0) 2079 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi117-63923-49"}]: dispatch 2024-03-04T15:57:38.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:38 smithi117 bash[28843]: audit 2024-03-04T15:57:37.173500+0000 mon.c (mon.2) 417 : audit 1 from='client.? 172.21.15.117:0/4101380182' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:38.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:38 smithi117 bash[28843]: audit 2024-03-04T15:57:37.174393+0000 mon.c (mon.2) 418 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:38.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:38 smithi117 bash[28843]: audit 2024-03-04T15:57:37.174671+0000 mon.a (mon.0) 2080 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi117-63915-39"}]: dispatch 2024-03-04T15:57:38.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:38 smithi117 bash[28843]: audit 2024-03-04T15:57:37.175889+0000 mon.a (mon.0) 2081 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:39.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[21480]: audit 2024-03-04T15:57:38.175697+0000 mon.a (mon.0) 2082 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi117-63923-49"}]': finished 2024-03-04T15:57:39.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[21480]: audit 2024-03-04T15:57:38.175892+0000 mon.a (mon.0) 2083 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi117-63915-39"}]': finished 2024-03-04T15:57:39.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[21480]: audit 2024-03-04T15:57:38.176302+0000 mon.a (mon.0) 2084 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:39.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[21480]: cluster 2024-03-04T15:57:38.184880+0000 mon.a (mon.0) 2085 : cluster 0 osdmap e269: 8 total, 8 up, 8 in 2024-03-04T15:57:39.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[21480]: audit 2024-03-04T15:57:38.211958+0000 mon.c (mon.2) 419 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:39.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[21480]: audit 2024-03-04T15:57:38.215946+0000 mon.a (mon.0) 2086 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:39.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[21480]: audit 2024-03-04T15:57:38.227139+0000 mon.c (mon.2) 420 : audit 1 from='client.? 172.21.15.117:0/2285618511' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:39.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[21480]: audit 2024-03-04T15:57:38.227613+0000 mon.a (mon.0) 2087 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:39.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[21480]: audit 2024-03-04T15:57:38.229433+0000 mon.c (mon.2) 421 : audit 1 from='client.? 172.21.15.117:0/2285618511' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:39.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[21480]: audit 2024-03-04T15:57:38.229717+0000 mon.a (mon.0) 2088 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:39.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[21480]: audit 2024-03-04T15:57:38.230351+0000 mon.c (mon.2) 422 : audit 1 from='client.? 172.21.15.117:0/2285618511' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi117-63915-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:39.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[21480]: audit 2024-03-04T15:57:38.230748+0000 mon.a (mon.0) 2089 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi117-63915-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:39.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[21480]: audit 2024-03-04T15:57:38.235798+0000 mon.c (mon.2) 423 : audit 1 from='client.? 172.21.15.117:0/3335546711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:39.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[21480]: audit 2024-03-04T15:57:38.236068+0000 mon.a (mon.0) 2090 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:39.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[21480]: audit 2024-03-04T15:57:38.237438+0000 mon.c (mon.2) 424 : audit 1 from='client.? 172.21.15.117:0/3335546711' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:39.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[28843]: audit 2024-03-04T15:57:38.175697+0000 mon.a (mon.0) 2082 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi117-63923-49"}]': finished 2024-03-04T15:57:39.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[28843]: audit 2024-03-04T15:57:38.175892+0000 mon.a (mon.0) 2083 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi117-63915-39"}]': finished 2024-03-04T15:57:39.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[28843]: audit 2024-03-04T15:57:38.176302+0000 mon.a (mon.0) 2084 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:39.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[28843]: cluster 2024-03-04T15:57:38.184880+0000 mon.a (mon.0) 2085 : cluster 0 osdmap e269: 8 total, 8 up, 8 in 2024-03-04T15:57:39.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[28843]: audit 2024-03-04T15:57:38.211958+0000 mon.c (mon.2) 419 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:39.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[28843]: audit 2024-03-04T15:57:38.215946+0000 mon.a (mon.0) 2086 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:39.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[28843]: audit 2024-03-04T15:57:38.227139+0000 mon.c (mon.2) 420 : audit 1 from='client.? 172.21.15.117:0/2285618511' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:39.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[28843]: audit 2024-03-04T15:57:38.227613+0000 mon.a (mon.0) 2087 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:39.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[28843]: audit 2024-03-04T15:57:38.229433+0000 mon.c (mon.2) 421 : audit 1 from='client.? 172.21.15.117:0/2285618511' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:39.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[28843]: audit 2024-03-04T15:57:38.229717+0000 mon.a (mon.0) 2088 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:39.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[28843]: audit 2024-03-04T15:57:38.230351+0000 mon.c (mon.2) 422 : audit 1 from='client.? 172.21.15.117:0/2285618511' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi117-63915-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:39.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[28843]: audit 2024-03-04T15:57:38.230748+0000 mon.a (mon.0) 2089 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi117-63915-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:39.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[28843]: audit 2024-03-04T15:57:38.235798+0000 mon.c (mon.2) 423 : audit 1 from='client.? 172.21.15.117:0/3335546711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:39.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[28843]: audit 2024-03-04T15:57:38.236068+0000 mon.a (mon.0) 2090 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:39.572 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[28843]: audit 2024-03-04T15:57:38.237438+0000 mon.c (mon.2) 424 : audit 1 from='client.? 172.21.15.117:0/3335546711' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:39.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[21480]: audit 2024-03-04T15:57:38.237738+0000 mon.a (mon.0) 2091 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:39.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[21480]: audit 2024-03-04T15:57:38.238270+0000 mon.c (mon.2) 425 : audit 1 from='client.? 172.21.15.117:0/3335546711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi117-63923-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:39.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[21480]: audit 2024-03-04T15:57:38.238576+0000 mon.a (mon.0) 2092 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi117-63923-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:39.573 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[21480]: audit 2024-03-04T15:57:38.698166+0000 mon.c (mon.2) 426 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:57:39.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[28843]: audit 2024-03-04T15:57:38.237738+0000 mon.a (mon.0) 2091 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:39.573 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[28843]: audit 2024-03-04T15:57:38.238270+0000 mon.c (mon.2) 425 : audit 1 from='client.? 172.21.15.117:0/3335546711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi117-63923-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:39.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[28843]: audit 2024-03-04T15:57:38.238576+0000 mon.a (mon.0) 2092 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi117-63923-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:39.574 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:39 smithi117 bash[28843]: audit 2024-03-04T15:57:38.698166+0000 mon.c (mon.2) 426 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:57:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:39 smithi151 bash[23756]: audit 2024-03-04T15:57:38.175697+0000 mon.a (mon.0) 2082 : audit 1 from='client.? 172.21.15.117:0/2143413079' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi117-63923-49"}]': finished 2024-03-04T15:57:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:39 smithi151 bash[23756]: audit 2024-03-04T15:57:38.175892+0000 mon.a (mon.0) 2083 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi117-63915-39"}]': finished 2024-03-04T15:57:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:39 smithi151 bash[23756]: audit 2024-03-04T15:57:38.176302+0000 mon.a (mon.0) 2084 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:39 smithi151 bash[23756]: cluster 2024-03-04T15:57:38.184880+0000 mon.a (mon.0) 2085 : cluster 0 osdmap e269: 8 total, 8 up, 8 in 2024-03-04T15:57:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:39 smithi151 bash[23756]: audit 2024-03-04T15:57:38.211958+0000 mon.c (mon.2) 419 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:39 smithi151 bash[23756]: audit 2024-03-04T15:57:38.215946+0000 mon.a (mon.0) 2086 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:39 smithi151 bash[23756]: audit 2024-03-04T15:57:38.227139+0000 mon.c (mon.2) 420 : audit 1 from='client.? 172.21.15.117:0/2285618511' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:39 smithi151 bash[23756]: audit 2024-03-04T15:57:38.227613+0000 mon.a (mon.0) 2087 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:39 smithi151 bash[23756]: audit 2024-03-04T15:57:38.229433+0000 mon.c (mon.2) 421 : audit 1 from='client.? 172.21.15.117:0/2285618511' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:39 smithi151 bash[23756]: audit 2024-03-04T15:57:38.229717+0000 mon.a (mon.0) 2088 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:39 smithi151 bash[23756]: audit 2024-03-04T15:57:38.230351+0000 mon.c (mon.2) 422 : audit 1 from='client.? 172.21.15.117:0/2285618511' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi117-63915-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:39 smithi151 bash[23756]: audit 2024-03-04T15:57:38.230748+0000 mon.a (mon.0) 2089 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi117-63915-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:39 smithi151 bash[23756]: audit 2024-03-04T15:57:38.235798+0000 mon.c (mon.2) 423 : audit 1 from='client.? 172.21.15.117:0/3335546711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:39 smithi151 bash[23756]: audit 2024-03-04T15:57:38.236068+0000 mon.a (mon.0) 2090 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:39 smithi151 bash[23756]: audit 2024-03-04T15:57:38.237438+0000 mon.c (mon.2) 424 : audit 1 from='client.? 172.21.15.117:0/3335546711' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:39 smithi151 bash[23756]: audit 2024-03-04T15:57:38.237738+0000 mon.a (mon.0) 2091 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:39 smithi151 bash[23756]: audit 2024-03-04T15:57:38.238270+0000 mon.c (mon.2) 425 : audit 1 from='client.? 172.21.15.117:0/3335546711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi117-63923-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:39 smithi151 bash[23756]: audit 2024-03-04T15:57:38.238576+0000 mon.a (mon.0) 2092 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi117-63923-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:39.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:39 smithi151 bash[23756]: audit 2024-03-04T15:57:38.698166+0000 mon.c (mon.2) 426 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:57:41.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:41 smithi151 bash[23756]: cluster 2024-03-04T15:57:38.895409+0000 mgr.y (mgr.24428) 324 : cluster 0 pgmap v410: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:57:41.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:41 smithi151 bash[23756]: audit 2024-03-04T15:57:39.177994+0000 mon.a (mon.0) 2093 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-33", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:57:41.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:41 smithi151 bash[23756]: audit 2024-03-04T15:57:39.178205+0000 mon.a (mon.0) 2094 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi117-63915-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:41.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:41 smithi151 bash[23756]: audit 2024-03-04T15:57:39.178320+0000 mon.a (mon.0) 2095 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi117-63923-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:41.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:41 smithi151 bash[23756]: cluster 2024-03-04T15:57:39.183971+0000 mon.a (mon.0) 2096 : cluster 0 osdmap e270: 8 total, 8 up, 8 in 2024-03-04T15:57:41.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:41 smithi151 bash[23756]: audit 2024-03-04T15:57:39.189457+0000 mon.c (mon.2) 427 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-33"}]: dispatch 2024-03-04T15:57:41.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:41 smithi151 bash[23756]: audit 2024-03-04T15:57:39.189783+0000 mon.c (mon.2) 428 : audit 1 from='client.? 172.21.15.117:0/2285618511' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi117-63915-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:41.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:41 smithi151 bash[23756]: audit 2024-03-04T15:57:39.189974+0000 mon.c (mon.2) 429 : audit 1 from='client.? 172.21.15.117:0/3335546711' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi117-63923-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:41.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:41 smithi151 bash[23756]: audit 2024-03-04T15:57:39.197024+0000 mon.a (mon.0) 2097 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-33"}]: dispatch 2024-03-04T15:57:41.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:41 smithi151 bash[23756]: audit 2024-03-04T15:57:39.197325+0000 mon.a (mon.0) 2098 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi117-63915-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:41.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:41 smithi151 bash[23756]: audit 2024-03-04T15:57:39.197593+0000 mon.a (mon.0) 2099 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi117-63923-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:41.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:41 smithi151 bash[23756]: audit 2024-03-04T15:57:39.489106+0000 mon.c (mon.2) 430 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:57:41.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:41 smithi151 bash[23756]: audit 2024-03-04T15:57:39.491024+0000 mon.c (mon.2) 431 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:57:41.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:41 smithi151 bash[23756]: audit 2024-03-04T15:57:39.499526+0000 mon.a (mon.0) 2100 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:57:41.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[21480]: cluster 2024-03-04T15:57:38.895409+0000 mgr.y (mgr.24428) 324 : cluster 0 pgmap v410: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:57:41.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[21480]: audit 2024-03-04T15:57:39.177994+0000 mon.a (mon.0) 2093 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-33", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:57:41.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[21480]: audit 2024-03-04T15:57:39.178205+0000 mon.a (mon.0) 2094 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi117-63915-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:41.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[21480]: audit 2024-03-04T15:57:39.178320+0000 mon.a (mon.0) 2095 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi117-63923-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:41.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[21480]: cluster 2024-03-04T15:57:39.183971+0000 mon.a (mon.0) 2096 : cluster 0 osdmap e270: 8 total, 8 up, 8 in 2024-03-04T15:57:41.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[21480]: audit 2024-03-04T15:57:39.189457+0000 mon.c (mon.2) 427 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-33"}]: dispatch 2024-03-04T15:57:41.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[21480]: audit 2024-03-04T15:57:39.189783+0000 mon.c (mon.2) 428 : audit 1 from='client.? 172.21.15.117:0/2285618511' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi117-63915-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:41.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[21480]: audit 2024-03-04T15:57:39.189974+0000 mon.c (mon.2) 429 : audit 1 from='client.? 172.21.15.117:0/3335546711' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi117-63923-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:41.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[21480]: audit 2024-03-04T15:57:39.197024+0000 mon.a (mon.0) 2097 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-33"}]: dispatch 2024-03-04T15:57:41.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[21480]: audit 2024-03-04T15:57:39.197325+0000 mon.a (mon.0) 2098 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi117-63915-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:41.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[21480]: audit 2024-03-04T15:57:39.197593+0000 mon.a (mon.0) 2099 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi117-63923-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:41.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[21480]: audit 2024-03-04T15:57:39.489106+0000 mon.c (mon.2) 430 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:57:41.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[21480]: audit 2024-03-04T15:57:39.491024+0000 mon.c (mon.2) 431 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:57:41.570 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[21480]: audit 2024-03-04T15:57:39.499526+0000 mon.a (mon.0) 2100 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:57:41.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[28843]: cluster 2024-03-04T15:57:38.895409+0000 mgr.y (mgr.24428) 324 : cluster 0 pgmap v410: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:57:41.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[28843]: audit 2024-03-04T15:57:39.177994+0000 mon.a (mon.0) 2093 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-33", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:57:41.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[28843]: audit 2024-03-04T15:57:39.178205+0000 mon.a (mon.0) 2094 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi117-63915-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:41.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[28843]: audit 2024-03-04T15:57:39.178320+0000 mon.a (mon.0) 2095 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi117-63923-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:41.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[28843]: cluster 2024-03-04T15:57:39.183971+0000 mon.a (mon.0) 2096 : cluster 0 osdmap e270: 8 total, 8 up, 8 in 2024-03-04T15:57:41.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[28843]: audit 2024-03-04T15:57:39.189457+0000 mon.c (mon.2) 427 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-33"}]: dispatch 2024-03-04T15:57:41.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[28843]: audit 2024-03-04T15:57:39.189783+0000 mon.c (mon.2) 428 : audit 1 from='client.? 172.21.15.117:0/2285618511' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi117-63915-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:41.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[28843]: audit 2024-03-04T15:57:39.189974+0000 mon.c (mon.2) 429 : audit 1 from='client.? 172.21.15.117:0/3335546711' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi117-63923-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:41.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[28843]: audit 2024-03-04T15:57:39.197024+0000 mon.a (mon.0) 2097 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-33"}]: dispatch 2024-03-04T15:57:41.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[28843]: audit 2024-03-04T15:57:39.197325+0000 mon.a (mon.0) 2098 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi117-63915-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:41.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[28843]: audit 2024-03-04T15:57:39.197593+0000 mon.a (mon.0) 2099 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi117-63923-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:41.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[28843]: audit 2024-03-04T15:57:39.489106+0000 mon.c (mon.2) 430 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:57:41.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[28843]: audit 2024-03-04T15:57:39.491024+0000 mon.c (mon.2) 431 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:57:41.571 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:41 smithi117 bash[28843]: audit 2024-03-04T15:57:39.499526+0000 mon.a (mon.0) 2100 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:57:42.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:42 smithi151 bash[23756]: audit 2024-03-04T15:57:40.194157+0000 mon.a (mon.0) 2101 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-33"}]': finished 2024-03-04T15:57:42.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:42 smithi151 bash[23756]: audit 2024-03-04T15:57:40.206073+0000 mon.c (mon.2) 432 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-33", "mode": "writeback"}]: dispatch 2024-03-04T15:57:42.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:42 smithi151 bash[23756]: cluster 2024-03-04T15:57:40.896181+0000 mgr.y (mgr.24428) 325 : cluster 0 pgmap v413: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:57:42.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:42 smithi151 bash[23756]: cluster 2024-03-04T15:57:41.096154+0000 mon.a (mon.0) 2102 : cluster 0 osdmap e271: 8 total, 8 up, 8 in 2024-03-04T15:57:42.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:42 smithi151 bash[23756]: audit 2024-03-04T15:57:41.097660+0000 mon.a (mon.0) 2103 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-33", "mode": "writeback"}]: dispatch 2024-03-04T15:57:42.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:42 smithi151 bash[23756]: cluster 2024-03-04T15:57:41.195545+0000 mon.a (mon.0) 2104 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:57:42.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:42 smithi151 bash[23756]: audit 2024-03-04T15:57:41.205184+0000 mon.a (mon.0) 2105 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi117-63915-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi117-63915-40"}]': finished 2024-03-04T15:57:42.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:42 smithi151 bash[23756]: audit 2024-03-04T15:57:41.205388+0000 mon.a (mon.0) 2106 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi117-63923-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi117-63923-50"}]': finished 2024-03-04T15:57:42.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:42 smithi151 bash[23756]: audit 2024-03-04T15:57:41.205517+0000 mon.a (mon.0) 2107 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-33", "mode": "writeback"}]': finished 2024-03-04T15:57:42.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:42 smithi151 bash[23756]: cluster 2024-03-04T15:57:41.213748+0000 mon.a (mon.0) 2108 : cluster 0 osdmap e272: 8 total, 8 up, 8 in 2024-03-04T15:57:42.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:42 smithi151 bash[23756]: audit 2024-03-04T15:57:41.456379+0000 mon.c (mon.2) 433 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:42.433 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:42 smithi151 bash[23756]: audit 2024-03-04T15:57:41.457003+0000 mon.a (mon.0) 2109 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:42.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[21480]: audit 2024-03-04T15:57:40.194157+0000 mon.a (mon.0) 2101 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-33"}]': finished 2024-03-04T15:57:42.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[21480]: audit 2024-03-04T15:57:40.206073+0000 mon.c (mon.2) 432 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-33", "mode": "writeback"}]: dispatch 2024-03-04T15:57:42.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[21480]: cluster 2024-03-04T15:57:40.896181+0000 mgr.y (mgr.24428) 325 : cluster 0 pgmap v413: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:57:42.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[21480]: cluster 2024-03-04T15:57:41.096154+0000 mon.a (mon.0) 2102 : cluster 0 osdmap e271: 8 total, 8 up, 8 in 2024-03-04T15:57:42.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[21480]: audit 2024-03-04T15:57:41.097660+0000 mon.a (mon.0) 2103 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-33", "mode": "writeback"}]: dispatch 2024-03-04T15:57:42.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[21480]: cluster 2024-03-04T15:57:41.195545+0000 mon.a (mon.0) 2104 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:57:42.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[21480]: audit 2024-03-04T15:57:41.205184+0000 mon.a (mon.0) 2105 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi117-63915-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi117-63915-40"}]': finished 2024-03-04T15:57:42.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[21480]: audit 2024-03-04T15:57:41.205388+0000 mon.a (mon.0) 2106 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi117-63923-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi117-63923-50"}]': finished 2024-03-04T15:57:42.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[21480]: audit 2024-03-04T15:57:41.205517+0000 mon.a (mon.0) 2107 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-33", "mode": "writeback"}]': finished 2024-03-04T15:57:42.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[21480]: cluster 2024-03-04T15:57:41.213748+0000 mon.a (mon.0) 2108 : cluster 0 osdmap e272: 8 total, 8 up, 8 in 2024-03-04T15:57:42.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[21480]: audit 2024-03-04T15:57:41.456379+0000 mon.c (mon.2) 433 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:42.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[21480]: audit 2024-03-04T15:57:41.457003+0000 mon.a (mon.0) 2109 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:42.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[28843]: audit 2024-03-04T15:57:40.194157+0000 mon.a (mon.0) 2101 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-33"}]': finished 2024-03-04T15:57:42.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[28843]: audit 2024-03-04T15:57:40.206073+0000 mon.c (mon.2) 432 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-33", "mode": "writeback"}]: dispatch 2024-03-04T15:57:42.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[28843]: cluster 2024-03-04T15:57:40.896181+0000 mgr.y (mgr.24428) 325 : cluster 0 pgmap v413: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:57:42.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[28843]: cluster 2024-03-04T15:57:41.096154+0000 mon.a (mon.0) 2102 : cluster 0 osdmap e271: 8 total, 8 up, 8 in 2024-03-04T15:57:42.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[28843]: audit 2024-03-04T15:57:41.097660+0000 mon.a (mon.0) 2103 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-33", "mode": "writeback"}]: dispatch 2024-03-04T15:57:42.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[28843]: cluster 2024-03-04T15:57:41.195545+0000 mon.a (mon.0) 2104 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:57:42.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[28843]: audit 2024-03-04T15:57:41.205184+0000 mon.a (mon.0) 2105 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi117-63915-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi117-63915-40"}]': finished 2024-03-04T15:57:42.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[28843]: audit 2024-03-04T15:57:41.205388+0000 mon.a (mon.0) 2106 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi117-63923-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi117-63923-50"}]': finished 2024-03-04T15:57:42.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[28843]: audit 2024-03-04T15:57:41.205517+0000 mon.a (mon.0) 2107 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-33", "mode": "writeback"}]': finished 2024-03-04T15:57:42.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[28843]: cluster 2024-03-04T15:57:41.213748+0000 mon.a (mon.0) 2108 : cluster 0 osdmap e272: 8 total, 8 up, 8 in 2024-03-04T15:57:42.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[28843]: audit 2024-03-04T15:57:41.456379+0000 mon.c (mon.2) 433 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:42.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:42 smithi117 bash[28843]: audit 2024-03-04T15:57:41.457003+0000 mon.a (mon.0) 2109 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:43.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:43 smithi117 bash[28843]: audit 2024-03-04T15:57:42.208035+0000 mon.a (mon.0) 2110 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:43.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:43 smithi117 bash[28843]: cluster 2024-03-04T15:57:42.217289+0000 mon.a (mon.0) 2111 : cluster 0 osdmap e273: 8 total, 8 up, 8 in 2024-03-04T15:57:43.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:43 smithi117 bash[28843]: audit 2024-03-04T15:57:42.219868+0000 mon.c (mon.2) 434 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-33"}]: dispatch 2024-03-04T15:57:43.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:43 smithi117 bash[28843]: audit 2024-03-04T15:57:42.222417+0000 mon.a (mon.0) 2112 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-33"}]: dispatch 2024-03-04T15:57:43.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:43 smithi117 bash[28843]: cluster 2024-03-04T15:57:43.096795+0000 mon.a (mon.0) 2113 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:43.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:43 smithi117 bash[21480]: audit 2024-03-04T15:57:42.208035+0000 mon.a (mon.0) 2110 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:43.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:43 smithi117 bash[21480]: cluster 2024-03-04T15:57:42.217289+0000 mon.a (mon.0) 2111 : cluster 0 osdmap e273: 8 total, 8 up, 8 in 2024-03-04T15:57:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:43 smithi117 bash[21480]: audit 2024-03-04T15:57:42.219868+0000 mon.c (mon.2) 434 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-33"}]: dispatch 2024-03-04T15:57:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:43 smithi117 bash[21480]: audit 2024-03-04T15:57:42.222417+0000 mon.a (mon.0) 2112 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-33"}]: dispatch 2024-03-04T15:57:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:43 smithi117 bash[21480]: cluster 2024-03-04T15:57:43.096795+0000 mon.a (mon.0) 2113 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:43 smithi151 bash[23756]: audit 2024-03-04T15:57:42.208035+0000 mon.a (mon.0) 2110 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:43 smithi151 bash[23756]: cluster 2024-03-04T15:57:42.217289+0000 mon.a (mon.0) 2111 : cluster 0 osdmap e273: 8 total, 8 up, 8 in 2024-03-04T15:57:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:43 smithi151 bash[23756]: audit 2024-03-04T15:57:42.219868+0000 mon.c (mon.2) 434 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-33"}]: dispatch 2024-03-04T15:57:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:43 smithi151 bash[23756]: audit 2024-03-04T15:57:42.222417+0000 mon.a (mon.0) 2112 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-33"}]: dispatch 2024-03-04T15:57:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:43 smithi151 bash[23756]: cluster 2024-03-04T15:57:43.096795+0000 mon.a (mon.0) 2113 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:44.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:44 smithi117 bash[21480]: cluster 2024-03-04T15:57:42.897018+0000 mgr.y (mgr.24428) 326 : cluster 0 pgmap v416: 244 pgs: 16 creating+peering, 228 active+clean; 4.6 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-04T15:57:44.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:44 smithi117 bash[21480]: cluster 2024-03-04T15:57:43.209377+0000 mon.a (mon.0) 2114 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:57:44.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:44 smithi117 bash[21480]: audit 2024-03-04T15:57:43.217290+0000 mon.a (mon.0) 2115 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-33"}]': finished 2024-03-04T15:57:44.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:44 smithi117 bash[21480]: cluster 2024-03-04T15:57:43.226861+0000 mon.a (mon.0) 2116 : cluster 0 osdmap e274: 8 total, 8 up, 8 in 2024-03-04T15:57:44.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:44 smithi117 bash[21480]: audit 2024-03-04T15:57:43.230288+0000 mon.c (mon.2) 435 : audit 1 from='client.? 172.21.15.117:0/2285618511' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:44.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:44 smithi117 bash[21480]: audit 2024-03-04T15:57:43.230752+0000 mon.c (mon.2) 436 : audit 1 from='client.? 172.21.15.117:0/3335546711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:44.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:44 smithi117 bash[21480]: audit 2024-03-04T15:57:43.232840+0000 mon.a (mon.0) 2117 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:44.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:44 smithi117 bash[21480]: audit 2024-03-04T15:57:43.233182+0000 mon.a (mon.0) 2118 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:44.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:44 smithi117 bash[21480]: audit 2024-03-04T15:57:43.693470+0000 mon.c (mon.2) 437 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:57:44.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:44 smithi117 bash[28843]: cluster 2024-03-04T15:57:42.897018+0000 mgr.y (mgr.24428) 326 : cluster 0 pgmap v416: 244 pgs: 16 creating+peering, 228 active+clean; 4.6 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-04T15:57:44.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:44 smithi117 bash[28843]: cluster 2024-03-04T15:57:43.209377+0000 mon.a (mon.0) 2114 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:57:44.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:44 smithi117 bash[28843]: audit 2024-03-04T15:57:43.217290+0000 mon.a (mon.0) 2115 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-33"}]': finished 2024-03-04T15:57:44.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:44 smithi117 bash[28843]: cluster 2024-03-04T15:57:43.226861+0000 mon.a (mon.0) 2116 : cluster 0 osdmap e274: 8 total, 8 up, 8 in 2024-03-04T15:57:44.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:44 smithi117 bash[28843]: audit 2024-03-04T15:57:43.230288+0000 mon.c (mon.2) 435 : audit 1 from='client.? 172.21.15.117:0/2285618511' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:44.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:44 smithi117 bash[28843]: audit 2024-03-04T15:57:43.230752+0000 mon.c (mon.2) 436 : audit 1 from='client.? 172.21.15.117:0/3335546711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:44.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:44 smithi117 bash[28843]: audit 2024-03-04T15:57:43.232840+0000 mon.a (mon.0) 2117 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:44.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:44 smithi117 bash[28843]: audit 2024-03-04T15:57:43.233182+0000 mon.a (mon.0) 2118 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:44.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:44 smithi117 bash[28843]: audit 2024-03-04T15:57:43.693470+0000 mon.c (mon.2) 437 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:57:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:44 smithi151 bash[23756]: cluster 2024-03-04T15:57:42.897018+0000 mgr.y (mgr.24428) 326 : cluster 0 pgmap v416: 244 pgs: 16 creating+peering, 228 active+clean; 4.6 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-04T15:57:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:44 smithi151 bash[23756]: cluster 2024-03-04T15:57:43.209377+0000 mon.a (mon.0) 2114 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:57:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:44 smithi151 bash[23756]: audit 2024-03-04T15:57:43.217290+0000 mon.a (mon.0) 2115 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-33"}]': finished 2024-03-04T15:57:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:44 smithi151 bash[23756]: cluster 2024-03-04T15:57:43.226861+0000 mon.a (mon.0) 2116 : cluster 0 osdmap e274: 8 total, 8 up, 8 in 2024-03-04T15:57:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:44 smithi151 bash[23756]: audit 2024-03-04T15:57:43.230288+0000 mon.c (mon.2) 435 : audit 1 from='client.? 172.21.15.117:0/2285618511' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:44 smithi151 bash[23756]: audit 2024-03-04T15:57:43.230752+0000 mon.c (mon.2) 436 : audit 1 from='client.? 172.21.15.117:0/3335546711' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:44 smithi151 bash[23756]: audit 2024-03-04T15:57:43.232840+0000 mon.a (mon.0) 2117 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:44 smithi151 bash[23756]: audit 2024-03-04T15:57:43.233182+0000 mon.a (mon.0) 2118 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:44 smithi151 bash[23756]: audit 2024-03-04T15:57:43.693470+0000 mon.c (mon.2) 437 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:57:45.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:45 smithi117 bash[28843]: audit 2024-03-04T15:57:44.221634+0000 mon.a (mon.0) 2119 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi117-63915-40"}]': finished 2024-03-04T15:57:45.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:45 smithi117 bash[28843]: audit 2024-03-04T15:57:44.221920+0000 mon.a (mon.0) 2120 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi117-63923-50"}]': finished 2024-03-04T15:57:45.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:45 smithi117 bash[28843]: cluster 2024-03-04T15:57:44.229081+0000 mon.a (mon.0) 2121 : cluster 0 osdmap e275: 8 total, 8 up, 8 in 2024-03-04T15:57:45.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:45 smithi117 bash[28843]: audit 2024-03-04T15:57:44.234284+0000 mon.c (mon.2) 438 : audit 1 from='client.? 172.21.15.117:0/2285618511' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:45.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:45 smithi117 bash[28843]: audit 2024-03-04T15:57:44.234567+0000 mon.c (mon.2) 439 : audit 1 from='client.? 172.21.15.117:0/3335546711' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:45.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:45 smithi117 bash[28843]: audit 2024-03-04T15:57:44.249679+0000 mon.a (mon.0) 2122 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:45.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:45 smithi117 bash[28843]: audit 2024-03-04T15:57:44.250727+0000 mon.a (mon.0) 2123 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:45.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:45 smithi117 bash[21480]: audit 2024-03-04T15:57:44.221634+0000 mon.a (mon.0) 2119 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi117-63915-40"}]': finished 2024-03-04T15:57:45.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:45 smithi117 bash[21480]: audit 2024-03-04T15:57:44.221920+0000 mon.a (mon.0) 2120 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi117-63923-50"}]': finished 2024-03-04T15:57:45.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:45 smithi117 bash[21480]: cluster 2024-03-04T15:57:44.229081+0000 mon.a (mon.0) 2121 : cluster 0 osdmap e275: 8 total, 8 up, 8 in 2024-03-04T15:57:45.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:45 smithi117 bash[21480]: audit 2024-03-04T15:57:44.234284+0000 mon.c (mon.2) 438 : audit 1 from='client.? 172.21.15.117:0/2285618511' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:45.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:45 smithi117 bash[21480]: audit 2024-03-04T15:57:44.234567+0000 mon.c (mon.2) 439 : audit 1 from='client.? 172.21.15.117:0/3335546711' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:45.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:45 smithi117 bash[21480]: audit 2024-03-04T15:57:44.249679+0000 mon.a (mon.0) 2122 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:45.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:45 smithi117 bash[21480]: audit 2024-03-04T15:57:44.250727+0000 mon.a (mon.0) 2123 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:45.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:45 smithi151 bash[23756]: audit 2024-03-04T15:57:44.221634+0000 mon.a (mon.0) 2119 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi117-63915-40"}]': finished 2024-03-04T15:57:45.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:45 smithi151 bash[23756]: audit 2024-03-04T15:57:44.221920+0000 mon.a (mon.0) 2120 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi117-63923-50"}]': finished 2024-03-04T15:57:45.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:45 smithi151 bash[23756]: cluster 2024-03-04T15:57:44.229081+0000 mon.a (mon.0) 2121 : cluster 0 osdmap e275: 8 total, 8 up, 8 in 2024-03-04T15:57:45.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:45 smithi151 bash[23756]: audit 2024-03-04T15:57:44.234284+0000 mon.c (mon.2) 438 : audit 1 from='client.? 172.21.15.117:0/2285618511' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:45.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:45 smithi151 bash[23756]: audit 2024-03-04T15:57:44.234567+0000 mon.c (mon.2) 439 : audit 1 from='client.? 172.21.15.117:0/3335546711' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:45.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:45 smithi151 bash[23756]: audit 2024-03-04T15:57:44.249679+0000 mon.a (mon.0) 2122 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi117-63915-40"}]: dispatch 2024-03-04T15:57:45.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:45 smithi151 bash[23756]: audit 2024-03-04T15:57:44.250727+0000 mon.a (mon.0) 2123 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi117-63923-50"}]: dispatch 2024-03-04T15:57:46.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:57:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:57:46.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[21480]: cluster 2024-03-04T15:57:44.897730+0000 mgr.y (mgr.24428) 327 : cluster 0 pgmap v419: 196 pgs: 196 active+clean; 4.6 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:57:46.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[21480]: audit 2024-03-04T15:57:45.250793+0000 mon.a (mon.0) 2124 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi117-63915-40"}]': finished 2024-03-04T15:57:46.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[21480]: audit 2024-03-04T15:57:45.251059+0000 mon.a (mon.0) 2125 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi117-63923-50"}]': finished 2024-03-04T15:57:46.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[21480]: cluster 2024-03-04T15:57:45.259636+0000 mon.a (mon.0) 2126 : cluster 0 osdmap e276: 8 total, 8 up, 8 in 2024-03-04T15:57:46.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[21480]: audit 2024-03-04T15:57:45.263107+0000 mon.c (mon.2) 440 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:46.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[21480]: audit 2024-03-04T15:57:45.265110+0000 mon.a (mon.0) 2127 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:46.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[21480]: audit 2024-03-04T15:57:45.298507+0000 mon.c (mon.2) 441 : audit 1 from='client.? 172.21.15.117:0/3407950749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:46.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[21480]: audit 2024-03-04T15:57:45.298914+0000 mon.a (mon.0) 2128 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:46.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[21480]: audit 2024-03-04T15:57:45.299522+0000 mon.c (mon.2) 442 : audit 1 from='client.? 172.21.15.117:0/3407950749' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:46.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[21480]: audit 2024-03-04T15:57:45.299845+0000 mon.a (mon.0) 2129 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:46.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[21480]: audit 2024-03-04T15:57:45.300394+0000 mon.c (mon.2) 443 : audit 1 from='client.? 172.21.15.117:0/3407950749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi117-63915-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:46.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[21480]: audit 2024-03-04T15:57:45.300676+0000 mon.a (mon.0) 2130 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi117-63915-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:46.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[28843]: cluster 2024-03-04T15:57:44.897730+0000 mgr.y (mgr.24428) 327 : cluster 0 pgmap v419: 196 pgs: 196 active+clean; 4.6 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:57:46.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[28843]: audit 2024-03-04T15:57:45.250793+0000 mon.a (mon.0) 2124 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi117-63915-40"}]': finished 2024-03-04T15:57:46.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[28843]: audit 2024-03-04T15:57:45.251059+0000 mon.a (mon.0) 2125 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi117-63923-50"}]': finished 2024-03-04T15:57:46.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[28843]: cluster 2024-03-04T15:57:45.259636+0000 mon.a (mon.0) 2126 : cluster 0 osdmap e276: 8 total, 8 up, 8 in 2024-03-04T15:57:46.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[28843]: audit 2024-03-04T15:57:45.263107+0000 mon.c (mon.2) 440 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:46.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[28843]: audit 2024-03-04T15:57:45.265110+0000 mon.a (mon.0) 2127 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:46.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[28843]: audit 2024-03-04T15:57:45.298507+0000 mon.c (mon.2) 441 : audit 1 from='client.? 172.21.15.117:0/3407950749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:46.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[28843]: audit 2024-03-04T15:57:45.298914+0000 mon.a (mon.0) 2128 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:46.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[28843]: audit 2024-03-04T15:57:45.299522+0000 mon.c (mon.2) 442 : audit 1 from='client.? 172.21.15.117:0/3407950749' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:46.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[28843]: audit 2024-03-04T15:57:45.299845+0000 mon.a (mon.0) 2129 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:46.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[28843]: audit 2024-03-04T15:57:45.300394+0000 mon.c (mon.2) 443 : audit 1 from='client.? 172.21.15.117:0/3407950749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi117-63915-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:46.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:46 smithi117 bash[28843]: audit 2024-03-04T15:57:45.300676+0000 mon.a (mon.0) 2130 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi117-63915-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:46.655 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:46 smithi151 bash[23756]: cluster 2024-03-04T15:57:44.897730+0000 mgr.y (mgr.24428) 327 : cluster 0 pgmap v419: 196 pgs: 196 active+clean; 4.6 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:57:46.655 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:46 smithi151 bash[23756]: audit 2024-03-04T15:57:45.250793+0000 mon.a (mon.0) 2124 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi117-63915-40"}]': finished 2024-03-04T15:57:46.655 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:46 smithi151 bash[23756]: audit 2024-03-04T15:57:45.251059+0000 mon.a (mon.0) 2125 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi117-63923-50"}]': finished 2024-03-04T15:57:46.655 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:46 smithi151 bash[23756]: cluster 2024-03-04T15:57:45.259636+0000 mon.a (mon.0) 2126 : cluster 0 osdmap e276: 8 total, 8 up, 8 in 2024-03-04T15:57:46.655 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:46 smithi151 bash[23756]: audit 2024-03-04T15:57:45.263107+0000 mon.c (mon.2) 440 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:46.655 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:46 smithi151 bash[23756]: audit 2024-03-04T15:57:45.265110+0000 mon.a (mon.0) 2127 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:46.655 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:46 smithi151 bash[23756]: audit 2024-03-04T15:57:45.298507+0000 mon.c (mon.2) 441 : audit 1 from='client.? 172.21.15.117:0/3407950749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:46.655 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:46 smithi151 bash[23756]: audit 2024-03-04T15:57:45.298914+0000 mon.a (mon.0) 2128 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:46.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:46 smithi151 bash[23756]: audit 2024-03-04T15:57:45.299522+0000 mon.c (mon.2) 442 : audit 1 from='client.? 172.21.15.117:0/3407950749' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:46.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:46 smithi151 bash[23756]: audit 2024-03-04T15:57:45.299845+0000 mon.a (mon.0) 2129 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:46.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:46 smithi151 bash[23756]: audit 2024-03-04T15:57:45.300394+0000 mon.c (mon.2) 443 : audit 1 from='client.? 172.21.15.117:0/3407950749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi117-63915-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:46.656 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:46 smithi151 bash[23756]: audit 2024-03-04T15:57:45.300676+0000 mon.a (mon.0) 2130 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi117-63915-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:46.931 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:57:46 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:57:48.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:47 smithi117 bash[21480]: audit 2024-03-04T15:57:46.267866+0000 mon.a (mon.0) 2131 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:48.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:47 smithi117 bash[21480]: audit 2024-03-04T15:57:46.268084+0000 mon.a (mon.0) 2132 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi117-63915-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:48.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:47 smithi117 bash[21480]: cluster 2024-03-04T15:57:46.276065+0000 mon.a (mon.0) 2133 : cluster 0 osdmap e277: 8 total, 8 up, 8 in 2024-03-04T15:57:48.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:47 smithi117 bash[21480]: audit 2024-03-04T15:57:46.279746+0000 mon.c (mon.2) 444 : audit 1 from='client.? 172.21.15.117:0/3407950749' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi117-63915-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:48.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:47 smithi117 bash[21480]: audit 2024-03-04T15:57:46.280654+0000 mon.a (mon.0) 2134 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi117-63915-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:48.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:47 smithi117 bash[21480]: audit 2024-03-04T15:57:46.283043+0000 mon.c (mon.2) 445 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:48.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:47 smithi117 bash[21480]: audit 2024-03-04T15:57:46.294647+0000 mon.a (mon.0) 2135 : audit 1 from='client.? 172.21.15.117:0/2454617559' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi117-63923-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:48.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:47 smithi117 bash[21480]: audit 2024-03-04T15:57:46.296045+0000 mon.a (mon.0) 2136 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:48.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:47 smithi117 bash[28843]: audit 2024-03-04T15:57:46.267866+0000 mon.a (mon.0) 2131 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:48.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:47 smithi117 bash[28843]: audit 2024-03-04T15:57:46.268084+0000 mon.a (mon.0) 2132 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi117-63915-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:48.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:47 smithi117 bash[28843]: cluster 2024-03-04T15:57:46.276065+0000 mon.a (mon.0) 2133 : cluster 0 osdmap e277: 8 total, 8 up, 8 in 2024-03-04T15:57:48.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:47 smithi117 bash[28843]: audit 2024-03-04T15:57:46.279746+0000 mon.c (mon.2) 444 : audit 1 from='client.? 172.21.15.117:0/3407950749' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi117-63915-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:48.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:47 smithi117 bash[28843]: audit 2024-03-04T15:57:46.280654+0000 mon.a (mon.0) 2134 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi117-63915-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:48.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:47 smithi117 bash[28843]: audit 2024-03-04T15:57:46.283043+0000 mon.c (mon.2) 445 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:48.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:47 smithi117 bash[28843]: audit 2024-03-04T15:57:46.294647+0000 mon.a (mon.0) 2135 : audit 1 from='client.? 172.21.15.117:0/2454617559' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi117-63923-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:48.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:47 smithi117 bash[28843]: audit 2024-03-04T15:57:46.296045+0000 mon.a (mon.0) 2136 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:48.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:47 smithi151 bash[23756]: audit 2024-03-04T15:57:46.267866+0000 mon.a (mon.0) 2131 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:48.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:47 smithi151 bash[23756]: audit 2024-03-04T15:57:46.268084+0000 mon.a (mon.0) 2132 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi117-63915-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:48.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:47 smithi151 bash[23756]: cluster 2024-03-04T15:57:46.276065+0000 mon.a (mon.0) 2133 : cluster 0 osdmap e277: 8 total, 8 up, 8 in 2024-03-04T15:57:48.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:47 smithi151 bash[23756]: audit 2024-03-04T15:57:46.279746+0000 mon.c (mon.2) 444 : audit 1 from='client.? 172.21.15.117:0/3407950749' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi117-63915-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:48.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:47 smithi151 bash[23756]: audit 2024-03-04T15:57:46.280654+0000 mon.a (mon.0) 2134 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi117-63915-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:48.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:47 smithi151 bash[23756]: audit 2024-03-04T15:57:46.283043+0000 mon.c (mon.2) 445 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:48.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:47 smithi151 bash[23756]: audit 2024-03-04T15:57:46.294647+0000 mon.a (mon.0) 2135 : audit 1 from='client.? 172.21.15.117:0/2454617559' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi117-63923-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:48.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:47 smithi151 bash[23756]: audit 2024-03-04T15:57:46.296045+0000 mon.a (mon.0) 2136 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:57:49.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[21480]: audit 2024-03-04T15:57:46.657228+0000 mgr.y (mgr.24428) 328 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:57:49.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[21480]: cluster 2024-03-04T15:57:46.899964+0000 mgr.y (mgr.24428) 329 : cluster 0 pgmap v422: 260 pgs: 32 creating+peering, 228 active+clean; 4.6 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T15:57:49.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[21480]: audit 2024-03-04T15:57:47.272044+0000 mon.a (mon.0) 2137 : audit 1 from='client.? 172.21.15.117:0/2454617559' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi117-63923-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:49.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[21480]: audit 2024-03-04T15:57:47.272178+0000 mon.a (mon.0) 2138 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-35", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:57:49.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[21480]: audit 2024-03-04T15:57:47.283353+0000 mon.c (mon.2) 446 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-35"}]: dispatch 2024-03-04T15:57:49.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[21480]: cluster 2024-03-04T15:57:47.739069+0000 mon.a (mon.0) 2139 : cluster 0 osdmap e278: 8 total, 8 up, 8 in 2024-03-04T15:57:49.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[21480]: audit 2024-03-04T15:57:47.749243+0000 mon.a (mon.0) 2140 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-35"}]: dispatch 2024-03-04T15:57:49.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[21480]: cluster 2024-03-04T15:57:48.098349+0000 mon.a (mon.0) 2141 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:49.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[21480]: audit 2024-03-04T15:57:48.275303+0000 mon.a (mon.0) 2142 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi117-63915-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi117-63915-41"}]': finished 2024-03-04T15:57:49.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[21480]: audit 2024-03-04T15:57:48.275593+0000 mon.a (mon.0) 2143 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-35"}]': finished 2024-03-04T15:57:49.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[21480]: cluster 2024-03-04T15:57:48.287194+0000 mon.a (mon.0) 2144 : cluster 0 osdmap e279: 8 total, 8 up, 8 in 2024-03-04T15:57:49.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[21480]: audit 2024-03-04T15:57:48.288967+0000 mon.c (mon.2) 447 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-35", "mode": "writeback"}]: dispatch 2024-03-04T15:57:49.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[21480]: audit 2024-03-04T15:57:48.294573+0000 mon.a (mon.0) 2145 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-35", "mode": "writeback"}]: dispatch 2024-03-04T15:57:49.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[21480]: audit 2024-03-04T15:57:48.344736+0000 mon.a (mon.0) 2146 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi117-63923-52"}]: dispatch 2024-03-04T15:57:49.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[21480]: audit 2024-03-04T15:57:48.345727+0000 mon.a (mon.0) 2147 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi117-63923-52"}]: dispatch 2024-03-04T15:57:49.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[21480]: audit 2024-03-04T15:57:48.346088+0000 mon.a (mon.0) 2148 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi117-63923-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:49.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[28843]: audit 2024-03-04T15:57:46.657228+0000 mgr.y (mgr.24428) 328 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:57:49.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[28843]: cluster 2024-03-04T15:57:46.899964+0000 mgr.y (mgr.24428) 329 : cluster 0 pgmap v422: 260 pgs: 32 creating+peering, 228 active+clean; 4.6 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T15:57:49.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[28843]: audit 2024-03-04T15:57:47.272044+0000 mon.a (mon.0) 2137 : audit 1 from='client.? 172.21.15.117:0/2454617559' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi117-63923-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:49.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[28843]: audit 2024-03-04T15:57:47.272178+0000 mon.a (mon.0) 2138 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-35", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:57:49.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[28843]: audit 2024-03-04T15:57:47.283353+0000 mon.c (mon.2) 446 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-35"}]: dispatch 2024-03-04T15:57:49.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[28843]: cluster 2024-03-04T15:57:47.739069+0000 mon.a (mon.0) 2139 : cluster 0 osdmap e278: 8 total, 8 up, 8 in 2024-03-04T15:57:49.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[28843]: audit 2024-03-04T15:57:47.749243+0000 mon.a (mon.0) 2140 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-35"}]: dispatch 2024-03-04T15:57:49.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[28843]: cluster 2024-03-04T15:57:48.098349+0000 mon.a (mon.0) 2141 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:49.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[28843]: audit 2024-03-04T15:57:48.275303+0000 mon.a (mon.0) 2142 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi117-63915-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi117-63915-41"}]': finished 2024-03-04T15:57:49.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[28843]: audit 2024-03-04T15:57:48.275593+0000 mon.a (mon.0) 2143 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-35"}]': finished 2024-03-04T15:57:49.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[28843]: cluster 2024-03-04T15:57:48.287194+0000 mon.a (mon.0) 2144 : cluster 0 osdmap e279: 8 total, 8 up, 8 in 2024-03-04T15:57:49.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[28843]: audit 2024-03-04T15:57:48.288967+0000 mon.c (mon.2) 447 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-35", "mode": "writeback"}]: dispatch 2024-03-04T15:57:49.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[28843]: audit 2024-03-04T15:57:48.294573+0000 mon.a (mon.0) 2145 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-35", "mode": "writeback"}]: dispatch 2024-03-04T15:57:49.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[28843]: audit 2024-03-04T15:57:48.344736+0000 mon.a (mon.0) 2146 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi117-63923-52"}]: dispatch 2024-03-04T15:57:49.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[28843]: audit 2024-03-04T15:57:48.345727+0000 mon.a (mon.0) 2147 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi117-63923-52"}]: dispatch 2024-03-04T15:57:49.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:48 smithi117 bash[28843]: audit 2024-03-04T15:57:48.346088+0000 mon.a (mon.0) 2148 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi117-63923-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:49.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:48 smithi151 bash[23756]: audit 2024-03-04T15:57:46.657228+0000 mgr.y (mgr.24428) 328 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:57:49.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:48 smithi151 bash[23756]: cluster 2024-03-04T15:57:46.899964+0000 mgr.y (mgr.24428) 329 : cluster 0 pgmap v422: 260 pgs: 32 creating+peering, 228 active+clean; 4.6 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T15:57:49.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:48 smithi151 bash[23756]: audit 2024-03-04T15:57:47.272044+0000 mon.a (mon.0) 2137 : audit 1 from='client.? 172.21.15.117:0/2454617559' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi117-63923-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:57:49.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:48 smithi151 bash[23756]: audit 2024-03-04T15:57:47.272178+0000 mon.a (mon.0) 2138 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-35", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:57:49.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:48 smithi151 bash[23756]: audit 2024-03-04T15:57:47.283353+0000 mon.c (mon.2) 446 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-35"}]: dispatch 2024-03-04T15:57:49.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:48 smithi151 bash[23756]: cluster 2024-03-04T15:57:47.739069+0000 mon.a (mon.0) 2139 : cluster 0 osdmap e278: 8 total, 8 up, 8 in 2024-03-04T15:57:49.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:48 smithi151 bash[23756]: audit 2024-03-04T15:57:47.749243+0000 mon.a (mon.0) 2140 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-35"}]: dispatch 2024-03-04T15:57:49.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:48 smithi151 bash[23756]: cluster 2024-03-04T15:57:48.098349+0000 mon.a (mon.0) 2141 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:49.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:48 smithi151 bash[23756]: audit 2024-03-04T15:57:48.275303+0000 mon.a (mon.0) 2142 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi117-63915-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi117-63915-41"}]': finished 2024-03-04T15:57:49.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:48 smithi151 bash[23756]: audit 2024-03-04T15:57:48.275593+0000 mon.a (mon.0) 2143 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-35"}]': finished 2024-03-04T15:57:49.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:48 smithi151 bash[23756]: cluster 2024-03-04T15:57:48.287194+0000 mon.a (mon.0) 2144 : cluster 0 osdmap e279: 8 total, 8 up, 8 in 2024-03-04T15:57:49.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:48 smithi151 bash[23756]: audit 2024-03-04T15:57:48.288967+0000 mon.c (mon.2) 447 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-35", "mode": "writeback"}]: dispatch 2024-03-04T15:57:49.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:48 smithi151 bash[23756]: audit 2024-03-04T15:57:48.294573+0000 mon.a (mon.0) 2145 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-35", "mode": "writeback"}]: dispatch 2024-03-04T15:57:49.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:48 smithi151 bash[23756]: audit 2024-03-04T15:57:48.344736+0000 mon.a (mon.0) 2146 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi117-63923-52"}]: dispatch 2024-03-04T15:57:49.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:48 smithi151 bash[23756]: audit 2024-03-04T15:57:48.345727+0000 mon.a (mon.0) 2147 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi117-63923-52"}]: dispatch 2024-03-04T15:57:49.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:48 smithi151 bash[23756]: audit 2024-03-04T15:57:48.346088+0000 mon.a (mon.0) 2148 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi117-63923-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:50.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:49 smithi117 bash[21480]: cluster 2024-03-04T15:57:48.900766+0000 mgr.y (mgr.24428) 330 : cluster 0 pgmap v425: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T15:57:50.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:49 smithi117 bash[21480]: cluster 2024-03-04T15:57:49.275954+0000 mon.a (mon.0) 2149 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:57:50.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:49 smithi117 bash[21480]: audit 2024-03-04T15:57:49.278494+0000 mon.a (mon.0) 2150 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-35", "mode": "writeback"}]': finished 2024-03-04T15:57:50.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:49 smithi117 bash[21480]: audit 2024-03-04T15:57:49.278721+0000 mon.a (mon.0) 2151 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi117-63923-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:50.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:49 smithi117 bash[21480]: cluster 2024-03-04T15:57:49.286361+0000 mon.a (mon.0) 2152 : cluster 0 osdmap e280: 8 total, 8 up, 8 in 2024-03-04T15:57:50.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:49 smithi117 bash[21480]: audit 2024-03-04T15:57:49.287599+0000 mon.a (mon.0) 2153 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi117-63923-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi117-63923-52"}]: dispatch 2024-03-04T15:57:50.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:49 smithi117 bash[28843]: cluster 2024-03-04T15:57:48.900766+0000 mgr.y (mgr.24428) 330 : cluster 0 pgmap v425: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T15:57:50.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:49 smithi117 bash[28843]: cluster 2024-03-04T15:57:49.275954+0000 mon.a (mon.0) 2149 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:57:50.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:49 smithi117 bash[28843]: audit 2024-03-04T15:57:49.278494+0000 mon.a (mon.0) 2150 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-35", "mode": "writeback"}]': finished 2024-03-04T15:57:50.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:49 smithi117 bash[28843]: audit 2024-03-04T15:57:49.278721+0000 mon.a (mon.0) 2151 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi117-63923-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:50.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:49 smithi117 bash[28843]: cluster 2024-03-04T15:57:49.286361+0000 mon.a (mon.0) 2152 : cluster 0 osdmap e280: 8 total, 8 up, 8 in 2024-03-04T15:57:50.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:49 smithi117 bash[28843]: audit 2024-03-04T15:57:49.287599+0000 mon.a (mon.0) 2153 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi117-63923-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi117-63923-52"}]: dispatch 2024-03-04T15:57:50.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:49 smithi151 bash[23756]: cluster 2024-03-04T15:57:48.900766+0000 mgr.y (mgr.24428) 330 : cluster 0 pgmap v425: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T15:57:50.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:49 smithi151 bash[23756]: cluster 2024-03-04T15:57:49.275954+0000 mon.a (mon.0) 2149 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:57:50.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:49 smithi151 bash[23756]: audit 2024-03-04T15:57:49.278494+0000 mon.a (mon.0) 2150 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-35", "mode": "writeback"}]': finished 2024-03-04T15:57:50.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:49 smithi151 bash[23756]: audit 2024-03-04T15:57:49.278721+0000 mon.a (mon.0) 2151 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi117-63923-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:50.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:49 smithi151 bash[23756]: cluster 2024-03-04T15:57:49.286361+0000 mon.a (mon.0) 2152 : cluster 0 osdmap e280: 8 total, 8 up, 8 in 2024-03-04T15:57:50.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:49 smithi151 bash[23756]: audit 2024-03-04T15:57:49.287599+0000 mon.a (mon.0) 2153 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi117-63923-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi117-63923-52"}]: dispatch 2024-03-04T15:57:51.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:51 smithi117 bash[21480]: cluster 2024-03-04T15:57:50.742983+0000 mon.a (mon.0) 2154 : cluster 0 osdmap e281: 8 total, 8 up, 8 in 2024-03-04T15:57:51.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:51 smithi117 bash[21480]: audit 2024-03-04T15:57:50.744360+0000 mon.c (mon.2) 448 : audit 1 from='client.? 172.21.15.117:0/3407950749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:51.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:51 smithi117 bash[21480]: audit 2024-03-04T15:57:50.744948+0000 mon.a (mon.0) 2155 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:51.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:51 smithi117 bash[21480]: cluster 2024-03-04T15:57:50.901569+0000 mgr.y (mgr.24428) 331 : cluster 0 pgmap v428: 228 pgs: 228 active+clean; 4.6 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:57:51.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:51 smithi117 bash[28843]: cluster 2024-03-04T15:57:50.742983+0000 mon.a (mon.0) 2154 : cluster 0 osdmap e281: 8 total, 8 up, 8 in 2024-03-04T15:57:51.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:51 smithi117 bash[28843]: audit 2024-03-04T15:57:50.744360+0000 mon.c (mon.2) 448 : audit 1 from='client.? 172.21.15.117:0/3407950749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:51.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:51 smithi117 bash[28843]: audit 2024-03-04T15:57:50.744948+0000 mon.a (mon.0) 2155 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:51.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:51 smithi117 bash[28843]: cluster 2024-03-04T15:57:50.901569+0000 mgr.y (mgr.24428) 331 : cluster 0 pgmap v428: 228 pgs: 228 active+clean; 4.6 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:57:51.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:51 smithi151 bash[23756]: cluster 2024-03-04T15:57:50.742983+0000 mon.a (mon.0) 2154 : cluster 0 osdmap e281: 8 total, 8 up, 8 in 2024-03-04T15:57:51.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:51 smithi151 bash[23756]: audit 2024-03-04T15:57:50.744360+0000 mon.c (mon.2) 448 : audit 1 from='client.? 172.21.15.117:0/3407950749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:51.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:51 smithi151 bash[23756]: audit 2024-03-04T15:57:50.744948+0000 mon.a (mon.0) 2155 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:51.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:51 smithi151 bash[23756]: cluster 2024-03-04T15:57:50.901569+0000 mgr.y (mgr.24428) 331 : cluster 0 pgmap v428: 228 pgs: 228 active+clean; 4.6 MiB data, 628 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:57:52.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:52 smithi117 bash[21480]: audit 2024-03-04T15:57:51.294868+0000 mon.a (mon.0) 2156 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi117-63923-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi117-63923-52"}]': finished 2024-03-04T15:57:52.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:52 smithi117 bash[21480]: audit 2024-03-04T15:57:51.295097+0000 mon.a (mon.0) 2157 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi117-63915-41"}]': finished 2024-03-04T15:57:52.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:52 smithi117 bash[21480]: cluster 2024-03-04T15:57:51.302577+0000 mon.a (mon.0) 2158 : cluster 0 osdmap e282: 8 total, 8 up, 8 in 2024-03-04T15:57:52.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:52 smithi117 bash[21480]: audit 2024-03-04T15:57:51.310866+0000 mon.c (mon.2) 449 : audit 1 from='client.? 172.21.15.117:0/3407950749' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:52.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:52 smithi117 bash[21480]: audit 2024-03-04T15:57:51.318241+0000 mon.a (mon.0) 2159 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:52.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:52 smithi117 bash[21480]: audit 2024-03-04T15:57:51.398233+0000 mon.c (mon.2) 450 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:52.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:52 smithi117 bash[21480]: audit 2024-03-04T15:57:51.398752+0000 mon.a (mon.0) 2160 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:52.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:52 smithi117 bash[28843]: audit 2024-03-04T15:57:51.294868+0000 mon.a (mon.0) 2156 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi117-63923-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi117-63923-52"}]': finished 2024-03-04T15:57:52.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:52 smithi117 bash[28843]: audit 2024-03-04T15:57:51.295097+0000 mon.a (mon.0) 2157 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi117-63915-41"}]': finished 2024-03-04T15:57:52.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:52 smithi117 bash[28843]: cluster 2024-03-04T15:57:51.302577+0000 mon.a (mon.0) 2158 : cluster 0 osdmap e282: 8 total, 8 up, 8 in 2024-03-04T15:57:52.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:52 smithi117 bash[28843]: audit 2024-03-04T15:57:51.310866+0000 mon.c (mon.2) 449 : audit 1 from='client.? 172.21.15.117:0/3407950749' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:52.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:52 smithi117 bash[28843]: audit 2024-03-04T15:57:51.318241+0000 mon.a (mon.0) 2159 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:52.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:52 smithi117 bash[28843]: audit 2024-03-04T15:57:51.398233+0000 mon.c (mon.2) 450 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:52.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:52 smithi117 bash[28843]: audit 2024-03-04T15:57:51.398752+0000 mon.a (mon.0) 2160 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:52 smithi151 bash[23756]: audit 2024-03-04T15:57:51.294868+0000 mon.a (mon.0) 2156 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi117-63923-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi117-63923-52"}]': finished 2024-03-04T15:57:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:52 smithi151 bash[23756]: audit 2024-03-04T15:57:51.295097+0000 mon.a (mon.0) 2157 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi117-63915-41"}]': finished 2024-03-04T15:57:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:52 smithi151 bash[23756]: cluster 2024-03-04T15:57:51.302577+0000 mon.a (mon.0) 2158 : cluster 0 osdmap e282: 8 total, 8 up, 8 in 2024-03-04T15:57:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:52 smithi151 bash[23756]: audit 2024-03-04T15:57:51.310866+0000 mon.c (mon.2) 449 : audit 1 from='client.? 172.21.15.117:0/3407950749' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:52 smithi151 bash[23756]: audit 2024-03-04T15:57:51.318241+0000 mon.a (mon.0) 2159 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi117-63915-41"}]: dispatch 2024-03-04T15:57:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:52 smithi151 bash[23756]: audit 2024-03-04T15:57:51.398233+0000 mon.c (mon.2) 450 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:52 smithi151 bash[23756]: audit 2024-03-04T15:57:51.398752+0000 mon.a (mon.0) 2160 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:53.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:53 smithi151 bash[23756]: audit 2024-03-04T15:57:52.306279+0000 mon.a (mon.0) 2161 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi117-63915-41"}]': finished 2024-03-04T15:57:53.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:53 smithi151 bash[23756]: audit 2024-03-04T15:57:52.306529+0000 mon.a (mon.0) 2162 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:53.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:53 smithi151 bash[23756]: cluster 2024-03-04T15:57:52.314416+0000 mon.a (mon.0) 2163 : cluster 0 osdmap e283: 8 total, 8 up, 8 in 2024-03-04T15:57:53.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:53 smithi151 bash[23756]: audit 2024-03-04T15:57:52.341124+0000 mon.c (mon.2) 451 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-35"}]: dispatch 2024-03-04T15:57:53.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:53 smithi151 bash[23756]: audit 2024-03-04T15:57:52.341699+0000 mon.a (mon.0) 2164 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-35"}]: dispatch 2024-03-04T15:57:53.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:53 smithi151 bash[23756]: audit 2024-03-04T15:57:52.347946+0000 mon.c (mon.2) 452 : audit 1 from='client.? 172.21.15.117:0/986178260' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:53.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:53 smithi151 bash[23756]: audit 2024-03-04T15:57:52.348321+0000 mon.a (mon.0) 2165 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:53.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:53 smithi151 bash[23756]: audit 2024-03-04T15:57:52.348949+0000 mon.c (mon.2) 453 : audit 1 from='client.? 172.21.15.117:0/986178260' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:53.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:53 smithi151 bash[23756]: audit 2024-03-04T15:57:52.349298+0000 mon.a (mon.0) 2166 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:53.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:53 smithi151 bash[23756]: audit 2024-03-04T15:57:52.349802+0000 mon.c (mon.2) 454 : audit 1 from='client.? 172.21.15.117:0/986178260' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi117-63915-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:53.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:53 smithi151 bash[23756]: audit 2024-03-04T15:57:52.350129+0000 mon.a (mon.0) 2167 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi117-63915-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:53.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:53 smithi151 bash[23756]: cluster 2024-03-04T15:57:52.902369+0000 mgr.y (mgr.24428) 332 : cluster 0 pgmap v431: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-04T15:57:53.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:53 smithi151 bash[23756]: cluster 2024-03-04T15:57:53.099815+0000 mon.a (mon.0) 2168 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:53.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[21480]: audit 2024-03-04T15:57:52.306279+0000 mon.a (mon.0) 2161 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi117-63915-41"}]': finished 2024-03-04T15:57:53.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[21480]: audit 2024-03-04T15:57:52.306529+0000 mon.a (mon.0) 2162 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:53.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[21480]: cluster 2024-03-04T15:57:52.314416+0000 mon.a (mon.0) 2163 : cluster 0 osdmap e283: 8 total, 8 up, 8 in 2024-03-04T15:57:53.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[21480]: audit 2024-03-04T15:57:52.341124+0000 mon.c (mon.2) 451 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-35"}]: dispatch 2024-03-04T15:57:53.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[21480]: audit 2024-03-04T15:57:52.341699+0000 mon.a (mon.0) 2164 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-35"}]: dispatch 2024-03-04T15:57:53.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[21480]: audit 2024-03-04T15:57:52.347946+0000 mon.c (mon.2) 452 : audit 1 from='client.? 172.21.15.117:0/986178260' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:53.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[21480]: audit 2024-03-04T15:57:52.348321+0000 mon.a (mon.0) 2165 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:53.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[21480]: audit 2024-03-04T15:57:52.348949+0000 mon.c (mon.2) 453 : audit 1 from='client.? 172.21.15.117:0/986178260' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:53.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[21480]: audit 2024-03-04T15:57:52.349298+0000 mon.a (mon.0) 2166 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:53.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[21480]: audit 2024-03-04T15:57:52.349802+0000 mon.c (mon.2) 454 : audit 1 from='client.? 172.21.15.117:0/986178260' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi117-63915-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:53.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[21480]: audit 2024-03-04T15:57:52.350129+0000 mon.a (mon.0) 2167 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi117-63915-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:53.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[21480]: cluster 2024-03-04T15:57:52.902369+0000 mgr.y (mgr.24428) 332 : cluster 0 pgmap v431: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-04T15:57:53.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[21480]: cluster 2024-03-04T15:57:53.099815+0000 mon.a (mon.0) 2168 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:53.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[28843]: audit 2024-03-04T15:57:52.306279+0000 mon.a (mon.0) 2161 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi117-63915-41"}]': finished 2024-03-04T15:57:53.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[28843]: audit 2024-03-04T15:57:52.306529+0000 mon.a (mon.0) 2162 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:53.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[28843]: cluster 2024-03-04T15:57:52.314416+0000 mon.a (mon.0) 2163 : cluster 0 osdmap e283: 8 total, 8 up, 8 in 2024-03-04T15:57:53.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[28843]: audit 2024-03-04T15:57:52.341124+0000 mon.c (mon.2) 451 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-35"}]: dispatch 2024-03-04T15:57:53.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[28843]: audit 2024-03-04T15:57:52.341699+0000 mon.a (mon.0) 2164 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-35"}]: dispatch 2024-03-04T15:57:53.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[28843]: audit 2024-03-04T15:57:52.347946+0000 mon.c (mon.2) 452 : audit 1 from='client.? 172.21.15.117:0/986178260' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:53.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[28843]: audit 2024-03-04T15:57:52.348321+0000 mon.a (mon.0) 2165 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:53.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[28843]: audit 2024-03-04T15:57:52.348949+0000 mon.c (mon.2) 453 : audit 1 from='client.? 172.21.15.117:0/986178260' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:53.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[28843]: audit 2024-03-04T15:57:52.349298+0000 mon.a (mon.0) 2166 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:53.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[28843]: audit 2024-03-04T15:57:52.349802+0000 mon.c (mon.2) 454 : audit 1 from='client.? 172.21.15.117:0/986178260' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi117-63915-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:53.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[28843]: audit 2024-03-04T15:57:52.350129+0000 mon.a (mon.0) 2167 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi117-63915-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:53.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[28843]: cluster 2024-03-04T15:57:52.902369+0000 mgr.y (mgr.24428) 332 : cluster 0 pgmap v431: 236 pgs: 8 creating+peering, 228 active+clean; 4.6 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-04T15:57:53.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:53 smithi117 bash[28843]: cluster 2024-03-04T15:57:53.099815+0000 mon.a (mon.0) 2168 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:55.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:54 smithi117 bash[21480]: audit 2024-03-04T15:57:53.317826+0000 mon.a (mon.0) 2169 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-35"}]': finished 2024-03-04T15:57:55.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:54 smithi117 bash[21480]: audit 2024-03-04T15:57:53.318084+0000 mon.a (mon.0) 2170 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi117-63915-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:55.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:54 smithi117 bash[21480]: cluster 2024-03-04T15:57:53.330513+0000 mon.a (mon.0) 2171 : cluster 0 osdmap e284: 8 total, 8 up, 8 in 2024-03-04T15:57:55.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:54 smithi117 bash[21480]: audit 2024-03-04T15:57:53.332917+0000 mon.c (mon.2) 455 : audit 1 from='client.? 172.21.15.117:0/986178260' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi117-63915-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:55.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:54 smithi117 bash[21480]: audit 2024-03-04T15:57:53.333895+0000 mon.a (mon.0) 2172 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi117-63915-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:55.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:54 smithi117 bash[21480]: audit 2024-03-04T15:57:53.335321+0000 mon.a (mon.0) 2173 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi117-63923-52"}]: dispatch 2024-03-04T15:57:55.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:54 smithi117 bash[28843]: audit 2024-03-04T15:57:53.317826+0000 mon.a (mon.0) 2169 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-35"}]': finished 2024-03-04T15:57:55.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:54 smithi117 bash[28843]: audit 2024-03-04T15:57:53.318084+0000 mon.a (mon.0) 2170 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi117-63915-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:55.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:54 smithi117 bash[28843]: cluster 2024-03-04T15:57:53.330513+0000 mon.a (mon.0) 2171 : cluster 0 osdmap e284: 8 total, 8 up, 8 in 2024-03-04T15:57:55.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:54 smithi117 bash[28843]: audit 2024-03-04T15:57:53.332917+0000 mon.c (mon.2) 455 : audit 1 from='client.? 172.21.15.117:0/986178260' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi117-63915-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:55.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:54 smithi117 bash[28843]: audit 2024-03-04T15:57:53.333895+0000 mon.a (mon.0) 2172 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi117-63915-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:55.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:54 smithi117 bash[28843]: audit 2024-03-04T15:57:53.335321+0000 mon.a (mon.0) 2173 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi117-63923-52"}]: dispatch 2024-03-04T15:57:55.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:54 smithi151 bash[23756]: audit 2024-03-04T15:57:53.317826+0000 mon.a (mon.0) 2169 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-35"}]': finished 2024-03-04T15:57:55.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:54 smithi151 bash[23756]: audit 2024-03-04T15:57:53.318084+0000 mon.a (mon.0) 2170 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi117-63915-42", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:55.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:54 smithi151 bash[23756]: cluster 2024-03-04T15:57:53.330513+0000 mon.a (mon.0) 2171 : cluster 0 osdmap e284: 8 total, 8 up, 8 in 2024-03-04T15:57:55.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:54 smithi151 bash[23756]: audit 2024-03-04T15:57:53.332917+0000 mon.c (mon.2) 455 : audit 1 from='client.? 172.21.15.117:0/986178260' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi117-63915-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:55.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:54 smithi151 bash[23756]: audit 2024-03-04T15:57:53.333895+0000 mon.a (mon.0) 2172 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi117-63915-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:55.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:54 smithi151 bash[23756]: audit 2024-03-04T15:57:53.335321+0000 mon.a (mon.0) 2173 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi117-63923-52"}]: dispatch 2024-03-04T15:57:56.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[21480]: audit 2024-03-04T15:57:54.321430+0000 mon.a (mon.0) 2174 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi117-63923-52"}]': finished 2024-03-04T15:57:56.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[21480]: cluster 2024-03-04T15:57:54.782706+0000 mon.a (mon.0) 2175 : cluster 0 osdmap e285: 8 total, 8 up, 8 in 2024-03-04T15:57:56.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[21480]: audit 2024-03-04T15:57:54.783886+0000 mon.a (mon.0) 2176 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi117-63923-52"}]: dispatch 2024-03-04T15:57:56.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[21480]: audit 2024-03-04T15:57:54.809390+0000 mon.c (mon.2) 456 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:56.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[21480]: audit 2024-03-04T15:57:54.809965+0000 mon.a (mon.0) 2177 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:56.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[21480]: cluster 2024-03-04T15:57:54.903258+0000 mgr.y (mgr.24428) 333 : cluster 0 pgmap v434: 228 pgs: 228 active+clean; 4.6 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-04T15:57:56.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[21480]: audit 2024-03-04T15:57:55.325516+0000 mon.a (mon.0) 2178 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi117-63915-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi117-63915-42"}]': finished 2024-03-04T15:57:56.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[21480]: audit 2024-03-04T15:57:55.325843+0000 mon.a (mon.0) 2179 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi117-63923-52"}]': finished 2024-03-04T15:57:56.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[21480]: audit 2024-03-04T15:57:55.326027+0000 mon.a (mon.0) 2180 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:56.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[21480]: cluster 2024-03-04T15:57:55.334527+0000 mon.a (mon.0) 2181 : cluster 0 osdmap e286: 8 total, 8 up, 8 in 2024-03-04T15:57:56.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[21480]: audit 2024-03-04T15:57:55.336552+0000 mon.c (mon.2) 457 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-35"}]: dispatch 2024-03-04T15:57:56.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[21480]: audit 2024-03-04T15:57:55.338467+0000 mon.a (mon.0) 2182 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-35"}]: dispatch 2024-03-04T15:57:56.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[21480]: audit 2024-03-04T15:57:55.367756+0000 mon.c (mon.2) 458 : audit 1 from='client.? 172.21.15.117:0/1447755774' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:57:56.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[21480]: audit 2024-03-04T15:57:55.368091+0000 mon.a (mon.0) 2183 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:57:56.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[21480]: audit 2024-03-04T15:57:55.369013+0000 mon.c (mon.2) 459 : audit 1 from='client.? 172.21.15.117:0/1447755774' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:57:56.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[21480]: audit 2024-03-04T15:57:55.369417+0000 mon.a (mon.0) 2184 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:57:56.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[21480]: audit 2024-03-04T15:57:55.369949+0000 mon.c (mon.2) 460 : audit 1 from='client.? 172.21.15.117:0/1447755774' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:56.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[21480]: audit 2024-03-04T15:57:55.370213+0000 mon.a (mon.0) 2185 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:56.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[28843]: audit 2024-03-04T15:57:54.321430+0000 mon.a (mon.0) 2174 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi117-63923-52"}]': finished 2024-03-04T15:57:56.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[28843]: cluster 2024-03-04T15:57:54.782706+0000 mon.a (mon.0) 2175 : cluster 0 osdmap e285: 8 total, 8 up, 8 in 2024-03-04T15:57:56.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[28843]: audit 2024-03-04T15:57:54.783886+0000 mon.a (mon.0) 2176 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi117-63923-52"}]: dispatch 2024-03-04T15:57:56.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[28843]: audit 2024-03-04T15:57:54.809390+0000 mon.c (mon.2) 456 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:56.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[28843]: audit 2024-03-04T15:57:54.809965+0000 mon.a (mon.0) 2177 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:56.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[28843]: cluster 2024-03-04T15:57:54.903258+0000 mgr.y (mgr.24428) 333 : cluster 0 pgmap v434: 228 pgs: 228 active+clean; 4.6 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-04T15:57:56.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[28843]: audit 2024-03-04T15:57:55.325516+0000 mon.a (mon.0) 2178 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi117-63915-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi117-63915-42"}]': finished 2024-03-04T15:57:56.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[28843]: audit 2024-03-04T15:57:55.325843+0000 mon.a (mon.0) 2179 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi117-63923-52"}]': finished 2024-03-04T15:57:56.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[28843]: audit 2024-03-04T15:57:55.326027+0000 mon.a (mon.0) 2180 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:56.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[28843]: cluster 2024-03-04T15:57:55.334527+0000 mon.a (mon.0) 2181 : cluster 0 osdmap e286: 8 total, 8 up, 8 in 2024-03-04T15:57:56.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[28843]: audit 2024-03-04T15:57:55.336552+0000 mon.c (mon.2) 457 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-35"}]: dispatch 2024-03-04T15:57:56.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[28843]: audit 2024-03-04T15:57:55.338467+0000 mon.a (mon.0) 2182 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-35"}]: dispatch 2024-03-04T15:57:56.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[28843]: audit 2024-03-04T15:57:55.367756+0000 mon.c (mon.2) 458 : audit 1 from='client.? 172.21.15.117:0/1447755774' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:57:56.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[28843]: audit 2024-03-04T15:57:55.368091+0000 mon.a (mon.0) 2183 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:57:56.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[28843]: audit 2024-03-04T15:57:55.369013+0000 mon.c (mon.2) 459 : audit 1 from='client.? 172.21.15.117:0/1447755774' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:57:56.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[28843]: audit 2024-03-04T15:57:55.369417+0000 mon.a (mon.0) 2184 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:57:56.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[28843]: audit 2024-03-04T15:57:55.369949+0000 mon.c (mon.2) 460 : audit 1 from='client.? 172.21.15.117:0/1447755774' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:56.071 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:55 smithi117 bash[28843]: audit 2024-03-04T15:57:55.370213+0000 mon.a (mon.0) 2185 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:55 smithi151 bash[23756]: audit 2024-03-04T15:57:54.321430+0000 mon.a (mon.0) 2174 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi117-63923-52"}]': finished 2024-03-04T15:57:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:55 smithi151 bash[23756]: cluster 2024-03-04T15:57:54.782706+0000 mon.a (mon.0) 2175 : cluster 0 osdmap e285: 8 total, 8 up, 8 in 2024-03-04T15:57:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:55 smithi151 bash[23756]: audit 2024-03-04T15:57:54.783886+0000 mon.a (mon.0) 2176 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi117-63923-52"}]: dispatch 2024-03-04T15:57:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:55 smithi151 bash[23756]: audit 2024-03-04T15:57:54.809390+0000 mon.c (mon.2) 456 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:56.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:55 smithi151 bash[23756]: audit 2024-03-04T15:57:54.809965+0000 mon.a (mon.0) 2177 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:57:56.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:55 smithi151 bash[23756]: cluster 2024-03-04T15:57:54.903258+0000 mgr.y (mgr.24428) 333 : cluster 0 pgmap v434: 228 pgs: 228 active+clean; 4.6 MiB data, 629 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 1 op/s 2024-03-04T15:57:56.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:55 smithi151 bash[23756]: audit 2024-03-04T15:57:55.325516+0000 mon.a (mon.0) 2178 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi117-63915-42", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi117-63915-42"}]': finished 2024-03-04T15:57:56.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:55 smithi151 bash[23756]: audit 2024-03-04T15:57:55.325843+0000 mon.a (mon.0) 2179 : audit 1 from='client.? 172.21.15.117:0/4150160335' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi117-63923-52"}]': finished 2024-03-04T15:57:56.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:55 smithi151 bash[23756]: audit 2024-03-04T15:57:55.326027+0000 mon.a (mon.0) 2180 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:57:56.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:55 smithi151 bash[23756]: cluster 2024-03-04T15:57:55.334527+0000 mon.a (mon.0) 2181 : cluster 0 osdmap e286: 8 total, 8 up, 8 in 2024-03-04T15:57:56.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:55 smithi151 bash[23756]: audit 2024-03-04T15:57:55.336552+0000 mon.c (mon.2) 457 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-35"}]: dispatch 2024-03-04T15:57:56.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:55 smithi151 bash[23756]: audit 2024-03-04T15:57:55.338467+0000 mon.a (mon.0) 2182 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-35"}]: dispatch 2024-03-04T15:57:56.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:55 smithi151 bash[23756]: audit 2024-03-04T15:57:55.367756+0000 mon.c (mon.2) 458 : audit 1 from='client.? 172.21.15.117:0/1447755774' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:57:56.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:55 smithi151 bash[23756]: audit 2024-03-04T15:57:55.368091+0000 mon.a (mon.0) 2183 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:57:56.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:55 smithi151 bash[23756]: audit 2024-03-04T15:57:55.369013+0000 mon.c (mon.2) 459 : audit 1 from='client.? 172.21.15.117:0/1447755774' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:57:56.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:55 smithi151 bash[23756]: audit 2024-03-04T15:57:55.369417+0000 mon.a (mon.0) 2184 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:57:56.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:55 smithi151 bash[23756]: audit 2024-03-04T15:57:55.369949+0000 mon.c (mon.2) 460 : audit 1 from='client.? 172.21.15.117:0/1447755774' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:56.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:55 smithi151 bash[23756]: audit 2024-03-04T15:57:55.370213+0000 mon.a (mon.0) 2185 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:57:56.567 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:57:56 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:57:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:57:56.932 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:57:56 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:57:56.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:56 smithi151 bash[23756]: cluster 2024-03-04T15:57:56.325724+0000 mon.a (mon.0) 2186 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:57:56.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:56 smithi151 bash[23756]: audit 2024-03-04T15:57:56.328158+0000 mon.a (mon.0) 2187 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-35"}]': finished 2024-03-04T15:57:56.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:56 smithi151 bash[23756]: audit 2024-03-04T15:57:56.328454+0000 mon.a (mon.0) 2188 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:56.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:56 smithi151 bash[23756]: cluster 2024-03-04T15:57:56.336728+0000 mon.a (mon.0) 2189 : cluster 0 osdmap e287: 8 total, 8 up, 8 in 2024-03-04T15:57:56.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:56 smithi151 bash[23756]: audit 2024-03-04T15:57:56.338615+0000 mon.c (mon.2) 461 : audit 1 from='client.? 172.21.15.117:0/1447755774' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi117-63923-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:57:56.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:56 smithi151 bash[23756]: audit 2024-03-04T15:57:56.345165+0000 mon.a (mon.0) 2190 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi117-63923-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:57:57.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:56 smithi117 bash[28843]: cluster 2024-03-04T15:57:56.325724+0000 mon.a (mon.0) 2186 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:57:57.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:56 smithi117 bash[28843]: audit 2024-03-04T15:57:56.328158+0000 mon.a (mon.0) 2187 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-35"}]': finished 2024-03-04T15:57:57.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:56 smithi117 bash[28843]: audit 2024-03-04T15:57:56.328454+0000 mon.a (mon.0) 2188 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:57.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:56 smithi117 bash[28843]: cluster 2024-03-04T15:57:56.336728+0000 mon.a (mon.0) 2189 : cluster 0 osdmap e287: 8 total, 8 up, 8 in 2024-03-04T15:57:57.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:56 smithi117 bash[28843]: audit 2024-03-04T15:57:56.338615+0000 mon.c (mon.2) 461 : audit 1 from='client.? 172.21.15.117:0/1447755774' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi117-63923-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:57:57.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:56 smithi117 bash[28843]: audit 2024-03-04T15:57:56.345165+0000 mon.a (mon.0) 2190 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi117-63923-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:57:57.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:56 smithi117 bash[21480]: cluster 2024-03-04T15:57:56.325724+0000 mon.a (mon.0) 2186 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:57:57.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:56 smithi117 bash[21480]: audit 2024-03-04T15:57:56.328158+0000 mon.a (mon.0) 2187 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-35"}]': finished 2024-03-04T15:57:57.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:56 smithi117 bash[21480]: audit 2024-03-04T15:57:56.328454+0000 mon.a (mon.0) 2188 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:57:57.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:56 smithi117 bash[21480]: cluster 2024-03-04T15:57:56.336728+0000 mon.a (mon.0) 2189 : cluster 0 osdmap e287: 8 total, 8 up, 8 in 2024-03-04T15:57:57.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:56 smithi117 bash[21480]: audit 2024-03-04T15:57:56.338615+0000 mon.c (mon.2) 461 : audit 1 from='client.? 172.21.15.117:0/1447755774' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi117-63923-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:57:57.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:56 smithi117 bash[21480]: audit 2024-03-04T15:57:56.345165+0000 mon.a (mon.0) 2190 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi117-63923-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:57:58.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:58 smithi117 bash[21480]: audit 2024-03-04T15:57:56.668671+0000 mgr.y (mgr.24428) 334 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:57:58.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:58 smithi117 bash[21480]: cluster 2024-03-04T15:57:56.905580+0000 mgr.y (mgr.24428) 335 : cluster 0 pgmap v437: 236 pgs: 236 active+clean; 4.6 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-04T15:57:58.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:58 smithi117 bash[21480]: cluster 2024-03-04T15:57:57.790412+0000 mon.a (mon.0) 2191 : cluster 0 osdmap e288: 8 total, 8 up, 8 in 2024-03-04T15:57:58.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:58 smithi117 bash[28843]: audit 2024-03-04T15:57:56.668671+0000 mgr.y (mgr.24428) 334 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:57:58.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:58 smithi117 bash[28843]: cluster 2024-03-04T15:57:56.905580+0000 mgr.y (mgr.24428) 335 : cluster 0 pgmap v437: 236 pgs: 236 active+clean; 4.6 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-04T15:57:58.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:58 smithi117 bash[28843]: cluster 2024-03-04T15:57:57.790412+0000 mon.a (mon.0) 2191 : cluster 0 osdmap e288: 8 total, 8 up, 8 in 2024-03-04T15:57:58.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:58 smithi151 bash[23756]: audit 2024-03-04T15:57:56.668671+0000 mgr.y (mgr.24428) 334 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:57:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:58 smithi151 bash[23756]: cluster 2024-03-04T15:57:56.905580+0000 mgr.y (mgr.24428) 335 : cluster 0 pgmap v437: 236 pgs: 236 active+clean; 4.6 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1023 B/s wr, 3 op/s 2024-03-04T15:57:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:58 smithi151 bash[23756]: cluster 2024-03-04T15:57:57.790412+0000 mon.a (mon.0) 2191 : cluster 0 osdmap e288: 8 total, 8 up, 8 in 2024-03-04T15:57:59.368 INFO:tasks.workunit.client.0.smithi117.stdout:e 2024-03-04T15:57:59.368 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (6964 ms) 2024-03-04T15:57:59.368 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForComplete 2024-03-04T15:57:59.368 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (7013 ms) 2024-03-04T15:57:59.368 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-03-04T15:57:59.368 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (6977 ms) 2024-03-04T15:57:59.368 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-03-04T15:57:59.368 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (7385 ms) 2024-03-04T15:57:59.368 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-03-04T15:57:59.369 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7028 ms) 2024-03-04T15:57:59.369 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-03-04T15:57:59.369 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7041 ms) 2024-03-04T15:57:59.369 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-03-04T15:57:59.369 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7089 ms) 2024-03-04T15:57:59.369 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-03-04T15:57:59.369 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (6837 ms) 2024-03-04T15:57:59.369 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-03-04T15:57:59.369 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (7041 ms) 2024-03-04T15:57:59.369 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-03-04T15:57:59.369 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7026 ms) 2024-03-04T15:57:59.369 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-03-04T15:57:59.369 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (6947 ms) 2024-03-04T15:57:59.369 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-03-04T15:57:59.369 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (6966 ms) 2024-03-04T15:57:59.370 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-03-04T15:57:59.370 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (7072 ms) 2024-03-04T15:57:59.370 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-03-04T15:57:59.370 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (7071 ms) 2024-03-04T15:57:59.370 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-03-04T15:57:59.370 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7056 ms) 2024-03-04T15:57:59.370 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-03-04T15:57:59.370 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7040 ms) 2024-03-04T15:57:59.370 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (112554 ms total) 2024-03-04T15:57:59.370 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: 2024-03-04T15:57:59.370 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [----------] Global test environment tear-down 2024-03-04T15:57:59.370 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [==========] 42 tests from 2 test suites ran. (239476 ms total) 2024-03-04T15:57:59.370 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio: [ PASSED ] 42 tests. 2024-03-04T15:57:59.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[21480]: audit 2024-03-04T15:57:57.791983+0000 mon.c (mon.2) 462 : audit 1 from='client.? 172.21.15.117:0/986178260' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:59.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[21480]: cluster 2024-03-04T15:57:58.231604+0000 mon.a (mon.0) 2192 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:59.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[21480]: audit 2024-03-04T15:57:58.232930+0000 mon.a (mon.0) 2193 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[21480]: audit 2024-03-04T15:57:58.335032+0000 mon.a (mon.0) 2194 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi117-63923-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi117-63923-53"}]': finished 2024-03-04T15:57:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[21480]: audit 2024-03-04T15:57:58.335243+0000 mon.a (mon.0) 2195 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi117-63915-42"}]': finished 2024-03-04T15:57:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[21480]: cluster 2024-03-04T15:57:58.342547+0000 mon.a (mon.0) 2196 : cluster 0 osdmap e289: 8 total, 8 up, 8 in 2024-03-04T15:57:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[21480]: audit 2024-03-04T15:57:58.344549+0000 mon.c (mon.2) 463 : audit 1 from='client.? 172.21.15.117:0/986178260' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[21480]: audit 2024-03-04T15:57:58.351171+0000 mon.a (mon.0) 2197 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[21480]: audit 2024-03-04T15:57:58.351190+0000 mon.c (mon.2) 464 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[21480]: audit 2024-03-04T15:57:58.355582+0000 mon.a (mon.0) 2198 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[21480]: audit 2024-03-04T15:57:58.726902+0000 mon.c (mon.2) 465 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:57:59.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[28843]: audit 2024-03-04T15:57:57.791983+0000 mon.c (mon.2) 462 : audit 1 from='client.? 172.21.15.117:0/986178260' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:59.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[28843]: cluster 2024-03-04T15:57:58.231604+0000 mon.a (mon.0) 2192 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:59.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[28843]: audit 2024-03-04T15:57:58.232930+0000 mon.a (mon.0) 2193 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:59.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[28843]: audit 2024-03-04T15:57:58.335032+0000 mon.a (mon.0) 2194 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi117-63923-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi117-63923-53"}]': finished 2024-03-04T15:57:59.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[28843]: audit 2024-03-04T15:57:58.335243+0000 mon.a (mon.0) 2195 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi117-63915-42"}]': finished 2024-03-04T15:57:59.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[28843]: cluster 2024-03-04T15:57:58.342547+0000 mon.a (mon.0) 2196 : cluster 0 osdmap e289: 8 total, 8 up, 8 in 2024-03-04T15:57:59.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[28843]: audit 2024-03-04T15:57:58.344549+0000 mon.c (mon.2) 463 : audit 1 from='client.? 172.21.15.117:0/986178260' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:59.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[28843]: audit 2024-03-04T15:57:58.351171+0000 mon.a (mon.0) 2197 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:59.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[28843]: audit 2024-03-04T15:57:58.351190+0000 mon.c (mon.2) 464 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:59.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[28843]: audit 2024-03-04T15:57:58.355582+0000 mon.a (mon.0) 2198 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:59.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:57:59 smithi117 bash[28843]: audit 2024-03-04T15:57:58.726902+0000 mon.c (mon.2) 465 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:57:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:59 smithi151 bash[23756]: audit 2024-03-04T15:57:57.791983+0000 mon.c (mon.2) 462 : audit 1 from='client.? 172.21.15.117:0/986178260' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:59 smithi151 bash[23756]: cluster 2024-03-04T15:57:58.231604+0000 mon.a (mon.0) 2192 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:57:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:59 smithi151 bash[23756]: audit 2024-03-04T15:57:58.232930+0000 mon.a (mon.0) 2193 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:59 smithi151 bash[23756]: audit 2024-03-04T15:57:58.335032+0000 mon.a (mon.0) 2194 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi117-63923-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi117-63923-53"}]': finished 2024-03-04T15:57:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:59 smithi151 bash[23756]: audit 2024-03-04T15:57:58.335243+0000 mon.a (mon.0) 2195 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi117-63915-42"}]': finished 2024-03-04T15:57:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:59 smithi151 bash[23756]: cluster 2024-03-04T15:57:58.342547+0000 mon.a (mon.0) 2196 : cluster 0 osdmap e289: 8 total, 8 up, 8 in 2024-03-04T15:57:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:59 smithi151 bash[23756]: audit 2024-03-04T15:57:58.344549+0000 mon.c (mon.2) 463 : audit 1 from='client.? 172.21.15.117:0/986178260' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:59 smithi151 bash[23756]: audit 2024-03-04T15:57:58.351171+0000 mon.a (mon.0) 2197 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi117-63915-42"}]: dispatch 2024-03-04T15:57:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:59 smithi151 bash[23756]: audit 2024-03-04T15:57:58.351190+0000 mon.c (mon.2) 464 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:59 smithi151 bash[23756]: audit 2024-03-04T15:57:58.355582+0000 mon.a (mon.0) 2198 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:57:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:57:59 smithi151 bash[23756]: audit 2024-03-04T15:57:58.726902+0000 mon.c (mon.2) 465 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:58:00.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:00 smithi117 bash[21480]: cluster 2024-03-04T15:57:58.906370+0000 mgr.y (mgr.24428) 336 : cluster 0 pgmap v440: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-03-04T15:58:00.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:00 smithi117 bash[21480]: audit 2024-03-04T15:57:59.346917+0000 mon.a (mon.0) 2199 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi117-63915-42"}]': finished 2024-03-04T15:58:00.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:00 smithi117 bash[21480]: audit 2024-03-04T15:57:59.347066+0000 mon.a (mon.0) 2200 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:00.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:00 smithi117 bash[21480]: cluster 2024-03-04T15:57:59.355496+0000 mon.a (mon.0) 2201 : cluster 0 osdmap e290: 8 total, 8 up, 8 in 2024-03-04T15:58:00.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:00 smithi117 bash[21480]: audit 2024-03-04T15:57:59.358698+0000 mon.c (mon.2) 466 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:00.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:00 smithi117 bash[21480]: audit 2024-03-04T15:57:59.359565+0000 mon.a (mon.0) 2202 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:00.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:00 smithi117 bash[28843]: cluster 2024-03-04T15:57:58.906370+0000 mgr.y (mgr.24428) 336 : cluster 0 pgmap v440: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-03-04T15:58:00.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:00 smithi117 bash[28843]: audit 2024-03-04T15:57:59.346917+0000 mon.a (mon.0) 2199 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi117-63915-42"}]': finished 2024-03-04T15:58:00.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:00 smithi117 bash[28843]: audit 2024-03-04T15:57:59.347066+0000 mon.a (mon.0) 2200 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:00.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:00 smithi117 bash[28843]: cluster 2024-03-04T15:57:59.355496+0000 mon.a (mon.0) 2201 : cluster 0 osdmap e290: 8 total, 8 up, 8 in 2024-03-04T15:58:00.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:00 smithi117 bash[28843]: audit 2024-03-04T15:57:59.358698+0000 mon.c (mon.2) 466 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:00.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:00 smithi117 bash[28843]: audit 2024-03-04T15:57:59.359565+0000 mon.a (mon.0) 2202 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:00 smithi151 bash[23756]: cluster 2024-03-04T15:57:58.906370+0000 mgr.y (mgr.24428) 336 : cluster 0 pgmap v440: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-03-04T15:58:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:00 smithi151 bash[23756]: audit 2024-03-04T15:57:59.346917+0000 mon.a (mon.0) 2199 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi117-63915-42"}]': finished 2024-03-04T15:58:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:00 smithi151 bash[23756]: audit 2024-03-04T15:57:59.347066+0000 mon.a (mon.0) 2200 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:00 smithi151 bash[23756]: cluster 2024-03-04T15:57:59.355496+0000 mon.a (mon.0) 2201 : cluster 0 osdmap e290: 8 total, 8 up, 8 in 2024-03-04T15:58:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:00 smithi151 bash[23756]: audit 2024-03-04T15:57:59.358698+0000 mon.c (mon.2) 466 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:00 smithi151 bash[23756]: audit 2024-03-04T15:57:59.359565+0000 mon.a (mon.0) 2202 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:01 smithi151 bash[23756]: audit 2024-03-04T15:58:00.349841+0000 mon.a (mon.0) 2203 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-37", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:58:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:01 smithi151 bash[23756]: cluster 2024-03-04T15:58:00.357894+0000 mon.a (mon.0) 2204 : cluster 0 osdmap e291: 8 total, 8 up, 8 in 2024-03-04T15:58:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:01 smithi151 bash[23756]: audit 2024-03-04T15:58:00.359078+0000 mon.c (mon.2) 467 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-37"}]: dispatch 2024-03-04T15:58:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:01 smithi151 bash[23756]: audit 2024-03-04T15:58:00.361454+0000 mon.c (mon.2) 468 : audit 1 from='client.? 172.21.15.117:0/1447755774' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:58:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:01 smithi151 bash[23756]: audit 2024-03-04T15:58:00.362880+0000 mon.a (mon.0) 2205 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-37"}]: dispatch 2024-03-04T15:58:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:01 smithi151 bash[23756]: audit 2024-03-04T15:58:00.365368+0000 mon.a (mon.0) 2206 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:58:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:01 smithi151 bash[23756]: cluster 2024-03-04T15:58:00.907197+0000 mgr.y (mgr.24428) 337 : cluster 0 pgmap v443: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:01.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:01 smithi117 bash[21480]: audit 2024-03-04T15:58:00.349841+0000 mon.a (mon.0) 2203 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-37", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:58:01.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:01 smithi117 bash[21480]: cluster 2024-03-04T15:58:00.357894+0000 mon.a (mon.0) 2204 : cluster 0 osdmap e291: 8 total, 8 up, 8 in 2024-03-04T15:58:01.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:01 smithi117 bash[21480]: audit 2024-03-04T15:58:00.359078+0000 mon.c (mon.2) 467 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-37"}]: dispatch 2024-03-04T15:58:01.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:01 smithi117 bash[21480]: audit 2024-03-04T15:58:00.361454+0000 mon.c (mon.2) 468 : audit 1 from='client.? 172.21.15.117:0/1447755774' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:58:01.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:01 smithi117 bash[21480]: audit 2024-03-04T15:58:00.362880+0000 mon.a (mon.0) 2205 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-37"}]: dispatch 2024-03-04T15:58:01.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:01 smithi117 bash[21480]: audit 2024-03-04T15:58:00.365368+0000 mon.a (mon.0) 2206 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:58:01.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:01 smithi117 bash[21480]: cluster 2024-03-04T15:58:00.907197+0000 mgr.y (mgr.24428) 337 : cluster 0 pgmap v443: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:01.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:01 smithi117 bash[28843]: audit 2024-03-04T15:58:00.349841+0000 mon.a (mon.0) 2203 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-37", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:58:01.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:01 smithi117 bash[28843]: cluster 2024-03-04T15:58:00.357894+0000 mon.a (mon.0) 2204 : cluster 0 osdmap e291: 8 total, 8 up, 8 in 2024-03-04T15:58:01.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:01 smithi117 bash[28843]: audit 2024-03-04T15:58:00.359078+0000 mon.c (mon.2) 467 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-37"}]: dispatch 2024-03-04T15:58:01.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:01 smithi117 bash[28843]: audit 2024-03-04T15:58:00.361454+0000 mon.c (mon.2) 468 : audit 1 from='client.? 172.21.15.117:0/1447755774' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:58:01.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:01 smithi117 bash[28843]: audit 2024-03-04T15:58:00.362880+0000 mon.a (mon.0) 2205 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-37"}]: dispatch 2024-03-04T15:58:01.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:01 smithi117 bash[28843]: audit 2024-03-04T15:58:00.365368+0000 mon.a (mon.0) 2206 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:58:01.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:01 smithi117 bash[28843]: cluster 2024-03-04T15:58:00.907197+0000 mgr.y (mgr.24428) 337 : cluster 0 pgmap v443: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 634 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:02 smithi151 bash[23756]: audit 2024-03-04T15:58:01.361365+0000 mon.a (mon.0) 2207 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-37"}]': finished 2024-03-04T15:58:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:02 smithi151 bash[23756]: audit 2024-03-04T15:58:01.361606+0000 mon.a (mon.0) 2208 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53"}]': finished 2024-03-04T15:58:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:02 smithi151 bash[23756]: cluster 2024-03-04T15:58:01.370895+0000 mon.a (mon.0) 2209 : cluster 0 osdmap e292: 8 total, 8 up, 8 in 2024-03-04T15:58:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:02 smithi151 bash[23756]: audit 2024-03-04T15:58:01.371561+0000 mon.c (mon.2) 469 : audit 1 from='client.? 172.21.15.117:0/1447755774' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:58:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:02 smithi151 bash[23756]: audit 2024-03-04T15:58:01.373702+0000 mon.c (mon.2) 470 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-37", "mode": "writeback"}]: dispatch 2024-03-04T15:58:02.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:02 smithi151 bash[23756]: audit 2024-03-04T15:58:01.375587+0000 mon.a (mon.0) 2210 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:58:02.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:02 smithi151 bash[23756]: audit 2024-03-04T15:58:01.376382+0000 mon.a (mon.0) 2211 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-37", "mode": "writeback"}]: dispatch 2024-03-04T15:58:02.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:02 smithi117 bash[21480]: audit 2024-03-04T15:58:01.361365+0000 mon.a (mon.0) 2207 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-37"}]': finished 2024-03-04T15:58:02.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:02 smithi117 bash[21480]: audit 2024-03-04T15:58:01.361606+0000 mon.a (mon.0) 2208 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53"}]': finished 2024-03-04T15:58:02.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:02 smithi117 bash[21480]: cluster 2024-03-04T15:58:01.370895+0000 mon.a (mon.0) 2209 : cluster 0 osdmap e292: 8 total, 8 up, 8 in 2024-03-04T15:58:02.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:02 smithi117 bash[21480]: audit 2024-03-04T15:58:01.371561+0000 mon.c (mon.2) 469 : audit 1 from='client.? 172.21.15.117:0/1447755774' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:58:02.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:02 smithi117 bash[21480]: audit 2024-03-04T15:58:01.373702+0000 mon.c (mon.2) 470 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-37", "mode": "writeback"}]: dispatch 2024-03-04T15:58:02.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:02 smithi117 bash[21480]: audit 2024-03-04T15:58:01.375587+0000 mon.a (mon.0) 2210 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:58:02.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:02 smithi117 bash[21480]: audit 2024-03-04T15:58:01.376382+0000 mon.a (mon.0) 2211 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-37", "mode": "writeback"}]: dispatch 2024-03-04T15:58:02.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:02 smithi117 bash[28843]: audit 2024-03-04T15:58:01.361365+0000 mon.a (mon.0) 2207 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-37"}]': finished 2024-03-04T15:58:02.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:02 smithi117 bash[28843]: audit 2024-03-04T15:58:01.361606+0000 mon.a (mon.0) 2208 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi117-63923-53"}]': finished 2024-03-04T15:58:02.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:02 smithi117 bash[28843]: cluster 2024-03-04T15:58:01.370895+0000 mon.a (mon.0) 2209 : cluster 0 osdmap e292: 8 total, 8 up, 8 in 2024-03-04T15:58:02.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:02 smithi117 bash[28843]: audit 2024-03-04T15:58:01.371561+0000 mon.c (mon.2) 469 : audit 1 from='client.? 172.21.15.117:0/1447755774' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:58:02.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:02 smithi117 bash[28843]: audit 2024-03-04T15:58:01.373702+0000 mon.c (mon.2) 470 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-37", "mode": "writeback"}]: dispatch 2024-03-04T15:58:02.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:02 smithi117 bash[28843]: audit 2024-03-04T15:58:01.375587+0000 mon.a (mon.0) 2210 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi117-63923-53"}]: dispatch 2024-03-04T15:58:02.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:02 smithi117 bash[28843]: audit 2024-03-04T15:58:01.376382+0000 mon.a (mon.0) 2211 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-37", "mode": "writeback"}]: dispatch 2024-03-04T15:58:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: cluster 2024-03-04T15:58:02.362298+0000 mon.a (mon.0) 2212 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:58:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: audit 2024-03-04T15:58:02.364812+0000 mon.a (mon.0) 2213 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi117-63923-53"}]': finished 2024-03-04T15:58:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: audit 2024-03-04T15:58:02.365035+0000 mon.a (mon.0) 2214 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-37", "mode": "writeback"}]': finished 2024-03-04T15:58:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: cluster 2024-03-04T15:58:02.371936+0000 mon.a (mon.0) 2215 : cluster 0 osdmap e293: 8 total, 8 up, 8 in 2024-03-04T15:58:03.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: audit 2024-03-04T15:58:02.416424+0000 mon.c (mon.2) 471 : audit 1 from='client.? 172.21.15.117:0/4278747072' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:03.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: audit 2024-03-04T15:58:02.416814+0000 mon.a (mon.0) 2216 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:03.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: audit 2024-03-04T15:58:02.417769+0000 mon.c (mon.2) 472 : audit 1 from='client.? 172.21.15.117:0/4278747072' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:03.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: audit 2024-03-04T15:58:02.418147+0000 mon.a (mon.0) 2217 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:03.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: audit 2024-03-04T15:58:02.418763+0000 mon.c (mon.2) 473 : audit 1 from='client.? 172.21.15.117:0/4278747072' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi117-63923-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:03.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: audit 2024-03-04T15:58:02.419112+0000 mon.a (mon.0) 2218 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi117-63923-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:03.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: audit 2024-03-04T15:58:02.453519+0000 mon.c (mon.2) 474 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:03.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: audit 2024-03-04T15:58:02.453941+0000 mon.a (mon.0) 2219 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:03.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: cluster 2024-03-04T15:58:02.908029+0000 mgr.y (mgr.24428) 338 : cluster 0 pgmap v446: 228 pgs: 228 active+clean; 4.6 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:03.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: cluster 2024-03-04T15:58:03.232784+0000 mon.a (mon.0) 2220 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:03.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: audit 2024-03-04T15:58:03.240318+0000 mon.a (mon.0) 2221 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi117-63923-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:03.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: audit 2024-03-04T15:58:03.240471+0000 mon.a (mon.0) 2222 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:58:03.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: cluster 2024-03-04T15:58:03.248044+0000 mon.a (mon.0) 2223 : cluster 0 osdmap e294: 8 total, 8 up, 8 in 2024-03-04T15:58:03.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: audit 2024-03-04T15:58:03.253348+0000 mon.c (mon.2) 475 : audit 1 from='client.? 172.21.15.117:0/4278747072' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi117-63923-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:03.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: audit 2024-03-04T15:58:03.253763+0000 mon.c (mon.2) 476 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-37"}]: dispatch 2024-03-04T15:58:03.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: audit 2024-03-04T15:58:03.264546+0000 mon.a (mon.0) 2224 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi117-63923-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:03.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:03 smithi151 bash[23756]: audit 2024-03-04T15:58:03.265405+0000 mon.a (mon.0) 2225 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-37"}]: dispatch 2024-03-04T15:58:03.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: cluster 2024-03-04T15:58:02.362298+0000 mon.a (mon.0) 2212 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:58:03.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: audit 2024-03-04T15:58:02.364812+0000 mon.a (mon.0) 2213 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi117-63923-53"}]': finished 2024-03-04T15:58:03.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: audit 2024-03-04T15:58:02.365035+0000 mon.a (mon.0) 2214 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-37", "mode": "writeback"}]': finished 2024-03-04T15:58:03.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: cluster 2024-03-04T15:58:02.371936+0000 mon.a (mon.0) 2215 : cluster 0 osdmap e293: 8 total, 8 up, 8 in 2024-03-04T15:58:03.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: audit 2024-03-04T15:58:02.416424+0000 mon.c (mon.2) 471 : audit 1 from='client.? 172.21.15.117:0/4278747072' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:03.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: audit 2024-03-04T15:58:02.416814+0000 mon.a (mon.0) 2216 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:03.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: audit 2024-03-04T15:58:02.417769+0000 mon.c (mon.2) 472 : audit 1 from='client.? 172.21.15.117:0/4278747072' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:03.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: audit 2024-03-04T15:58:02.418147+0000 mon.a (mon.0) 2217 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:03.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: audit 2024-03-04T15:58:02.418763+0000 mon.c (mon.2) 473 : audit 1 from='client.? 172.21.15.117:0/4278747072' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi117-63923-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:03.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: audit 2024-03-04T15:58:02.419112+0000 mon.a (mon.0) 2218 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi117-63923-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:03.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: audit 2024-03-04T15:58:02.453519+0000 mon.c (mon.2) 474 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:03.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: audit 2024-03-04T15:58:02.453941+0000 mon.a (mon.0) 2219 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:03.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: cluster 2024-03-04T15:58:02.908029+0000 mgr.y (mgr.24428) 338 : cluster 0 pgmap v446: 228 pgs: 228 active+clean; 4.6 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:03.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: cluster 2024-03-04T15:58:03.232784+0000 mon.a (mon.0) 2220 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:03.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: audit 2024-03-04T15:58:03.240318+0000 mon.a (mon.0) 2221 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi117-63923-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:03.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: audit 2024-03-04T15:58:03.240471+0000 mon.a (mon.0) 2222 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:58:03.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: cluster 2024-03-04T15:58:02.362298+0000 mon.a (mon.0) 2212 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:58:03.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: audit 2024-03-04T15:58:02.364812+0000 mon.a (mon.0) 2213 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi117-63923-53"}]': finished 2024-03-04T15:58:03.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: audit 2024-03-04T15:58:02.365035+0000 mon.a (mon.0) 2214 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-37", "mode": "writeback"}]': finished 2024-03-04T15:58:03.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: cluster 2024-03-04T15:58:02.371936+0000 mon.a (mon.0) 2215 : cluster 0 osdmap e293: 8 total, 8 up, 8 in 2024-03-04T15:58:03.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: audit 2024-03-04T15:58:02.416424+0000 mon.c (mon.2) 471 : audit 1 from='client.? 172.21.15.117:0/4278747072' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:03.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: audit 2024-03-04T15:58:02.416814+0000 mon.a (mon.0) 2216 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:03.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: audit 2024-03-04T15:58:02.417769+0000 mon.c (mon.2) 472 : audit 1 from='client.? 172.21.15.117:0/4278747072' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:03.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: audit 2024-03-04T15:58:02.418147+0000 mon.a (mon.0) 2217 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:03.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: audit 2024-03-04T15:58:02.418763+0000 mon.c (mon.2) 473 : audit 1 from='client.? 172.21.15.117:0/4278747072' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi117-63923-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:03.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: audit 2024-03-04T15:58:02.419112+0000 mon.a (mon.0) 2218 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi117-63923-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:03.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: audit 2024-03-04T15:58:02.453519+0000 mon.c (mon.2) 474 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:03.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: audit 2024-03-04T15:58:02.453941+0000 mon.a (mon.0) 2219 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:03.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: cluster 2024-03-04T15:58:02.908029+0000 mgr.y (mgr.24428) 338 : cluster 0 pgmap v446: 228 pgs: 228 active+clean; 4.6 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:03.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: cluster 2024-03-04T15:58:03.232784+0000 mon.a (mon.0) 2220 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:03.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: audit 2024-03-04T15:58:03.240318+0000 mon.a (mon.0) 2221 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi117-63923-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:03.822 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: audit 2024-03-04T15:58:03.240471+0000 mon.a (mon.0) 2222 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:58:03.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: cluster 2024-03-04T15:58:03.248044+0000 mon.a (mon.0) 2223 : cluster 0 osdmap e294: 8 total, 8 up, 8 in 2024-03-04T15:58:03.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: audit 2024-03-04T15:58:03.253348+0000 mon.c (mon.2) 475 : audit 1 from='client.? 172.21.15.117:0/4278747072' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi117-63923-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:03.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: audit 2024-03-04T15:58:03.253763+0000 mon.c (mon.2) 476 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-37"}]: dispatch 2024-03-04T15:58:03.822 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: audit 2024-03-04T15:58:03.264546+0000 mon.a (mon.0) 2224 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi117-63923-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:03.823 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[21480]: audit 2024-03-04T15:58:03.265405+0000 mon.a (mon.0) 2225 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-37"}]: dispatch 2024-03-04T15:58:03.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: cluster 2024-03-04T15:58:03.248044+0000 mon.a (mon.0) 2223 : cluster 0 osdmap e294: 8 total, 8 up, 8 in 2024-03-04T15:58:03.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: audit 2024-03-04T15:58:03.253348+0000 mon.c (mon.2) 475 : audit 1 from='client.? 172.21.15.117:0/4278747072' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi117-63923-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:03.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: audit 2024-03-04T15:58:03.253763+0000 mon.c (mon.2) 476 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-37"}]: dispatch 2024-03-04T15:58:03.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: audit 2024-03-04T15:58:03.264546+0000 mon.a (mon.0) 2224 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi117-63923-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:03.823 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:03 smithi117 bash[28843]: audit 2024-03-04T15:58:03.265405+0000 mon.a (mon.0) 2225 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-37"}]: dispatch 2024-03-04T15:58:05.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:04 smithi117 bash[21480]: cluster 2024-03-04T15:58:04.257523+0000 mon.a (mon.0) 2226 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:58:05.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:04 smithi117 bash[21480]: audit 2024-03-04T15:58:04.259965+0000 mon.a (mon.0) 2227 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-37"}]': finished 2024-03-04T15:58:05.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:04 smithi117 bash[28843]: cluster 2024-03-04T15:58:04.257523+0000 mon.a (mon.0) 2226 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:58:05.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:04 smithi117 bash[28843]: audit 2024-03-04T15:58:04.259965+0000 mon.a (mon.0) 2227 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-37"}]': finished 2024-03-04T15:58:05.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:04 smithi151 bash[23756]: cluster 2024-03-04T15:58:04.257523+0000 mon.a (mon.0) 2226 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:58:05.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:04 smithi151 bash[23756]: audit 2024-03-04T15:58:04.259965+0000 mon.a (mon.0) 2227 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-37"}]': finished 2024-03-04T15:58:06.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:05 smithi117 bash[21480]: cluster 2024-03-04T15:58:04.723228+0000 mon.a (mon.0) 2228 : cluster 0 osdmap e295: 8 total, 8 up, 8 in 2024-03-04T15:58:06.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:05 smithi117 bash[21480]: cluster 2024-03-04T15:58:04.908880+0000 mgr.y (mgr.24428) 339 : cluster 0 pgmap v449: 228 pgs: 228 active+clean; 4.6 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:06.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:05 smithi117 bash[21480]: audit 2024-03-04T15:58:05.263162+0000 mon.a (mon.0) 2229 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi117-63923-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi117-63923-54"}]': finished 2024-03-04T15:58:06.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:05 smithi117 bash[21480]: cluster 2024-03-04T15:58:05.270553+0000 mon.a (mon.0) 2230 : cluster 0 osdmap e296: 8 total, 8 up, 8 in 2024-03-04T15:58:06.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:05 smithi117 bash[28843]: cluster 2024-03-04T15:58:04.723228+0000 mon.a (mon.0) 2228 : cluster 0 osdmap e295: 8 total, 8 up, 8 in 2024-03-04T15:58:06.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:05 smithi117 bash[28843]: cluster 2024-03-04T15:58:04.908880+0000 mgr.y (mgr.24428) 339 : cluster 0 pgmap v449: 228 pgs: 228 active+clean; 4.6 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:06.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:05 smithi117 bash[28843]: audit 2024-03-04T15:58:05.263162+0000 mon.a (mon.0) 2229 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi117-63923-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi117-63923-54"}]': finished 2024-03-04T15:58:06.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:05 smithi117 bash[28843]: cluster 2024-03-04T15:58:05.270553+0000 mon.a (mon.0) 2230 : cluster 0 osdmap e296: 8 total, 8 up, 8 in 2024-03-04T15:58:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:05 smithi151 bash[23756]: cluster 2024-03-04T15:58:04.723228+0000 mon.a (mon.0) 2228 : cluster 0 osdmap e295: 8 total, 8 up, 8 in 2024-03-04T15:58:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:05 smithi151 bash[23756]: cluster 2024-03-04T15:58:04.908880+0000 mgr.y (mgr.24428) 339 : cluster 0 pgmap v449: 228 pgs: 228 active+clean; 4.6 MiB data, 634 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:05 smithi151 bash[23756]: audit 2024-03-04T15:58:05.263162+0000 mon.a (mon.0) 2229 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi117-63923-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi117-63923-54"}]': finished 2024-03-04T15:58:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:05 smithi151 bash[23756]: cluster 2024-03-04T15:58:05.270553+0000 mon.a (mon.0) 2230 : cluster 0 osdmap e296: 8 total, 8 up, 8 in 2024-03-04T15:58:06.567 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:58:06 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:58:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:58:06.931 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:58:06 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:58:07.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:07 smithi117 bash[21480]: audit 2024-03-04T15:58:06.281647+0000 mon.c (mon.2) 477 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:07.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:07 smithi117 bash[21480]: cluster 2024-03-04T15:58:06.281858+0000 mon.a (mon.0) 2231 : cluster 0 osdmap e297: 8 total, 8 up, 8 in 2024-03-04T15:58:07.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:07 smithi117 bash[21480]: audit 2024-03-04T15:58:06.285095+0000 mon.a (mon.0) 2232 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:07.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:07 smithi117 bash[28843]: audit 2024-03-04T15:58:06.281647+0000 mon.c (mon.2) 477 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:07.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:07 smithi117 bash[28843]: cluster 2024-03-04T15:58:06.281858+0000 mon.a (mon.0) 2231 : cluster 0 osdmap e297: 8 total, 8 up, 8 in 2024-03-04T15:58:07.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:07 smithi117 bash[28843]: audit 2024-03-04T15:58:06.285095+0000 mon.a (mon.0) 2232 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:07.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:07 smithi151 bash[23756]: audit 2024-03-04T15:58:06.281647+0000 mon.c (mon.2) 477 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:07.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:07 smithi151 bash[23756]: cluster 2024-03-04T15:58:06.281858+0000 mon.a (mon.0) 2231 : cluster 0 osdmap e297: 8 total, 8 up, 8 in 2024-03-04T15:58:07.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:07 smithi151 bash[23756]: audit 2024-03-04T15:58:06.285095+0000 mon.a (mon.0) 2232 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:08.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:08 smithi117 bash[21480]: audit 2024-03-04T15:58:06.680093+0000 mgr.y (mgr.24428) 340 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:58:08.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:08 smithi117 bash[21480]: cluster 2024-03-04T15:58:06.911794+0000 mgr.y (mgr.24428) 341 : cluster 0 pgmap v452: 236 pgs: 32 creating+peering, 204 active+clean; 4.6 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-04T15:58:08.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:08 smithi117 bash[21480]: audit 2024-03-04T15:58:07.278615+0000 mon.a (mon.0) 2233 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:08.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:08 smithi117 bash[21480]: cluster 2024-03-04T15:58:07.288723+0000 mon.a (mon.0) 2234 : cluster 0 osdmap e298: 8 total, 8 up, 8 in 2024-03-04T15:58:08.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:08 smithi117 bash[21480]: audit 2024-03-04T15:58:07.295222+0000 mon.c (mon.2) 478 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:08.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:08 smithi117 bash[21480]: audit 2024-03-04T15:58:07.296183+0000 mon.a (mon.0) 2235 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:08.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:08 smithi117 bash[21480]: audit 2024-03-04T15:58:07.300493+0000 mon.c (mon.2) 479 : audit 1 from='client.? 172.21.15.117:0/4278747072' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:08.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:08 smithi117 bash[21480]: audit 2024-03-04T15:58:07.301794+0000 mon.a (mon.0) 2236 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:08.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:08 smithi117 bash[21480]: cluster 2024-03-04T15:58:08.239493+0000 mon.a (mon.0) 2237 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:08.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:08 smithi117 bash[28843]: audit 2024-03-04T15:58:06.680093+0000 mgr.y (mgr.24428) 340 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:58:08.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:08 smithi117 bash[28843]: cluster 2024-03-04T15:58:06.911794+0000 mgr.y (mgr.24428) 341 : cluster 0 pgmap v452: 236 pgs: 32 creating+peering, 204 active+clean; 4.6 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-04T15:58:08.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:08 smithi117 bash[28843]: audit 2024-03-04T15:58:07.278615+0000 mon.a (mon.0) 2233 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:08.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:08 smithi117 bash[28843]: cluster 2024-03-04T15:58:07.288723+0000 mon.a (mon.0) 2234 : cluster 0 osdmap e298: 8 total, 8 up, 8 in 2024-03-04T15:58:08.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:08 smithi117 bash[28843]: audit 2024-03-04T15:58:07.295222+0000 mon.c (mon.2) 478 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:08.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:08 smithi117 bash[28843]: audit 2024-03-04T15:58:07.296183+0000 mon.a (mon.0) 2235 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:08.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:08 smithi117 bash[28843]: audit 2024-03-04T15:58:07.300493+0000 mon.c (mon.2) 479 : audit 1 from='client.? 172.21.15.117:0/4278747072' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:08.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:08 smithi117 bash[28843]: audit 2024-03-04T15:58:07.301794+0000 mon.a (mon.0) 2236 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:08.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:08 smithi117 bash[28843]: cluster 2024-03-04T15:58:08.239493+0000 mon.a (mon.0) 2237 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:08.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:08 smithi151 bash[23756]: audit 2024-03-04T15:58:06.680093+0000 mgr.y (mgr.24428) 340 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:58:08.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:08 smithi151 bash[23756]: cluster 2024-03-04T15:58:06.911794+0000 mgr.y (mgr.24428) 341 : cluster 0 pgmap v452: 236 pgs: 32 creating+peering, 204 active+clean; 4.6 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-04T15:58:08.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:08 smithi151 bash[23756]: audit 2024-03-04T15:58:07.278615+0000 mon.a (mon.0) 2233 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:08.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:08 smithi151 bash[23756]: cluster 2024-03-04T15:58:07.288723+0000 mon.a (mon.0) 2234 : cluster 0 osdmap e298: 8 total, 8 up, 8 in 2024-03-04T15:58:08.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:08 smithi151 bash[23756]: audit 2024-03-04T15:58:07.295222+0000 mon.c (mon.2) 478 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:08.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:08 smithi151 bash[23756]: audit 2024-03-04T15:58:07.296183+0000 mon.a (mon.0) 2235 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:08.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:08 smithi151 bash[23756]: audit 2024-03-04T15:58:07.300493+0000 mon.c (mon.2) 479 : audit 1 from='client.? 172.21.15.117:0/4278747072' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:08.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:08 smithi151 bash[23756]: audit 2024-03-04T15:58:07.301794+0000 mon.a (mon.0) 2236 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:08.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:08 smithi151 bash[23756]: cluster 2024-03-04T15:58:08.239493+0000 mon.a (mon.0) 2237 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:09.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:09 smithi117 bash[21480]: audit 2024-03-04T15:58:08.290823+0000 mon.a (mon.0) 2238 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-39", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:58:09.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:09 smithi117 bash[21480]: audit 2024-03-04T15:58:08.291069+0000 mon.a (mon.0) 2239 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi117-63923-54"}]': finished 2024-03-04T15:58:09.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:09 smithi117 bash[21480]: cluster 2024-03-04T15:58:08.298031+0000 mon.a (mon.0) 2240 : cluster 0 osdmap e299: 8 total, 8 up, 8 in 2024-03-04T15:58:09.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:09 smithi117 bash[21480]: audit 2024-03-04T15:58:08.299356+0000 mon.c (mon.2) 480 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-39"}]: dispatch 2024-03-04T15:58:09.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:09 smithi117 bash[21480]: audit 2024-03-04T15:58:08.300463+0000 mon.a (mon.0) 2241 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-39"}]: dispatch 2024-03-04T15:58:09.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:09 smithi117 bash[21480]: audit 2024-03-04T15:58:08.300740+0000 mon.c (mon.2) 481 : audit 1 from='client.? 172.21.15.117:0/4278747072' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:09.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:09 smithi117 bash[21480]: audit 2024-03-04T15:58:08.302357+0000 mon.a (mon.0) 2242 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:09.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:09 smithi117 bash[21480]: cluster 2024-03-04T15:58:08.912661+0000 mgr.y (mgr.24428) 342 : cluster 0 pgmap v455: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T15:58:09.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:09 smithi117 bash[28843]: audit 2024-03-04T15:58:08.290823+0000 mon.a (mon.0) 2238 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-39", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:58:09.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:09 smithi117 bash[28843]: audit 2024-03-04T15:58:08.291069+0000 mon.a (mon.0) 2239 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi117-63923-54"}]': finished 2024-03-04T15:58:09.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:09 smithi117 bash[28843]: cluster 2024-03-04T15:58:08.298031+0000 mon.a (mon.0) 2240 : cluster 0 osdmap e299: 8 total, 8 up, 8 in 2024-03-04T15:58:09.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:09 smithi117 bash[28843]: audit 2024-03-04T15:58:08.299356+0000 mon.c (mon.2) 480 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-39"}]: dispatch 2024-03-04T15:58:09.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:09 smithi117 bash[28843]: audit 2024-03-04T15:58:08.300463+0000 mon.a (mon.0) 2241 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-39"}]: dispatch 2024-03-04T15:58:09.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:09 smithi117 bash[28843]: audit 2024-03-04T15:58:08.300740+0000 mon.c (mon.2) 481 : audit 1 from='client.? 172.21.15.117:0/4278747072' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:09.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:09 smithi117 bash[28843]: audit 2024-03-04T15:58:08.302357+0000 mon.a (mon.0) 2242 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:09.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:09 smithi117 bash[28843]: cluster 2024-03-04T15:58:08.912661+0000 mgr.y (mgr.24428) 342 : cluster 0 pgmap v455: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T15:58:09.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:09 smithi151 bash[23756]: audit 2024-03-04T15:58:08.290823+0000 mon.a (mon.0) 2238 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-39", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:58:09.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:09 smithi151 bash[23756]: audit 2024-03-04T15:58:08.291069+0000 mon.a (mon.0) 2239 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi117-63923-54"}]': finished 2024-03-04T15:58:09.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:09 smithi151 bash[23756]: cluster 2024-03-04T15:58:08.298031+0000 mon.a (mon.0) 2240 : cluster 0 osdmap e299: 8 total, 8 up, 8 in 2024-03-04T15:58:09.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:09 smithi151 bash[23756]: audit 2024-03-04T15:58:08.299356+0000 mon.c (mon.2) 480 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-39"}]: dispatch 2024-03-04T15:58:09.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:09 smithi151 bash[23756]: audit 2024-03-04T15:58:08.300463+0000 mon.a (mon.0) 2241 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-39"}]: dispatch 2024-03-04T15:58:09.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:09 smithi151 bash[23756]: audit 2024-03-04T15:58:08.300740+0000 mon.c (mon.2) 481 : audit 1 from='client.? 172.21.15.117:0/4278747072' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:09.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:09 smithi151 bash[23756]: audit 2024-03-04T15:58:08.302357+0000 mon.a (mon.0) 2242 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi117-63923-54"}]: dispatch 2024-03-04T15:58:09.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:09 smithi151 bash[23756]: cluster 2024-03-04T15:58:08.912661+0000 mgr.y (mgr.24428) 342 : cluster 0 pgmap v455: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T15:58:10.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:10 smithi117 bash[28843]: audit 2024-03-04T15:58:09.300382+0000 mon.a (mon.0) 2243 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-39"}]': finished 2024-03-04T15:58:10.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:10 smithi117 bash[28843]: audit 2024-03-04T15:58:09.300569+0000 mon.a (mon.0) 2244 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi117-63923-54"}]': finished 2024-03-04T15:58:10.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:10 smithi117 bash[28843]: cluster 2024-03-04T15:58:09.311150+0000 mon.a (mon.0) 2245 : cluster 0 osdmap e300: 8 total, 8 up, 8 in 2024-03-04T15:58:10.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:10 smithi117 bash[28843]: audit 2024-03-04T15:58:09.311402+0000 mon.c (mon.2) 482 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-39", "mode": "writeback"}]: dispatch 2024-03-04T15:58:10.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:10 smithi117 bash[28843]: audit 2024-03-04T15:58:09.316730+0000 mon.a (mon.0) 2246 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-39", "mode": "writeback"}]: dispatch 2024-03-04T15:58:10.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:10 smithi117 bash[28843]: audit 2024-03-04T15:58:09.348067+0000 mon.a (mon.0) 2247 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi117-63923-55"}]: dispatch 2024-03-04T15:58:10.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:10 smithi117 bash[28843]: audit 2024-03-04T15:58:09.349026+0000 mon.a (mon.0) 2248 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi117-63923-55"}]: dispatch 2024-03-04T15:58:10.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:10 smithi117 bash[28843]: audit 2024-03-04T15:58:09.349447+0000 mon.a (mon.0) 2249 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi117-63923-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:10.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:10 smithi117 bash[21480]: audit 2024-03-04T15:58:09.300382+0000 mon.a (mon.0) 2243 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-39"}]': finished 2024-03-04T15:58:10.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:10 smithi117 bash[21480]: audit 2024-03-04T15:58:09.300569+0000 mon.a (mon.0) 2244 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi117-63923-54"}]': finished 2024-03-04T15:58:10.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:10 smithi117 bash[21480]: cluster 2024-03-04T15:58:09.311150+0000 mon.a (mon.0) 2245 : cluster 0 osdmap e300: 8 total, 8 up, 8 in 2024-03-04T15:58:10.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:10 smithi117 bash[21480]: audit 2024-03-04T15:58:09.311402+0000 mon.c (mon.2) 482 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-39", "mode": "writeback"}]: dispatch 2024-03-04T15:58:10.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:10 smithi117 bash[21480]: audit 2024-03-04T15:58:09.316730+0000 mon.a (mon.0) 2246 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-39", "mode": "writeback"}]: dispatch 2024-03-04T15:58:10.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:10 smithi117 bash[21480]: audit 2024-03-04T15:58:09.348067+0000 mon.a (mon.0) 2247 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi117-63923-55"}]: dispatch 2024-03-04T15:58:10.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:10 smithi117 bash[21480]: audit 2024-03-04T15:58:09.349026+0000 mon.a (mon.0) 2248 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi117-63923-55"}]: dispatch 2024-03-04T15:58:10.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:10 smithi117 bash[21480]: audit 2024-03-04T15:58:09.349447+0000 mon.a (mon.0) 2249 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi117-63923-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:10 smithi151 bash[23756]: audit 2024-03-04T15:58:09.300382+0000 mon.a (mon.0) 2243 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-39"}]': finished 2024-03-04T15:58:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:10 smithi151 bash[23756]: audit 2024-03-04T15:58:09.300569+0000 mon.a (mon.0) 2244 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi117-63923-54"}]': finished 2024-03-04T15:58:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:10 smithi151 bash[23756]: cluster 2024-03-04T15:58:09.311150+0000 mon.a (mon.0) 2245 : cluster 0 osdmap e300: 8 total, 8 up, 8 in 2024-03-04T15:58:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:10 smithi151 bash[23756]: audit 2024-03-04T15:58:09.311402+0000 mon.c (mon.2) 482 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-39", "mode": "writeback"}]: dispatch 2024-03-04T15:58:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:10 smithi151 bash[23756]: audit 2024-03-04T15:58:09.316730+0000 mon.a (mon.0) 2246 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-39", "mode": "writeback"}]: dispatch 2024-03-04T15:58:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:10 smithi151 bash[23756]: audit 2024-03-04T15:58:09.348067+0000 mon.a (mon.0) 2247 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi117-63923-55"}]: dispatch 2024-03-04T15:58:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:10 smithi151 bash[23756]: audit 2024-03-04T15:58:09.349026+0000 mon.a (mon.0) 2248 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi117-63923-55"}]: dispatch 2024-03-04T15:58:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:10 smithi151 bash[23756]: audit 2024-03-04T15:58:09.349447+0000 mon.a (mon.0) 2249 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi117-63923-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:12.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:11 smithi117 bash[21480]: cluster 2024-03-04T15:58:10.300808+0000 mon.a (mon.0) 2250 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:58:12.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:11 smithi117 bash[21480]: audit 2024-03-04T15:58:10.303087+0000 mon.a (mon.0) 2251 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-39", "mode": "writeback"}]': finished 2024-03-04T15:58:12.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:11 smithi117 bash[21480]: audit 2024-03-04T15:58:10.303280+0000 mon.a (mon.0) 2252 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi117-63923-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:12.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:11 smithi117 bash[21480]: cluster 2024-03-04T15:58:10.311945+0000 mon.a (mon.0) 2253 : cluster 0 osdmap e301: 8 total, 8 up, 8 in 2024-03-04T15:58:12.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:11 smithi117 bash[21480]: audit 2024-03-04T15:58:10.315578+0000 mon.a (mon.0) 2254 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi117-63923-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi117-63923-55"}]: dispatch 2024-03-04T15:58:12.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:11 smithi117 bash[21480]: audit 2024-03-04T15:58:10.549091+0000 mon.c (mon.2) 483 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:12.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:11 smithi117 bash[21480]: audit 2024-03-04T15:58:10.549707+0000 mon.a (mon.0) 2255 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:12.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:11 smithi117 bash[21480]: cluster 2024-03-04T15:58:10.913516+0000 mgr.y (mgr.24428) 343 : cluster 0 pgmap v458: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:12.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:11 smithi117 bash[21480]: audit 2024-03-04T15:58:11.306939+0000 mon.a (mon.0) 2256 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:58:12.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:11 smithi117 bash[28843]: cluster 2024-03-04T15:58:10.300808+0000 mon.a (mon.0) 2250 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:58:12.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:11 smithi117 bash[28843]: audit 2024-03-04T15:58:10.303087+0000 mon.a (mon.0) 2251 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-39", "mode": "writeback"}]': finished 2024-03-04T15:58:12.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:11 smithi117 bash[28843]: audit 2024-03-04T15:58:10.303280+0000 mon.a (mon.0) 2252 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi117-63923-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:12.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:11 smithi117 bash[28843]: cluster 2024-03-04T15:58:10.311945+0000 mon.a (mon.0) 2253 : cluster 0 osdmap e301: 8 total, 8 up, 8 in 2024-03-04T15:58:12.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:11 smithi117 bash[28843]: audit 2024-03-04T15:58:10.315578+0000 mon.a (mon.0) 2254 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi117-63923-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi117-63923-55"}]: dispatch 2024-03-04T15:58:12.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:11 smithi117 bash[28843]: audit 2024-03-04T15:58:10.549091+0000 mon.c (mon.2) 483 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:12.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:11 smithi117 bash[28843]: audit 2024-03-04T15:58:10.549707+0000 mon.a (mon.0) 2255 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:12.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:11 smithi117 bash[28843]: cluster 2024-03-04T15:58:10.913516+0000 mgr.y (mgr.24428) 343 : cluster 0 pgmap v458: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:12.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:11 smithi117 bash[28843]: audit 2024-03-04T15:58:11.306939+0000 mon.a (mon.0) 2256 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:58:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:11 smithi151 bash[23756]: cluster 2024-03-04T15:58:10.300808+0000 mon.a (mon.0) 2250 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:58:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:11 smithi151 bash[23756]: audit 2024-03-04T15:58:10.303087+0000 mon.a (mon.0) 2251 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-39", "mode": "writeback"}]': finished 2024-03-04T15:58:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:11 smithi151 bash[23756]: audit 2024-03-04T15:58:10.303280+0000 mon.a (mon.0) 2252 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi117-63923-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:11 smithi151 bash[23756]: cluster 2024-03-04T15:58:10.311945+0000 mon.a (mon.0) 2253 : cluster 0 osdmap e301: 8 total, 8 up, 8 in 2024-03-04T15:58:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:11 smithi151 bash[23756]: audit 2024-03-04T15:58:10.315578+0000 mon.a (mon.0) 2254 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi117-63923-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi117-63923-55"}]: dispatch 2024-03-04T15:58:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:11 smithi151 bash[23756]: audit 2024-03-04T15:58:10.549091+0000 mon.c (mon.2) 483 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:11 smithi151 bash[23756]: audit 2024-03-04T15:58:10.549707+0000 mon.a (mon.0) 2255 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:11 smithi151 bash[23756]: cluster 2024-03-04T15:58:10.913516+0000 mgr.y (mgr.24428) 343 : cluster 0 pgmap v458: 228 pgs: 32 creating+peering, 196 active+clean; 4.6 MiB data, 619 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:11 smithi151 bash[23756]: audit 2024-03-04T15:58:11.306939+0000 mon.a (mon.0) 2256 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:58:13.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:12 smithi117 bash[28843]: audit 2024-03-04T15:58:11.316823+0000 mon.c (mon.2) 484 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-39"}]: dispatch 2024-03-04T15:58:13.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:12 smithi117 bash[28843]: cluster 2024-03-04T15:58:11.764893+0000 mon.a (mon.0) 2257 : cluster 0 osdmap e302: 8 total, 8 up, 8 in 2024-03-04T15:58:13.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:12 smithi117 bash[28843]: audit 2024-03-04T15:58:11.766487+0000 mon.a (mon.0) 2258 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-39"}]: dispatch 2024-03-04T15:58:13.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:12 smithi117 bash[28843]: cluster 2024-03-04T15:58:12.307539+0000 mon.a (mon.0) 2259 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:58:13.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:12 smithi117 bash[28843]: audit 2024-03-04T15:58:12.310024+0000 mon.a (mon.0) 2260 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi117-63923-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi117-63923-55"}]': finished 2024-03-04T15:58:13.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:12 smithi117 bash[28843]: audit 2024-03-04T15:58:12.310204+0000 mon.a (mon.0) 2261 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-39"}]': finished 2024-03-04T15:58:13.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:12 smithi117 bash[28843]: cluster 2024-03-04T15:58:12.315607+0000 mon.a (mon.0) 2262 : cluster 0 osdmap e303: 8 total, 8 up, 8 in 2024-03-04T15:58:13.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:12 smithi117 bash[21480]: audit 2024-03-04T15:58:11.316823+0000 mon.c (mon.2) 484 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-39"}]: dispatch 2024-03-04T15:58:13.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:12 smithi117 bash[21480]: cluster 2024-03-04T15:58:11.764893+0000 mon.a (mon.0) 2257 : cluster 0 osdmap e302: 8 total, 8 up, 8 in 2024-03-04T15:58:13.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:12 smithi117 bash[21480]: audit 2024-03-04T15:58:11.766487+0000 mon.a (mon.0) 2258 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-39"}]: dispatch 2024-03-04T15:58:13.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:12 smithi117 bash[21480]: cluster 2024-03-04T15:58:12.307539+0000 mon.a (mon.0) 2259 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:58:13.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:12 smithi117 bash[21480]: audit 2024-03-04T15:58:12.310024+0000 mon.a (mon.0) 2260 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi117-63923-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi117-63923-55"}]': finished 2024-03-04T15:58:13.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:12 smithi117 bash[21480]: audit 2024-03-04T15:58:12.310204+0000 mon.a (mon.0) 2261 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-39"}]': finished 2024-03-04T15:58:13.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:12 smithi117 bash[21480]: cluster 2024-03-04T15:58:12.315607+0000 mon.a (mon.0) 2262 : cluster 0 osdmap e303: 8 total, 8 up, 8 in 2024-03-04T15:58:13.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:12 smithi151 bash[23756]: audit 2024-03-04T15:58:11.316823+0000 mon.c (mon.2) 484 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-39"}]: dispatch 2024-03-04T15:58:13.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:12 smithi151 bash[23756]: cluster 2024-03-04T15:58:11.764893+0000 mon.a (mon.0) 2257 : cluster 0 osdmap e302: 8 total, 8 up, 8 in 2024-03-04T15:58:13.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:12 smithi151 bash[23756]: audit 2024-03-04T15:58:11.766487+0000 mon.a (mon.0) 2258 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-39"}]: dispatch 2024-03-04T15:58:13.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:12 smithi151 bash[23756]: cluster 2024-03-04T15:58:12.307539+0000 mon.a (mon.0) 2259 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:58:13.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:12 smithi151 bash[23756]: audit 2024-03-04T15:58:12.310024+0000 mon.a (mon.0) 2260 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi117-63923-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi117-63923-55"}]': finished 2024-03-04T15:58:13.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:12 smithi151 bash[23756]: audit 2024-03-04T15:58:12.310204+0000 mon.a (mon.0) 2261 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-39"}]': finished 2024-03-04T15:58:13.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:12 smithi151 bash[23756]: cluster 2024-03-04T15:58:12.315607+0000 mon.a (mon.0) 2262 : cluster 0 osdmap e303: 8 total, 8 up, 8 in 2024-03-04T15:58:13.352 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-03-04T15:58:13.352 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-03-04T15:58:13.353 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-03-04T15:58:13.353 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: seed 64061 2024-03-04T15:58:13.353 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-03-04T15:58:13.353 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (30 ms) 2024-03-04T15:58:13.353 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-03-04T15:58:13.353 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (20716 ms) 2024-03-04T15:58:13.353 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-03-04T15:58:13.353 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (47 ms) 2024-03-04T15:58:13.353 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (20793 ms total) 2024-03-04T15:58:13.353 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: 2024-03-04T15:58:13.353 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-03-04T15:58:13.353 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-03-04T15:58:13.353 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (16992 ms) 2024-03-04T15:58:13.353 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-03-04T15:58:13.354 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (12852 ms) 2024-03-04T15:58:13.354 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-03-04T15:58:13.354 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (12832 ms) 2024-03-04T15:58:13.354 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-03-04T15:58:13.354 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: my_snaps [3] 2024-03-04T15:58:13.354 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: my_snaps [4,3] 2024-03-04T15:58:13.354 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: my_snaps [5,4,3] 2024-03-04T15:58:13.354 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: my_snaps [6,5,4,3] 2024-03-04T15:58:13.354 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: promoting some heads 2024-03-04T15:58:13.354 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: promoting from clones for snap 6 2024-03-04T15:58:13.354 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: promoting from clones for snap 5 2024-03-04T15:58:13.354 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: promoting from clones for snap 4 2024-03-04T15:58:13.354 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: promoting from clones for snap 3 2024-03-04T15:58:13.354 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: waiting for scrubs... 2024-03-04T15:58:13.354 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: done waiting 2024-03-04T15:58:13.355 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (45909 ms) 2024-03-04T15:58:13.355 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-03-04T15:58:13.355 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (9965 ms) 2024-03-04T15:58:13.355 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-03-04T15:58:13.355 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (8422 ms) 2024-03-04T15:58:13.355 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-03-04T15:58:13.355 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (7945 ms) 2024-03-04T15:58:13.355 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-03-04T15:58:13.355 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8029 ms) 2024-03-04T15:58:13.355 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-03-04T15:58:13.355 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10104 ms) 2024-03-04T15:58:13.355 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-03-04T15:58:13.355 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (9293 ms) 2024-03-04T15:58:13.355 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-03-04T15:58:13.355 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (9603 ms) 2024-03-04T15:58:13.355 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-03-04T15:58:13.356 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (12911 ms) 2024-03-04T15:58:13.356 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-03-04T15:58:13.356 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (8054 ms) 2024-03-04T15:58:13.356 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-03-04T15:58:13.356 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8090 ms) 2024-03-04T15:58:13.356 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-03-04T15:58:13.356 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13541 ms) 2024-03-04T15:58:13.356 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-03-04T15:58:13.356 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7482 ms) 2024-03-04T15:58:13.356 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-03-04T15:58:13.356 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8083 ms) 2024-03-04T15:58:14.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:13 smithi117 bash[28843]: cluster 2024-03-04T15:58:12.914023+0000 mgr.y (mgr.24428) 344 : cluster 0 pgmap v461: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 96 op/s 2024-03-04T15:58:14.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:13 smithi117 bash[28843]: cluster 2024-03-04T15:58:13.241264+0000 mon.a (mon.0) 2263 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:14.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:13 smithi117 bash[28843]: cluster 2024-03-04T15:58:13.338229+0000 mon.a (mon.0) 2264 : cluster 0 osdmap e304: 8 total, 8 up, 8 in 2024-03-04T15:58:14.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:13 smithi117 bash[21480]: cluster 2024-03-04T15:58:12.914023+0000 mgr.y (mgr.24428) 344 : cluster 0 pgmap v461: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 96 op/s 2024-03-04T15:58:14.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:13 smithi117 bash[21480]: cluster 2024-03-04T15:58:13.241264+0000 mon.a (mon.0) 2263 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:14.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:13 smithi117 bash[21480]: cluster 2024-03-04T15:58:13.338229+0000 mon.a (mon.0) 2264 : cluster 0 osdmap e304: 8 total, 8 up, 8 in 2024-03-04T15:58:14.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:13 smithi151 bash[23756]: cluster 2024-03-04T15:58:12.914023+0000 mgr.y (mgr.24428) 344 : cluster 0 pgmap v461: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.9 MiB/s wr, 96 op/s 2024-03-04T15:58:14.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:13 smithi151 bash[23756]: cluster 2024-03-04T15:58:13.241264+0000 mon.a (mon.0) 2263 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:14.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:13 smithi151 bash[23756]: cluster 2024-03-04T15:58:13.338229+0000 mon.a (mon.0) 2264 : cluster 0 osdmap e304: 8 total, 8 up, 8 in 2024-03-04T15:58:15.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:14 smithi117 bash[21480]: audit 2024-03-04T15:58:13.759960+0000 mon.c (mon.2) 485 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:58:15.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:14 smithi117 bash[21480]: cluster 2024-03-04T15:58:14.348200+0000 mon.a (mon.0) 2265 : cluster 0 osdmap e305: 8 total, 8 up, 8 in 2024-03-04T15:58:15.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:14 smithi117 bash[21480]: audit 2024-03-04T15:58:14.350830+0000 mon.c (mon.2) 486 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:15.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:14 smithi117 bash[21480]: audit 2024-03-04T15:58:14.361060+0000 mon.a (mon.0) 2266 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi117-63923-55"}]: dispatch 2024-03-04T15:58:15.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:14 smithi117 bash[21480]: audit 2024-03-04T15:58:14.367260+0000 mon.a (mon.0) 2267 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:15.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:14 smithi117 bash[28843]: audit 2024-03-04T15:58:13.759960+0000 mon.c (mon.2) 485 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:58:15.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:14 smithi117 bash[28843]: cluster 2024-03-04T15:58:14.348200+0000 mon.a (mon.0) 2265 : cluster 0 osdmap e305: 8 total, 8 up, 8 in 2024-03-04T15:58:15.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:14 smithi117 bash[28843]: audit 2024-03-04T15:58:14.350830+0000 mon.c (mon.2) 486 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:15.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:14 smithi117 bash[28843]: audit 2024-03-04T15:58:14.361060+0000 mon.a (mon.0) 2266 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi117-63923-55"}]: dispatch 2024-03-04T15:58:15.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:14 smithi117 bash[28843]: audit 2024-03-04T15:58:14.367260+0000 mon.a (mon.0) 2267 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:15.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:14 smithi151 bash[23756]: audit 2024-03-04T15:58:13.759960+0000 mon.c (mon.2) 485 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:58:15.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:14 smithi151 bash[23756]: cluster 2024-03-04T15:58:14.348200+0000 mon.a (mon.0) 2265 : cluster 0 osdmap e305: 8 total, 8 up, 8 in 2024-03-04T15:58:15.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:14 smithi151 bash[23756]: audit 2024-03-04T15:58:14.350830+0000 mon.c (mon.2) 486 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:15.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:14 smithi151 bash[23756]: audit 2024-03-04T15:58:14.361060+0000 mon.a (mon.0) 2266 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi117-63923-55"}]: dispatch 2024-03-04T15:58:15.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:14 smithi151 bash[23756]: audit 2024-03-04T15:58:14.367260+0000 mon.a (mon.0) 2267 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:16.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:15 smithi117 bash[28843]: cluster 2024-03-04T15:58:14.914820+0000 mgr.y (mgr.24428) 345 : cluster 0 pgmap v464: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-04T15:58:16.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:15 smithi117 bash[28843]: audit 2024-03-04T15:58:15.337399+0000 mon.a (mon.0) 2268 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi117-63923-55"}]': finished 2024-03-04T15:58:16.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:15 smithi117 bash[28843]: audit 2024-03-04T15:58:15.337612+0000 mon.a (mon.0) 2269 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:16.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:15 smithi117 bash[28843]: cluster 2024-03-04T15:58:15.351676+0000 mon.a (mon.0) 2270 : cluster 0 osdmap e306: 8 total, 8 up, 8 in 2024-03-04T15:58:16.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:15 smithi117 bash[28843]: audit 2024-03-04T15:58:15.353742+0000 mon.c (mon.2) 487 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:16.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:15 smithi117 bash[28843]: audit 2024-03-04T15:58:15.356817+0000 mon.a (mon.0) 2271 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi117-63923-55"}]: dispatch 2024-03-04T15:58:16.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:15 smithi117 bash[28843]: audit 2024-03-04T15:58:15.357854+0000 mon.a (mon.0) 2272 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:16.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:15 smithi117 bash[21480]: cluster 2024-03-04T15:58:14.914820+0000 mgr.y (mgr.24428) 345 : cluster 0 pgmap v464: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-04T15:58:16.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:15 smithi117 bash[21480]: audit 2024-03-04T15:58:15.337399+0000 mon.a (mon.0) 2268 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi117-63923-55"}]': finished 2024-03-04T15:58:16.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:15 smithi117 bash[21480]: audit 2024-03-04T15:58:15.337612+0000 mon.a (mon.0) 2269 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:16.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:15 smithi117 bash[21480]: cluster 2024-03-04T15:58:15.351676+0000 mon.a (mon.0) 2270 : cluster 0 osdmap e306: 8 total, 8 up, 8 in 2024-03-04T15:58:16.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:15 smithi117 bash[21480]: audit 2024-03-04T15:58:15.353742+0000 mon.c (mon.2) 487 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:16.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:15 smithi117 bash[21480]: audit 2024-03-04T15:58:15.356817+0000 mon.a (mon.0) 2271 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi117-63923-55"}]: dispatch 2024-03-04T15:58:16.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:15 smithi117 bash[21480]: audit 2024-03-04T15:58:15.357854+0000 mon.a (mon.0) 2272 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:16.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:15 smithi151 bash[23756]: cluster 2024-03-04T15:58:14.914820+0000 mgr.y (mgr.24428) 345 : cluster 0 pgmap v464: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 631 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-04T15:58:16.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:15 smithi151 bash[23756]: audit 2024-03-04T15:58:15.337399+0000 mon.a (mon.0) 2268 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi117-63923-55"}]': finished 2024-03-04T15:58:16.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:15 smithi151 bash[23756]: audit 2024-03-04T15:58:15.337612+0000 mon.a (mon.0) 2269 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:16.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:15 smithi151 bash[23756]: cluster 2024-03-04T15:58:15.351676+0000 mon.a (mon.0) 2270 : cluster 0 osdmap e306: 8 total, 8 up, 8 in 2024-03-04T15:58:16.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:15 smithi151 bash[23756]: audit 2024-03-04T15:58:15.353742+0000 mon.c (mon.2) 487 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:16.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:15 smithi151 bash[23756]: audit 2024-03-04T15:58:15.356817+0000 mon.a (mon.0) 2271 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi117-63923-55"}]: dispatch 2024-03-04T15:58:16.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:15 smithi151 bash[23756]: audit 2024-03-04T15:58:15.357854+0000 mon.a (mon.0) 2272 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:16.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:58:16 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:58:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:58:16.931 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:58:16 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:58:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:17 smithi151 bash[23756]: audit 2024-03-04T15:58:16.340707+0000 mon.a (mon.0) 2273 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi117-63923-55"}]': finished 2024-03-04T15:58:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:17 smithi151 bash[23756]: audit 2024-03-04T15:58:16.340901+0000 mon.a (mon.0) 2274 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-41", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:58:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:17 smithi151 bash[23756]: audit 2024-03-04T15:58:16.346250+0000 mon.c (mon.2) 488 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T15:58:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:17 smithi151 bash[23756]: cluster 2024-03-04T15:58:16.348512+0000 mon.a (mon.0) 2275 : cluster 0 osdmap e307: 8 total, 8 up, 8 in 2024-03-04T15:58:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:17 smithi151 bash[23756]: audit 2024-03-04T15:58:16.350022+0000 mon.a (mon.0) 2276 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T15:58:17.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:17 smithi151 bash[23756]: audit 2024-03-04T15:58:16.369992+0000 mon.c (mon.2) 489 : audit 1 from='client.? 172.21.15.117:0/669892112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:17.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:17 smithi151 bash[23756]: audit 2024-03-04T15:58:16.370457+0000 mon.a (mon.0) 2277 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:17.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:17 smithi151 bash[23756]: audit 2024-03-04T15:58:16.371182+0000 mon.c (mon.2) 490 : audit 1 from='client.? 172.21.15.117:0/669892112' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:17.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:17 smithi151 bash[23756]: audit 2024-03-04T15:58:16.371448+0000 mon.a (mon.0) 2278 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:17.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:17 smithi151 bash[23756]: audit 2024-03-04T15:58:16.371950+0000 mon.c (mon.2) 491 : audit 1 from='client.? 172.21.15.117:0/669892112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi117-63923-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:17.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:17 smithi151 bash[23756]: audit 2024-03-04T15:58:16.372215+0000 mon.a (mon.0) 2279 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi117-63923-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:17.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:17 smithi151 bash[23756]: audit 2024-03-04T15:58:16.681882+0000 mgr.y (mgr.24428) 346 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:58:17.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:17 smithi151 bash[23756]: cluster 2024-03-04T15:58:16.916979+0000 mgr.y (mgr.24428) 347 : cluster 0 pgmap v467: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:17.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[28843]: audit 2024-03-04T15:58:16.340707+0000 mon.a (mon.0) 2273 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi117-63923-55"}]': finished 2024-03-04T15:58:17.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[28843]: audit 2024-03-04T15:58:16.340901+0000 mon.a (mon.0) 2274 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-41", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:58:17.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[28843]: audit 2024-03-04T15:58:16.346250+0000 mon.c (mon.2) 488 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T15:58:17.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[28843]: cluster 2024-03-04T15:58:16.348512+0000 mon.a (mon.0) 2275 : cluster 0 osdmap e307: 8 total, 8 up, 8 in 2024-03-04T15:58:17.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[28843]: audit 2024-03-04T15:58:16.350022+0000 mon.a (mon.0) 2276 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T15:58:17.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[28843]: audit 2024-03-04T15:58:16.369992+0000 mon.c (mon.2) 489 : audit 1 from='client.? 172.21.15.117:0/669892112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:17.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[28843]: audit 2024-03-04T15:58:16.370457+0000 mon.a (mon.0) 2277 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:17.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[28843]: audit 2024-03-04T15:58:16.371182+0000 mon.c (mon.2) 490 : audit 1 from='client.? 172.21.15.117:0/669892112' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:17.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[28843]: audit 2024-03-04T15:58:16.371448+0000 mon.a (mon.0) 2278 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:17.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[28843]: audit 2024-03-04T15:58:16.371950+0000 mon.c (mon.2) 491 : audit 1 from='client.? 172.21.15.117:0/669892112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi117-63923-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:17.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[28843]: audit 2024-03-04T15:58:16.372215+0000 mon.a (mon.0) 2279 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi117-63923-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:17.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[28843]: audit 2024-03-04T15:58:16.681882+0000 mgr.y (mgr.24428) 346 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:58:17.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[28843]: cluster 2024-03-04T15:58:16.916979+0000 mgr.y (mgr.24428) 347 : cluster 0 pgmap v467: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:17.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[21480]: audit 2024-03-04T15:58:16.340707+0000 mon.a (mon.0) 2273 : audit 1 from='client.? 172.21.15.117:0/251077004' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi117-63923-55"}]': finished 2024-03-04T15:58:17.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[21480]: audit 2024-03-04T15:58:16.340901+0000 mon.a (mon.0) 2274 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-41", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:58:17.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[21480]: audit 2024-03-04T15:58:16.346250+0000 mon.c (mon.2) 488 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T15:58:17.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[21480]: cluster 2024-03-04T15:58:16.348512+0000 mon.a (mon.0) 2275 : cluster 0 osdmap e307: 8 total, 8 up, 8 in 2024-03-04T15:58:17.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[21480]: audit 2024-03-04T15:58:16.350022+0000 mon.a (mon.0) 2276 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T15:58:17.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[21480]: audit 2024-03-04T15:58:16.369992+0000 mon.c (mon.2) 489 : audit 1 from='client.? 172.21.15.117:0/669892112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:17.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[21480]: audit 2024-03-04T15:58:16.370457+0000 mon.a (mon.0) 2277 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:17.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[21480]: audit 2024-03-04T15:58:16.371182+0000 mon.c (mon.2) 490 : audit 1 from='client.? 172.21.15.117:0/669892112' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:17.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[21480]: audit 2024-03-04T15:58:16.371448+0000 mon.a (mon.0) 2278 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:17.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[21480]: audit 2024-03-04T15:58:16.371950+0000 mon.c (mon.2) 491 : audit 1 from='client.? 172.21.15.117:0/669892112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi117-63923-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:17.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[21480]: audit 2024-03-04T15:58:16.372215+0000 mon.a (mon.0) 2279 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi117-63923-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:17.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[21480]: audit 2024-03-04T15:58:16.681882+0000 mgr.y (mgr.24428) 346 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:58:17.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:17 smithi117 bash[21480]: cluster 2024-03-04T15:58:16.916979+0000 mgr.y (mgr.24428) 347 : cluster 0 pgmap v467: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:19.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:18 smithi117 bash[21480]: audit 2024-03-04T15:58:17.355744+0000 mon.a (mon.0) 2280 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_count","val": "2"}]': finished 2024-03-04T15:58:19.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:18 smithi117 bash[21480]: audit 2024-03-04T15:58:17.355960+0000 mon.a (mon.0) 2281 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi117-63923-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:19.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:18 smithi117 bash[21480]: cluster 2024-03-04T15:58:17.362889+0000 mon.a (mon.0) 2282 : cluster 0 osdmap e308: 8 total, 8 up, 8 in 2024-03-04T15:58:19.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:18 smithi117 bash[21480]: audit 2024-03-04T15:58:17.366277+0000 mon.c (mon.2) 492 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:58:19.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:18 smithi117 bash[21480]: audit 2024-03-04T15:58:17.366904+0000 mon.c (mon.2) 493 : audit 1 from='client.? 172.21.15.117:0/669892112' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi117-63923-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:19.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:18 smithi117 bash[21480]: audit 2024-03-04T15:58:17.367526+0000 mon.a (mon.0) 2283 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:58:19.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:18 smithi117 bash[21480]: audit 2024-03-04T15:58:17.368432+0000 mon.a (mon.0) 2284 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi117-63923-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:19.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:18 smithi117 bash[28843]: audit 2024-03-04T15:58:17.355744+0000 mon.a (mon.0) 2280 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_count","val": "2"}]': finished 2024-03-04T15:58:19.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:18 smithi117 bash[28843]: audit 2024-03-04T15:58:17.355960+0000 mon.a (mon.0) 2281 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi117-63923-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:19.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:18 smithi117 bash[28843]: cluster 2024-03-04T15:58:17.362889+0000 mon.a (mon.0) 2282 : cluster 0 osdmap e308: 8 total, 8 up, 8 in 2024-03-04T15:58:19.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:18 smithi117 bash[28843]: audit 2024-03-04T15:58:17.366277+0000 mon.c (mon.2) 492 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:58:19.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:18 smithi117 bash[28843]: audit 2024-03-04T15:58:17.366904+0000 mon.c (mon.2) 493 : audit 1 from='client.? 172.21.15.117:0/669892112' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi117-63923-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:19.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:18 smithi117 bash[28843]: audit 2024-03-04T15:58:17.367526+0000 mon.a (mon.0) 2283 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:58:19.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:18 smithi117 bash[28843]: audit 2024-03-04T15:58:17.368432+0000 mon.a (mon.0) 2284 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi117-63923-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:19.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:18 smithi151 bash[23756]: audit 2024-03-04T15:58:17.355744+0000 mon.a (mon.0) 2280 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_count","val": "2"}]': finished 2024-03-04T15:58:19.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:18 smithi151 bash[23756]: audit 2024-03-04T15:58:17.355960+0000 mon.a (mon.0) 2281 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi117-63923-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:19.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:18 smithi151 bash[23756]: cluster 2024-03-04T15:58:17.362889+0000 mon.a (mon.0) 2282 : cluster 0 osdmap e308: 8 total, 8 up, 8 in 2024-03-04T15:58:19.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:18 smithi151 bash[23756]: audit 2024-03-04T15:58:17.366277+0000 mon.c (mon.2) 492 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:58:19.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:18 smithi151 bash[23756]: audit 2024-03-04T15:58:17.366904+0000 mon.c (mon.2) 493 : audit 1 from='client.? 172.21.15.117:0/669892112' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi117-63923-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:19.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:18 smithi151 bash[23756]: audit 2024-03-04T15:58:17.367526+0000 mon.a (mon.0) 2283 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:58:19.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:18 smithi151 bash[23756]: audit 2024-03-04T15:58:17.368432+0000 mon.a (mon.0) 2284 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi117-63923-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:20.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:19 smithi117 bash[21480]: audit 2024-03-04T15:58:18.359097+0000 mon.a (mon.0) 2285 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_period","val": "600"}]': finished 2024-03-04T15:58:20.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:19 smithi117 bash[21480]: audit 2024-03-04T15:58:18.368830+0000 mon.c (mon.2) 494 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T15:58:20.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:19 smithi117 bash[21480]: cluster 2024-03-04T15:58:18.814924+0000 mon.a (mon.0) 2286 : cluster 0 osdmap e309: 8 total, 8 up, 8 in 2024-03-04T15:58:20.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:19 smithi117 bash[21480]: audit 2024-03-04T15:58:18.816620+0000 mon.a (mon.0) 2287 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T15:58:20.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:19 smithi117 bash[21480]: cluster 2024-03-04T15:58:18.917776+0000 mgr.y (mgr.24428) 348 : cluster 0 pgmap v470: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:20.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:19 smithi117 bash[21480]: audit 2024-03-04T15:58:19.361968+0000 mon.a (mon.0) 2288 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi117-63923-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi117-63923-56"}]': finished 2024-03-04T15:58:20.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:19 smithi117 bash[21480]: audit 2024-03-04T15:58:19.362241+0000 mon.a (mon.0) 2289 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-04T15:58:20.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:19 smithi117 bash[21480]: cluster 2024-03-04T15:58:19.371321+0000 mon.a (mon.0) 2290 : cluster 0 osdmap e310: 8 total, 8 up, 8 in 2024-03-04T15:58:20.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:19 smithi117 bash[28843]: audit 2024-03-04T15:58:18.359097+0000 mon.a (mon.0) 2285 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_period","val": "600"}]': finished 2024-03-04T15:58:20.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:19 smithi117 bash[28843]: audit 2024-03-04T15:58:18.368830+0000 mon.c (mon.2) 494 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T15:58:20.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:19 smithi117 bash[28843]: cluster 2024-03-04T15:58:18.814924+0000 mon.a (mon.0) 2286 : cluster 0 osdmap e309: 8 total, 8 up, 8 in 2024-03-04T15:58:20.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:19 smithi117 bash[28843]: audit 2024-03-04T15:58:18.816620+0000 mon.a (mon.0) 2287 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T15:58:20.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:19 smithi117 bash[28843]: cluster 2024-03-04T15:58:18.917776+0000 mgr.y (mgr.24428) 348 : cluster 0 pgmap v470: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:20.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:19 smithi117 bash[28843]: audit 2024-03-04T15:58:19.361968+0000 mon.a (mon.0) 2288 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi117-63923-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi117-63923-56"}]': finished 2024-03-04T15:58:20.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:19 smithi117 bash[28843]: audit 2024-03-04T15:58:19.362241+0000 mon.a (mon.0) 2289 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-04T15:58:20.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:19 smithi117 bash[28843]: cluster 2024-03-04T15:58:19.371321+0000 mon.a (mon.0) 2290 : cluster 0 osdmap e310: 8 total, 8 up, 8 in 2024-03-04T15:58:20.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:19 smithi151 bash[23756]: audit 2024-03-04T15:58:18.359097+0000 mon.a (mon.0) 2285 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_period","val": "600"}]': finished 2024-03-04T15:58:20.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:19 smithi151 bash[23756]: audit 2024-03-04T15:58:18.368830+0000 mon.c (mon.2) 494 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T15:58:20.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:19 smithi151 bash[23756]: cluster 2024-03-04T15:58:18.814924+0000 mon.a (mon.0) 2286 : cluster 0 osdmap e309: 8 total, 8 up, 8 in 2024-03-04T15:58:20.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:19 smithi151 bash[23756]: audit 2024-03-04T15:58:18.816620+0000 mon.a (mon.0) 2287 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T15:58:20.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:19 smithi151 bash[23756]: cluster 2024-03-04T15:58:18.917776+0000 mgr.y (mgr.24428) 348 : cluster 0 pgmap v470: 228 pgs: 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:20.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:19 smithi151 bash[23756]: audit 2024-03-04T15:58:19.361968+0000 mon.a (mon.0) 2288 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi117-63923-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi117-63923-56"}]': finished 2024-03-04T15:58:20.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:19 smithi151 bash[23756]: audit 2024-03-04T15:58:19.362241+0000 mon.a (mon.0) 2289 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-04T15:58:20.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:19 smithi151 bash[23756]: cluster 2024-03-04T15:58:19.371321+0000 mon.a (mon.0) 2290 : cluster 0 osdmap e310: 8 total, 8 up, 8 in 2024-03-04T15:58:21.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:21 smithi151 bash[23756]: cluster 2024-03-04T15:58:20.372455+0000 mon.a (mon.0) 2291 : cluster 0 osdmap e311: 8 total, 8 up, 8 in 2024-03-04T15:58:21.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:21 smithi151 bash[23756]: audit 2024-03-04T15:58:20.439846+0000 mon.c (mon.2) 495 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:21.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:21 smithi151 bash[23756]: audit 2024-03-04T15:58:20.440523+0000 mon.a (mon.0) 2292 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:21.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:21 smithi151 bash[23756]: audit 2024-03-04T15:58:20.441627+0000 mon.c (mon.2) 496 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-41"}]: dispatch 2024-03-04T15:58:21.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:21 smithi151 bash[23756]: audit 2024-03-04T15:58:20.442294+0000 mon.a (mon.0) 2293 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-41"}]: dispatch 2024-03-04T15:58:21.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:21 smithi151 bash[23756]: cluster 2024-03-04T15:58:20.918616+0000 mgr.y (mgr.24428) 349 : cluster 0 pgmap v473: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:21.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:21 smithi117 bash[21480]: cluster 2024-03-04T15:58:20.372455+0000 mon.a (mon.0) 2291 : cluster 0 osdmap e311: 8 total, 8 up, 8 in 2024-03-04T15:58:21.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:21 smithi117 bash[21480]: audit 2024-03-04T15:58:20.439846+0000 mon.c (mon.2) 495 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:21.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:21 smithi117 bash[21480]: audit 2024-03-04T15:58:20.440523+0000 mon.a (mon.0) 2292 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:21.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:21 smithi117 bash[21480]: audit 2024-03-04T15:58:20.441627+0000 mon.c (mon.2) 496 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-41"}]: dispatch 2024-03-04T15:58:21.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:21 smithi117 bash[21480]: audit 2024-03-04T15:58:20.442294+0000 mon.a (mon.0) 2293 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-41"}]: dispatch 2024-03-04T15:58:21.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:21 smithi117 bash[21480]: cluster 2024-03-04T15:58:20.918616+0000 mgr.y (mgr.24428) 349 : cluster 0 pgmap v473: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:21.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:21 smithi117 bash[28843]: cluster 2024-03-04T15:58:20.372455+0000 mon.a (mon.0) 2291 : cluster 0 osdmap e311: 8 total, 8 up, 8 in 2024-03-04T15:58:21.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:21 smithi117 bash[28843]: audit 2024-03-04T15:58:20.439846+0000 mon.c (mon.2) 495 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:21.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:21 smithi117 bash[28843]: audit 2024-03-04T15:58:20.440523+0000 mon.a (mon.0) 2292 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:21.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:21 smithi117 bash[28843]: audit 2024-03-04T15:58:20.441627+0000 mon.c (mon.2) 496 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-41"}]: dispatch 2024-03-04T15:58:21.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:21 smithi117 bash[28843]: audit 2024-03-04T15:58:20.442294+0000 mon.a (mon.0) 2293 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-41"}]: dispatch 2024-03-04T15:58:21.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:21 smithi117 bash[28843]: cluster 2024-03-04T15:58:20.918616+0000 mgr.y (mgr.24428) 349 : cluster 0 pgmap v473: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 636 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:22.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:22 smithi151 bash[23756]: cluster 2024-03-04T15:58:21.365009+0000 mon.a (mon.0) 2294 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:22.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:22 smithi151 bash[23756]: audit 2024-03-04T15:58:21.375163+0000 mon.a (mon.0) 2295 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-41"}]': finished 2024-03-04T15:58:22.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:22 smithi151 bash[23756]: cluster 2024-03-04T15:58:21.386086+0000 mon.a (mon.0) 2296 : cluster 0 osdmap e312: 8 total, 8 up, 8 in 2024-03-04T15:58:22.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:22 smithi151 bash[23756]: audit 2024-03-04T15:58:21.389614+0000 mon.c (mon.2) 497 : audit 1 from='client.? 172.21.15.117:0/669892112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:22.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:22 smithi151 bash[23756]: audit 2024-03-04T15:58:21.391818+0000 mon.a (mon.0) 2297 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:22.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:22 smithi117 bash[21480]: cluster 2024-03-04T15:58:21.365009+0000 mon.a (mon.0) 2294 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:22.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:22 smithi117 bash[21480]: audit 2024-03-04T15:58:21.375163+0000 mon.a (mon.0) 2295 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-41"}]': finished 2024-03-04T15:58:22.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:22 smithi117 bash[21480]: cluster 2024-03-04T15:58:21.386086+0000 mon.a (mon.0) 2296 : cluster 0 osdmap e312: 8 total, 8 up, 8 in 2024-03-04T15:58:22.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:22 smithi117 bash[21480]: audit 2024-03-04T15:58:21.389614+0000 mon.c (mon.2) 497 : audit 1 from='client.? 172.21.15.117:0/669892112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:22.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:22 smithi117 bash[21480]: audit 2024-03-04T15:58:21.391818+0000 mon.a (mon.0) 2297 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:22.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:22 smithi117 bash[28843]: cluster 2024-03-04T15:58:21.365009+0000 mon.a (mon.0) 2294 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:22.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:22 smithi117 bash[28843]: audit 2024-03-04T15:58:21.375163+0000 mon.a (mon.0) 2295 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-41"}]': finished 2024-03-04T15:58:22.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:22 smithi117 bash[28843]: cluster 2024-03-04T15:58:21.386086+0000 mon.a (mon.0) 2296 : cluster 0 osdmap e312: 8 total, 8 up, 8 in 2024-03-04T15:58:22.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:22 smithi117 bash[28843]: audit 2024-03-04T15:58:21.389614+0000 mon.c (mon.2) 497 : audit 1 from='client.? 172.21.15.117:0/669892112' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:22.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:22 smithi117 bash[28843]: audit 2024-03-04T15:58:21.391818+0000 mon.a (mon.0) 2297 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:23.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:23 smithi151 bash[23756]: audit 2024-03-04T15:58:22.378167+0000 mon.a (mon.0) 2298 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi117-63923-56"}]': finished 2024-03-04T15:58:23.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:23 smithi151 bash[23756]: cluster 2024-03-04T15:58:22.386035+0000 mon.a (mon.0) 2299 : cluster 0 osdmap e313: 8 total, 8 up, 8 in 2024-03-04T15:58:23.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:23 smithi151 bash[23756]: audit 2024-03-04T15:58:22.388224+0000 mon.c (mon.2) 498 : audit 1 from='client.? 172.21.15.117:0/669892112' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:23.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:23 smithi151 bash[23756]: audit 2024-03-04T15:58:22.389330+0000 mon.a (mon.0) 2300 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:23.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:23 smithi151 bash[23756]: cluster 2024-03-04T15:58:22.919399+0000 mgr.y (mgr.24428) 350 : cluster 0 pgmap v476: 196 pgs: 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:23.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:23 smithi151 bash[23756]: audit 2024-03-04T15:58:23.382637+0000 mon.a (mon.0) 2301 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi117-63923-56"}]': finished 2024-03-04T15:58:23.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:23 smithi151 bash[23756]: cluster 2024-03-04T15:58:23.394718+0000 mon.a (mon.0) 2302 : cluster 0 osdmap e314: 8 total, 8 up, 8 in 2024-03-04T15:58:23.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:23 smithi117 bash[21480]: audit 2024-03-04T15:58:22.378167+0000 mon.a (mon.0) 2298 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi117-63923-56"}]': finished 2024-03-04T15:58:23.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:23 smithi117 bash[21480]: cluster 2024-03-04T15:58:22.386035+0000 mon.a (mon.0) 2299 : cluster 0 osdmap e313: 8 total, 8 up, 8 in 2024-03-04T15:58:23.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:23 smithi117 bash[21480]: audit 2024-03-04T15:58:22.388224+0000 mon.c (mon.2) 498 : audit 1 from='client.? 172.21.15.117:0/669892112' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:23.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:23 smithi117 bash[21480]: audit 2024-03-04T15:58:22.389330+0000 mon.a (mon.0) 2300 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:23.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:23 smithi117 bash[21480]: cluster 2024-03-04T15:58:22.919399+0000 mgr.y (mgr.24428) 350 : cluster 0 pgmap v476: 196 pgs: 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:23.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:23 smithi117 bash[21480]: audit 2024-03-04T15:58:23.382637+0000 mon.a (mon.0) 2301 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi117-63923-56"}]': finished 2024-03-04T15:58:23.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:23 smithi117 bash[21480]: cluster 2024-03-04T15:58:23.394718+0000 mon.a (mon.0) 2302 : cluster 0 osdmap e314: 8 total, 8 up, 8 in 2024-03-04T15:58:23.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:23 smithi117 bash[28843]: audit 2024-03-04T15:58:22.378167+0000 mon.a (mon.0) 2298 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi117-63923-56"}]': finished 2024-03-04T15:58:23.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:23 smithi117 bash[28843]: cluster 2024-03-04T15:58:22.386035+0000 mon.a (mon.0) 2299 : cluster 0 osdmap e313: 8 total, 8 up, 8 in 2024-03-04T15:58:23.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:23 smithi117 bash[28843]: audit 2024-03-04T15:58:22.388224+0000 mon.c (mon.2) 498 : audit 1 from='client.? 172.21.15.117:0/669892112' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:23.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:23 smithi117 bash[28843]: audit 2024-03-04T15:58:22.389330+0000 mon.a (mon.0) 2300 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi117-63923-56"}]: dispatch 2024-03-04T15:58:23.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:23 smithi117 bash[28843]: cluster 2024-03-04T15:58:22.919399+0000 mgr.y (mgr.24428) 350 : cluster 0 pgmap v476: 196 pgs: 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:23.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:23 smithi117 bash[28843]: audit 2024-03-04T15:58:23.382637+0000 mon.a (mon.0) 2301 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi117-63923-56"}]': finished 2024-03-04T15:58:23.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:23 smithi117 bash[28843]: cluster 2024-03-04T15:58:23.394718+0000 mon.a (mon.0) 2302 : cluster 0 osdmap e314: 8 total, 8 up, 8 in 2024-03-04T15:58:24.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:24 smithi151 bash[23756]: audit 2024-03-04T15:58:23.397594+0000 mon.c (mon.2) 499 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:24.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:24 smithi151 bash[23756]: audit 2024-03-04T15:58:23.403033+0000 mon.a (mon.0) 2303 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:24.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:24 smithi151 bash[23756]: audit 2024-03-04T15:58:23.438712+0000 mon.a (mon.0) 2304 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi117-63923-57"}]: dispatch 2024-03-04T15:58:24.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:24 smithi151 bash[23756]: audit 2024-03-04T15:58:23.439700+0000 mon.a (mon.0) 2305 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi117-63923-57"}]: dispatch 2024-03-04T15:58:24.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:24 smithi151 bash[23756]: audit 2024-03-04T15:58:23.440097+0000 mon.a (mon.0) 2306 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi117-63923-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:24.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:24 smithi151 bash[23756]: audit 2024-03-04T15:58:24.385641+0000 mon.a (mon.0) 2307 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:24.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:24 smithi151 bash[23756]: audit 2024-03-04T15:58:24.385873+0000 mon.a (mon.0) 2308 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi117-63923-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:24.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:24 smithi151 bash[23756]: cluster 2024-03-04T15:58:24.395989+0000 mon.a (mon.0) 2309 : cluster 0 osdmap e315: 8 total, 8 up, 8 in 2024-03-04T15:58:24.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:24 smithi151 bash[23756]: audit 2024-03-04T15:58:24.398226+0000 mon.c (mon.2) 500 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi117-64061-6","var": "pg_num","format": "json"}]: dispatch 2024-03-04T15:58:24.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:24 smithi151 bash[23756]: audit 2024-03-04T15:58:24.402760+0000 mon.a (mon.0) 2310 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi117-63923-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi117-63923-57"}]: dispatch 2024-03-04T15:58:24.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:24 smithi151 bash[23756]: audit 2024-03-04T15:58:24.408283+0000 mon.c (mon.2) 501 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:24.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:24 smithi151 bash[23756]: audit 2024-03-04T15:58:24.409624+0000 mon.a (mon.0) 2311 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:24.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[21480]: audit 2024-03-04T15:58:23.397594+0000 mon.c (mon.2) 499 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:24.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[21480]: audit 2024-03-04T15:58:23.403033+0000 mon.a (mon.0) 2303 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:24.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[21480]: audit 2024-03-04T15:58:23.438712+0000 mon.a (mon.0) 2304 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi117-63923-57"}]: dispatch 2024-03-04T15:58:24.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[21480]: audit 2024-03-04T15:58:23.439700+0000 mon.a (mon.0) 2305 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi117-63923-57"}]: dispatch 2024-03-04T15:58:24.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[21480]: audit 2024-03-04T15:58:23.440097+0000 mon.a (mon.0) 2306 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi117-63923-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:24.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[21480]: audit 2024-03-04T15:58:24.385641+0000 mon.a (mon.0) 2307 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:24.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[21480]: audit 2024-03-04T15:58:24.385873+0000 mon.a (mon.0) 2308 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi117-63923-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:24.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[21480]: cluster 2024-03-04T15:58:24.395989+0000 mon.a (mon.0) 2309 : cluster 0 osdmap e315: 8 total, 8 up, 8 in 2024-03-04T15:58:24.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[21480]: audit 2024-03-04T15:58:24.398226+0000 mon.c (mon.2) 500 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi117-64061-6","var": "pg_num","format": "json"}]: dispatch 2024-03-04T15:58:24.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[21480]: audit 2024-03-04T15:58:24.402760+0000 mon.a (mon.0) 2310 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi117-63923-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi117-63923-57"}]: dispatch 2024-03-04T15:58:24.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[21480]: audit 2024-03-04T15:58:24.408283+0000 mon.c (mon.2) 501 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:24.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[21480]: audit 2024-03-04T15:58:24.409624+0000 mon.a (mon.0) 2311 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:24.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[28843]: audit 2024-03-04T15:58:23.397594+0000 mon.c (mon.2) 499 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:24.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[28843]: audit 2024-03-04T15:58:23.403033+0000 mon.a (mon.0) 2303 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:24.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[28843]: audit 2024-03-04T15:58:23.438712+0000 mon.a (mon.0) 2304 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi117-63923-57"}]: dispatch 2024-03-04T15:58:24.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[28843]: audit 2024-03-04T15:58:23.439700+0000 mon.a (mon.0) 2305 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi117-63923-57"}]: dispatch 2024-03-04T15:58:24.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[28843]: audit 2024-03-04T15:58:23.440097+0000 mon.a (mon.0) 2306 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi117-63923-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:24.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[28843]: audit 2024-03-04T15:58:24.385641+0000 mon.a (mon.0) 2307 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:24.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[28843]: audit 2024-03-04T15:58:24.385873+0000 mon.a (mon.0) 2308 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi117-63923-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:24.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[28843]: cluster 2024-03-04T15:58:24.395989+0000 mon.a (mon.0) 2309 : cluster 0 osdmap e315: 8 total, 8 up, 8 in 2024-03-04T15:58:24.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[28843]: audit 2024-03-04T15:58:24.398226+0000 mon.c (mon.2) 500 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi117-64061-6","var": "pg_num","format": "json"}]: dispatch 2024-03-04T15:58:24.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[28843]: audit 2024-03-04T15:58:24.402760+0000 mon.a (mon.0) 2310 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi117-63923-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi117-63923-57"}]: dispatch 2024-03-04T15:58:24.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[28843]: audit 2024-03-04T15:58:24.408283+0000 mon.c (mon.2) 501 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:24.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:24 smithi117 bash[28843]: audit 2024-03-04T15:58:24.409624+0000 mon.a (mon.0) 2311 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:26.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:25 smithi151 bash[23756]: cluster 2024-03-04T15:58:24.920210+0000 mgr.y (mgr.24428) 351 : cluster 0 pgmap v479: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:25 smithi151 bash[23756]: audit 2024-03-04T15:58:25.388818+0000 mon.a (mon.0) 2312 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-43", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:58:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:25 smithi151 bash[23756]: audit 2024-03-04T15:58:25.400838+0000 mon.c (mon.2) 502 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-04T15:58:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:25 smithi151 bash[23756]: cluster 2024-03-04T15:58:25.846081+0000 mon.a (mon.0) 2313 : cluster 0 osdmap e316: 8 total, 8 up, 8 in 2024-03-04T15:58:26.317 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:58:26 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:58:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:58:26.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:25 smithi117 bash[21480]: cluster 2024-03-04T15:58:24.920210+0000 mgr.y (mgr.24428) 351 : cluster 0 pgmap v479: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:26.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:25 smithi117 bash[21480]: audit 2024-03-04T15:58:25.388818+0000 mon.a (mon.0) 2312 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-43", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:58:26.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:25 smithi117 bash[21480]: audit 2024-03-04T15:58:25.400838+0000 mon.c (mon.2) 502 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-04T15:58:26.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:25 smithi117 bash[21480]: cluster 2024-03-04T15:58:25.846081+0000 mon.a (mon.0) 2313 : cluster 0 osdmap e316: 8 total, 8 up, 8 in 2024-03-04T15:58:26.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:25 smithi117 bash[28843]: cluster 2024-03-04T15:58:24.920210+0000 mgr.y (mgr.24428) 351 : cluster 0 pgmap v479: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:26.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:25 smithi117 bash[28843]: audit 2024-03-04T15:58:25.388818+0000 mon.a (mon.0) 2312 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-43", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:58:26.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:25 smithi117 bash[28843]: audit 2024-03-04T15:58:25.400838+0000 mon.c (mon.2) 502 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-04T15:58:26.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:25 smithi117 bash[28843]: cluster 2024-03-04T15:58:25.846081+0000 mon.a (mon.0) 2313 : cluster 0 osdmap e316: 8 total, 8 up, 8 in 2024-03-04T15:58:27.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:58:26 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:58:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:26 smithi151 bash[23756]: audit 2024-03-04T15:58:25.847678+0000 mon.a (mon.0) 2314 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-04T15:58:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:26 smithi151 bash[23756]: audit 2024-03-04T15:58:26.392024+0000 mon.a (mon.0) 2315 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi117-63923-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi117-63923-57"}]': finished 2024-03-04T15:58:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:26 smithi151 bash[23756]: audit 2024-03-04T15:58:26.392302+0000 mon.a (mon.0) 2316 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_count","val": "8"}]': finished 2024-03-04T15:58:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:26 smithi151 bash[23756]: audit 2024-03-04T15:58:26.399486+0000 mon.c (mon.2) 503 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:58:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:26 smithi151 bash[23756]: cluster 2024-03-04T15:58:26.400000+0000 mon.a (mon.0) 2317 : cluster 0 osdmap e317: 8 total, 8 up, 8 in 2024-03-04T15:58:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:26 smithi151 bash[23756]: audit 2024-03-04T15:58:26.406674+0000 mon.a (mon.0) 2318 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:58:27.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:26 smithi117 bash[28843]: audit 2024-03-04T15:58:25.847678+0000 mon.a (mon.0) 2314 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-04T15:58:27.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:26 smithi117 bash[28843]: audit 2024-03-04T15:58:26.392024+0000 mon.a (mon.0) 2315 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi117-63923-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi117-63923-57"}]': finished 2024-03-04T15:58:27.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:26 smithi117 bash[28843]: audit 2024-03-04T15:58:26.392302+0000 mon.a (mon.0) 2316 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_count","val": "8"}]': finished 2024-03-04T15:58:27.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:26 smithi117 bash[28843]: audit 2024-03-04T15:58:26.399486+0000 mon.c (mon.2) 503 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:58:27.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:26 smithi117 bash[28843]: cluster 2024-03-04T15:58:26.400000+0000 mon.a (mon.0) 2317 : cluster 0 osdmap e317: 8 total, 8 up, 8 in 2024-03-04T15:58:27.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:26 smithi117 bash[28843]: audit 2024-03-04T15:58:26.406674+0000 mon.a (mon.0) 2318 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:58:27.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:26 smithi117 bash[21480]: audit 2024-03-04T15:58:25.847678+0000 mon.a (mon.0) 2314 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_count","val": "8"}]: dispatch 2024-03-04T15:58:27.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:26 smithi117 bash[21480]: audit 2024-03-04T15:58:26.392024+0000 mon.a (mon.0) 2315 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi117-63923-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi117-63923-57"}]': finished 2024-03-04T15:58:27.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:26 smithi117 bash[21480]: audit 2024-03-04T15:58:26.392302+0000 mon.a (mon.0) 2316 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_count","val": "8"}]': finished 2024-03-04T15:58:27.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:26 smithi117 bash[21480]: audit 2024-03-04T15:58:26.399486+0000 mon.c (mon.2) 503 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:58:27.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:26 smithi117 bash[21480]: cluster 2024-03-04T15:58:26.400000+0000 mon.a (mon.0) 2317 : cluster 0 osdmap e317: 8 total, 8 up, 8 in 2024-03-04T15:58:27.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:26 smithi117 bash[21480]: audit 2024-03-04T15:58:26.406674+0000 mon.a (mon.0) 2318 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:58:28.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:27 smithi151 bash[23756]: audit 2024-03-04T15:58:26.686472+0000 mgr.y (mgr.24428) 352 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:58:28.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:27 smithi151 bash[23756]: cluster 2024-03-04T15:58:26.921045+0000 mgr.y (mgr.24428) 353 : cluster 0 pgmap v482: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:28.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:27 smithi151 bash[23756]: audit 2024-03-04T15:58:27.394802+0000 mon.a (mon.0) 2319 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_period","val": "600"}]': finished 2024-03-04T15:58:28.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:27 smithi151 bash[23756]: cluster 2024-03-04T15:58:27.402872+0000 mon.a (mon.0) 2320 : cluster 0 osdmap e318: 8 total, 8 up, 8 in 2024-03-04T15:58:28.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:27 smithi151 bash[23756]: audit 2024-03-04T15:58:27.411129+0000 mon.c (mon.2) 504 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-04T15:58:28.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:27 smithi151 bash[23756]: audit 2024-03-04T15:58:27.412226+0000 mon.a (mon.0) 2321 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-04T15:58:28.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:27 smithi117 bash[21480]: audit 2024-03-04T15:58:26.686472+0000 mgr.y (mgr.24428) 352 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:58:28.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:27 smithi117 bash[21480]: cluster 2024-03-04T15:58:26.921045+0000 mgr.y (mgr.24428) 353 : cluster 0 pgmap v482: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:28.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:27 smithi117 bash[21480]: audit 2024-03-04T15:58:27.394802+0000 mon.a (mon.0) 2319 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_period","val": "600"}]': finished 2024-03-04T15:58:28.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:27 smithi117 bash[21480]: cluster 2024-03-04T15:58:27.402872+0000 mon.a (mon.0) 2320 : cluster 0 osdmap e318: 8 total, 8 up, 8 in 2024-03-04T15:58:28.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:27 smithi117 bash[21480]: audit 2024-03-04T15:58:27.411129+0000 mon.c (mon.2) 504 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-04T15:58:28.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:27 smithi117 bash[21480]: audit 2024-03-04T15:58:27.412226+0000 mon.a (mon.0) 2321 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-04T15:58:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:27 smithi117 bash[28843]: audit 2024-03-04T15:58:26.686472+0000 mgr.y (mgr.24428) 352 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:58:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:27 smithi117 bash[28843]: cluster 2024-03-04T15:58:26.921045+0000 mgr.y (mgr.24428) 353 : cluster 0 pgmap v482: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:27 smithi117 bash[28843]: audit 2024-03-04T15:58:27.394802+0000 mon.a (mon.0) 2319 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_period","val": "600"}]': finished 2024-03-04T15:58:28.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:27 smithi117 bash[28843]: cluster 2024-03-04T15:58:27.402872+0000 mon.a (mon.0) 2320 : cluster 0 osdmap e318: 8 total, 8 up, 8 in 2024-03-04T15:58:28.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:27 smithi117 bash[28843]: audit 2024-03-04T15:58:27.411129+0000 mon.c (mon.2) 504 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-04T15:58:28.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:27 smithi117 bash[28843]: audit 2024-03-04T15:58:27.412226+0000 mon.a (mon.0) 2321 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-03-04T15:58:29.019 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-03-04T15:58:29.019 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: hmm, no HitSet yet 2024-03-04T15:58:29.019 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: ok, hit_set contains 275:602f83fe:::foo:head 2024-03-04T15:58:29.019 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9033 ms) 2024-03-04T15:58:29.019 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-03-04T15:58:29.019 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg_num = 32 2024-03-04T15:58:29.020 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 0 ls 1709567909,0 2024-03-04T15:58:29.020 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 1 ls 1709567909,0 2024-03-04T15:58:29.020 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 2 ls 1709567909,0 2024-03-04T15:58:29.020 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 3 ls 1709567909,0 2024-03-04T15:58:29.020 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 4 ls 1709567909,0 2024-03-04T15:58:29.020 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 5 ls 1709567909,0 2024-03-04T15:58:29.020 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 6 ls 1709567909,0 2024-03-04T15:58:29.020 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 7 ls 1709567909,0 2024-03-04T15:58:29.020 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 8 ls 1709567909,0 2024-03-04T15:58:29.020 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 9 ls 1709567909,0 2024-03-04T15:58:29.020 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 10 ls 1709567909,0 2024-03-04T15:58:29.020 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 11 ls 1709567909,0 2024-03-04T15:58:29.020 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 12 ls 1709567909,0 2024-03-04T15:58:29.021 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 13 ls 1709567909,0 2024-03-04T15:58:29.021 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 14 ls 1709567909,0 2024-03-04T15:58:29.021 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 15 ls 1709567909,0 2024-03-04T15:58:29.021 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 16 ls 1709567909,0 2024-03-04T15:58:29.021 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 17 ls 1709567909,0 2024-03-04T15:58:29.021 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 18 ls 1709567909,0 2024-03-04T15:58:29.021 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 19 ls 1709567909,0 2024-03-04T15:58:29.021 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 20 ls 1709567909,0 2024-03-04T15:58:29.021 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 21 ls 1709567909,0 2024-03-04T15:58:29.021 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 22 ls 1709567909,0 2024-03-04T15:58:29.021 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 23 ls 1709567909,0 2024-03-04T15:58:29.021 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 24 ls 1709567909,0 2024-03-04T15:58:29.021 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 25 ls 1709567909,0 2024-03-04T15:58:29.021 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 26 ls 1709567909,0 2024-03-04T15:58:29.021 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 27 ls 1709567909,0 2024-03-04T15:58:29.022 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 28 ls 1709567909,0 2024-03-04T15:58:29.022 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 29 ls 1709567909,0 2024-03-04T15:58:29.022 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 30 ls 1709567909,0 2024-03-04T15:58:29.022 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg 31 ls 1709567909,0 2024-03-04T15:58:29.022 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: pg_num = 32 2024-03-04T15:58:29.022 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:6cac518f:::0:head 2024-03-04T15:58:29.022 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:02547ec2:::1:head 2024-03-04T15:58:29.022 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:f905c69b:::2:head 2024-03-04T15:58:29.023 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:cfc208b3:::3:head 2024-03-04T15:58:29.023 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:d83876eb:::4:head 2024-03-04T15:58:29.023 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:b29083e3:::5:head 2024-03-04T15:58:29.023 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:c4fdafeb:::6:head 2024-03-04T15:58:29.023 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:5c6b0b28:::7:head 2024-03-04T15:58:29.023 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:bd63b0f1:::8:head 2024-03-04T15:58:29.023 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:e960b815:::9:head 2024-03-04T15:58:29.023 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:52ea6a34:::10:head 2024-03-04T15:58:29.023 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:89d3ae78:::11:head 2024-03-04T15:58:29.023 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:de5d7c5f:::12:head 2024-03-04T15:58:29.023 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:566253c9:::13:head 2024-03-04T15:58:29.023 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:62a1935d:::14:head 2024-03-04T15:58:29.023 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:863748b0:::15:head 2024-03-04T15:58:29.024 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:3958e169:::16:head 2024-03-04T15:58:29.024 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:4d4dabf9:::17:head 2024-03-04T15:58:29.024 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:8391935d:::18:head 2024-03-04T15:58:29.024 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:28883081:::19:head 2024-03-04T15:58:29.024 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:69259c59:::20:head 2024-03-04T15:58:29.024 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:4bdb80b7:::21:head 2024-03-04T15:58:29.024 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:a11c5d71:::22:head 2024-03-04T15:58:29.024 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:271af37b:::23:head 2024-03-04T15:58:29.024 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:95b121be:::24:head 2024-03-04T15:58:29.024 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:58d1031b:::25:head 2024-03-04T15:58:29.024 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:0a050783:::26:head 2024-03-04T15:58:29.024 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:c709704c:::27:head 2024-03-04T15:58:29.024 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:cbe56eaf:::28:head 2024-03-04T15:58:29.024 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:86b4b162:::29:head 2024-03-04T15:58:29.025 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:70d89383:::30:head 2024-03-04T15:58:29.025 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:dd450c7c:::31:head 2024-03-04T15:58:29.025 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:6d5729b1:::32:head 2024-03-04T15:58:29.025 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:c388f3fb:::33:head 2024-03-04T15:58:29.025 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:56cfea31:::34:head 2024-03-04T15:58:29.025 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:9dbc1bf7:::35:head 2024-03-04T15:58:29.025 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:40b74ccd:::36:head 2024-03-04T15:58:29.025 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:4d5aaf42:::37:head 2024-03-04T15:58:29.025 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:920f362c:::38:head 2024-03-04T15:58:29.025 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:6cc53222:::39:head 2024-03-04T15:58:29.025 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:9cad833f:::40:head 2024-03-04T15:58:29.025 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:1ea84d41:::41:head 2024-03-04T15:58:29.025 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:c4480ef6:::42:head 2024-03-04T15:58:29.025 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:a694361e:::43:head 2024-03-04T15:58:29.025 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:d1bd33e9:::44:head 2024-03-04T15:58:29.026 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:ddc2cd5d:::45:head 2024-03-04T15:58:29.026 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:2b782207:::46:head 2024-03-04T15:58:29.026 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:7b187fca:::47:head 2024-03-04T15:58:29.026 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:90ecdf6f:::48:head 2024-03-04T15:58:29.026 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:a5ed95fe:::49:head 2024-03-04T15:58:29.026 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:ea0eaa55:::50:head 2024-03-04T15:58:29.026 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:f33ef17b:::51:head 2024-03-04T15:58:29.026 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:a0d1b2f6:::52:head 2024-03-04T15:58:29.026 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:60c5229e:::53:head 2024-03-04T15:58:29.026 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:edcbc575:::54:head 2024-03-04T15:58:29.026 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:102cf253:::55:head 2024-03-04T15:58:29.026 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:efb7fb0b:::56:head 2024-03-04T15:58:29.026 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:50d0a326:::57:head 2024-03-04T15:58:29.026 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:d4dc5daf:::58:head 2024-03-04T15:58:29.026 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:3a130462:::59:head 2024-03-04T15:58:29.027 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:ec87ed71:::60:head 2024-03-04T15:58:29.027 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:d5bc9454:::61:head 2024-03-04T15:58:29.027 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:3ddfe313:::62:head 2024-03-04T15:58:29.027 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:7c2816b9:::63:head 2024-03-04T15:58:29.027 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:47e00e4d:::64:head 2024-03-04T15:58:29.027 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:c6410c18:::65:head 2024-03-04T15:58:29.027 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:b48ed237:::66:head 2024-03-04T15:58:29.027 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:cd63ad31:::67:head 2024-03-04T15:58:29.027 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:b179e92b:::68:head 2024-03-04T15:58:29.027 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:0d9f741a:::69:head 2024-03-04T15:58:29.027 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:6d3352ae:::70:head 2024-03-04T15:58:29.027 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:c6d5c19e:::71:head 2024-03-04T15:58:29.027 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:bc4729c3:::72:head 2024-03-04T15:58:29.027 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:77e930b9:::73:head 2024-03-04T15:58:29.027 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:0abeecfd:::74:head 2024-03-04T15:58:29.028 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:b7c37e15:::75:head 2024-03-04T15:58:29.028 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:b6378398:::76:head 2024-03-04T15:58:29.028 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:02bd68de:::77:head 2024-03-04T15:58:29.028 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:cc795d2d:::78:head 2024-03-04T15:58:29.028 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:630d4fea:::79:head 2024-03-04T15:58:29.028 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:e0d29ef5:::80:head 2024-03-04T15:58:29.028 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:fd6f13d2:::81:head 2024-03-04T15:58:29.028 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:606461d5:::82:head 2024-03-04T15:58:29.028 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:eadbdc43:::83:head 2024-03-04T15:58:29.028 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:8761d0bb:::84:head 2024-03-04T15:58:29.028 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:9ef0186f:::85:head 2024-03-04T15:58:29.028 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:e0d41294:::86:head 2024-03-04T15:58:29.028 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:961de695:::87:head 2024-03-04T15:58:29.028 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:1423148f:::88:head 2024-03-04T15:58:29.028 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:633a8fa2:::89:head 2024-03-04T15:58:29.029 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:a8653809:::90:head 2024-03-04T15:58:29.029 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:3dac8b33:::91:head 2024-03-04T15:58:29.029 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:35aad435:::92:head 2024-03-04T15:58:29.029 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:f6dcc343:::93:head 2024-03-04T15:58:29.029 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:dbbdad87:::94:head 2024-03-04T15:58:29.029 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:1cb48ce0:::95:head 2024-03-04T15:58:29.029 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:03cd461c:::96:head 2024-03-04T15:58:29.029 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:17a4ea99:::97:head 2024-03-04T15:58:29.029 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:9993c9a7:::98:head 2024-03-04T15:58:29.030 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:6394211c:::99:head 2024-03-04T15:58:29.030 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:94c7ae57:::100:head 2024-03-04T15:58:29.030 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:6fdee5bb:::101:head 2024-03-04T15:58:29.030 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:9a477fd1:::102:head 2024-03-04T15:58:29.030 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:eb850916:::103:head 2024-03-04T15:58:29.030 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:affc56b9:::104:head 2024-03-04T15:58:29.030 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:b42dc814:::105:head 2024-03-04T15:58:29.030 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:f319f8f0:::106:head 2024-03-04T15:58:29.030 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:9a40b9de:::107:head 2024-03-04T15:58:29.030 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:8b524f28:::108:head 2024-03-04T15:58:29.030 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:e3de589f:::109:head 2024-03-04T15:58:29.030 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:90f90a5b:::110:head 2024-03-04T15:58:29.030 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:a7b4f1d7:::111:head 2024-03-04T15:58:29.031 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:af51766e:::112:head 2024-03-04T15:58:29.031 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:b6f90bd1:::113:head 2024-03-04T15:58:29.031 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:e0261208:::114:head 2024-03-04T15:58:29.031 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:c9569ef7:::115:head 2024-03-04T15:58:29.031 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:61bebe50:::116:head 2024-03-04T15:58:29.031 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:fe93412b:::117:head 2024-03-04T15:58:29.031 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:d3d38bee:::118:head 2024-03-04T15:58:29.031 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:3100ba0c:::119:head 2024-03-04T15:58:29.031 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:d0560ada:::120:head 2024-03-04T15:58:29.031 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:f0ea8b35:::121:head 2024-03-04T15:58:29.031 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:766f231a:::122:head 2024-03-04T15:58:29.031 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:a07a2582:::123:head 2024-03-04T15:58:29.031 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:bd7c6b3a:::124:head 2024-03-04T15:58:29.031 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:fb2ddaff:::125:head 2024-03-04T15:58:29.032 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:4408e1fe:::126:head 2024-03-04T15:58:29.032 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:ee1df7a7:::127:head 2024-03-04T15:58:29.032 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:c3002909:::128:head 2024-03-04T15:58:29.032 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:4f48ffa9:::129:head 2024-03-04T15:58:29.032 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:edf38733:::130:head 2024-03-04T15:58:29.032 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:c08425c0:::131:head 2024-03-04T15:58:29.032 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:5f902d98:::132:head 2024-03-04T15:58:29.032 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:41ea2c93:::133:head 2024-03-04T15:58:29.032 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:813cee13:::134:head 2024-03-04T15:58:29.032 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:0131818d:::135:head 2024-03-04T15:58:29.032 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:26ba5a85:::136:head 2024-03-04T15:58:29.032 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:381b8a5a:::137:head 2024-03-04T15:58:29.032 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:28797e47:::138:head 2024-03-04T15:58:29.032 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:bfca7f22:::139:head 2024-03-04T15:58:29.032 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:36807075:::140:head 2024-03-04T15:58:29.033 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:80b03975:::141:head 2024-03-04T15:58:29.033 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:5c15709b:::142:head 2024-03-04T15:58:29.033 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:f39ea15e:::143:head 2024-03-04T15:58:29.033 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:ea992956:::144:head 2024-03-04T15:58:29.033 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:48887b1c:::145:head 2024-03-04T15:58:29.033 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:9f24a9dd:::146:head 2024-03-04T15:58:29.033 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:987f100b:::147:head 2024-03-04T15:58:29.034 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:d2dd3581:::148:head 2024-03-04T15:58:29.034 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:7fed1808:::149:head 2024-03-04T15:58:29.034 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:c80b70e9:::150:head 2024-03-04T15:58:29.034 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:85ed90f9:::151:head 2024-03-04T15:58:29.034 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:36428b24:::152:head 2024-03-04T15:58:29.034 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:d044c34a:::153:head 2024-03-04T15:58:29.034 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:7c18bf58:::154:head 2024-03-04T15:58:29.034 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:d1c21232:::155:head 2024-03-04T15:58:29.034 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:a7a3c575:::156:head 2024-03-04T15:58:29.034 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:87da0633:::157:head 2024-03-04T15:58:29.034 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:d5ac3822:::158:head 2024-03-04T15:58:29.034 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:3f20522d:::159:head 2024-03-04T15:58:29.034 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:6ca26563:::160:head 2024-03-04T15:58:29.034 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:532ce135:::161:head 2024-03-04T15:58:29.034 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:c78863e6:::162:head 2024-03-04T15:58:29.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:28 smithi151 bash[23756]: cluster 2024-03-04T15:58:27.850709+0000 mon.a (mon.0) 2322 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:28 smithi151 bash[23756]: audit 2024-03-04T15:58:28.397494+0000 mon.a (mon.0) 2323 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-03-04T15:58:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:28 smithi151 bash[23756]: cluster 2024-03-04T15:58:28.404607+0000 mon.a (mon.0) 2324 : cluster 0 osdmap e319: 8 total, 8 up, 8 in 2024-03-04T15:58:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:28 smithi151 bash[23756]: audit 2024-03-04T15:58:28.409028+0000 mon.a (mon.0) 2325 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi117-63923-57"}]: dispatch 2024-03-04T15:58:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:28 smithi151 bash[23756]: audit 2024-03-04T15:58:28.794290+0000 mon.c (mon.2) 505 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:58:29.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:28 smithi117 bash[21480]: cluster 2024-03-04T15:58:27.850709+0000 mon.a (mon.0) 2322 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:29.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:28 smithi117 bash[21480]: audit 2024-03-04T15:58:28.397494+0000 mon.a (mon.0) 2323 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-03-04T15:58:29.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:28 smithi117 bash[21480]: cluster 2024-03-04T15:58:28.404607+0000 mon.a (mon.0) 2324 : cluster 0 osdmap e319: 8 total, 8 up, 8 in 2024-03-04T15:58:29.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:28 smithi117 bash[21480]: audit 2024-03-04T15:58:28.409028+0000 mon.a (mon.0) 2325 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi117-63923-57"}]: dispatch 2024-03-04T15:58:29.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:28 smithi117 bash[21480]: audit 2024-03-04T15:58:28.794290+0000 mon.c (mon.2) 505 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:58:29.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:28 smithi117 bash[28843]: cluster 2024-03-04T15:58:27.850709+0000 mon.a (mon.0) 2322 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:29.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:28 smithi117 bash[28843]: audit 2024-03-04T15:58:28.397494+0000 mon.a (mon.0) 2323 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-03-04T15:58:29.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:28 smithi117 bash[28843]: cluster 2024-03-04T15:58:28.404607+0000 mon.a (mon.0) 2324 : cluster 0 osdmap e319: 8 total, 8 up, 8 in 2024-03-04T15:58:29.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:28 smithi117 bash[28843]: audit 2024-03-04T15:58:28.409028+0000 mon.a (mon.0) 2325 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi117-63923-57"}]: dispatch 2024-03-04T15:58:29.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:28 smithi117 bash[28843]: audit 2024-03-04T15:58:28.794290+0000 mon.c (mon.2) 505 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:58:30.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:29 smithi151 bash[23756]: cluster 2024-03-04T15:58:28.921908+0000 mgr.y (mgr.24428) 354 : cluster 0 pgmap v485: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:30.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:29 smithi151 bash[23756]: audit 2024-03-04T15:58:29.022351+0000 mon.c (mon.2) 506 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi117-64061-43","var": "pg_num","format": "json"}]: dispatch 2024-03-04T15:58:30.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:29 smithi151 bash[23756]: audit 2024-03-04T15:58:29.339863+0000 mon.c (mon.2) 507 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:30.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:29 smithi151 bash[23756]: audit 2024-03-04T15:58:29.340567+0000 mon.a (mon.0) 2326 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:30.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:29 smithi151 bash[23756]: audit 2024-03-04T15:58:29.341834+0000 mon.c (mon.2) 508 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-43"}]: dispatch 2024-03-04T15:58:30.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:29 smithi151 bash[23756]: audit 2024-03-04T15:58:29.342491+0000 mon.a (mon.0) 2327 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-43"}]: dispatch 2024-03-04T15:58:30.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:29 smithi151 bash[23756]: audit 2024-03-04T15:58:29.400086+0000 mon.a (mon.0) 2328 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi117-63923-57"}]': finished 2024-03-04T15:58:30.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:29 smithi151 bash[23756]: audit 2024-03-04T15:58:29.400241+0000 mon.a (mon.0) 2329 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-43"}]': finished 2024-03-04T15:58:30.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:29 smithi151 bash[23756]: cluster 2024-03-04T15:58:29.409842+0000 mon.a (mon.0) 2330 : cluster 0 osdmap e320: 8 total, 8 up, 8 in 2024-03-04T15:58:30.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:29 smithi151 bash[23756]: audit 2024-03-04T15:58:29.412469+0000 mon.a (mon.0) 2331 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi117-63923-57"}]: dispatch 2024-03-04T15:58:30.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:29 smithi117 bash[21480]: cluster 2024-03-04T15:58:28.921908+0000 mgr.y (mgr.24428) 354 : cluster 0 pgmap v485: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:30.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:29 smithi117 bash[21480]: audit 2024-03-04T15:58:29.022351+0000 mon.c (mon.2) 506 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi117-64061-43","var": "pg_num","format": "json"}]: dispatch 2024-03-04T15:58:30.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:29 smithi117 bash[21480]: audit 2024-03-04T15:58:29.339863+0000 mon.c (mon.2) 507 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:30.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:29 smithi117 bash[21480]: audit 2024-03-04T15:58:29.340567+0000 mon.a (mon.0) 2326 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:30.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:29 smithi117 bash[21480]: audit 2024-03-04T15:58:29.341834+0000 mon.c (mon.2) 508 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-43"}]: dispatch 2024-03-04T15:58:30.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:29 smithi117 bash[21480]: audit 2024-03-04T15:58:29.342491+0000 mon.a (mon.0) 2327 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-43"}]: dispatch 2024-03-04T15:58:30.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:29 smithi117 bash[21480]: audit 2024-03-04T15:58:29.400086+0000 mon.a (mon.0) 2328 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi117-63923-57"}]': finished 2024-03-04T15:58:30.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:29 smithi117 bash[21480]: audit 2024-03-04T15:58:29.400241+0000 mon.a (mon.0) 2329 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-43"}]': finished 2024-03-04T15:58:30.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:29 smithi117 bash[21480]: cluster 2024-03-04T15:58:29.409842+0000 mon.a (mon.0) 2330 : cluster 0 osdmap e320: 8 total, 8 up, 8 in 2024-03-04T15:58:30.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:29 smithi117 bash[21480]: audit 2024-03-04T15:58:29.412469+0000 mon.a (mon.0) 2331 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi117-63923-57"}]: dispatch 2024-03-04T15:58:30.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:29 smithi117 bash[28843]: cluster 2024-03-04T15:58:28.921908+0000 mgr.y (mgr.24428) 354 : cluster 0 pgmap v485: 228 pgs: 228 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:30.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:29 smithi117 bash[28843]: audit 2024-03-04T15:58:29.022351+0000 mon.c (mon.2) 506 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi117-64061-43","var": "pg_num","format": "json"}]: dispatch 2024-03-04T15:58:30.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:29 smithi117 bash[28843]: audit 2024-03-04T15:58:29.339863+0000 mon.c (mon.2) 507 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:30.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:29 smithi117 bash[28843]: audit 2024-03-04T15:58:29.340567+0000 mon.a (mon.0) 2326 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:30.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:29 smithi117 bash[28843]: audit 2024-03-04T15:58:29.341834+0000 mon.c (mon.2) 508 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-43"}]: dispatch 2024-03-04T15:58:30.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:29 smithi117 bash[28843]: audit 2024-03-04T15:58:29.342491+0000 mon.a (mon.0) 2327 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-43"}]: dispatch 2024-03-04T15:58:30.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:29 smithi117 bash[28843]: audit 2024-03-04T15:58:29.400086+0000 mon.a (mon.0) 2328 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi117-63923-57"}]': finished 2024-03-04T15:58:30.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:29 smithi117 bash[28843]: audit 2024-03-04T15:58:29.400241+0000 mon.a (mon.0) 2329 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-43"}]': finished 2024-03-04T15:58:30.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:29 smithi117 bash[28843]: cluster 2024-03-04T15:58:29.409842+0000 mon.a (mon.0) 2330 : cluster 0 osdmap e320: 8 total, 8 up, 8 in 2024-03-04T15:58:30.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:29 smithi117 bash[28843]: audit 2024-03-04T15:58:29.412469+0000 mon.a (mon.0) 2331 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi117-63923-57"}]: dispatch 2024-03-04T15:58:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:31 smithi151 bash[23756]: audit 2024-03-04T15:58:30.403457+0000 mon.a (mon.0) 2332 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi117-63923-57"}]': finished 2024-03-04T15:58:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:31 smithi151 bash[23756]: cluster 2024-03-04T15:58:30.411057+0000 mon.a (mon.0) 2333 : cluster 0 osdmap e321: 8 total, 8 up, 8 in 2024-03-04T15:58:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:31 smithi151 bash[23756]: audit 2024-03-04T15:58:30.460410+0000 mon.c (mon.2) 509 : audit 1 from='client.? 172.21.15.117:0/973126100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:31 smithi151 bash[23756]: audit 2024-03-04T15:58:30.864080+0000 mon.a (mon.0) 2334 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:31 smithi151 bash[23756]: audit 2024-03-04T15:58:30.865077+0000 mon.c (mon.2) 510 : audit 1 from='client.? 172.21.15.117:0/973126100' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:31 smithi151 bash[23756]: audit 2024-03-04T15:58:30.865478+0000 mon.a (mon.0) 2335 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:31 smithi151 bash[23756]: audit 2024-03-04T15:58:30.866164+0000 mon.c (mon.2) 511 : audit 1 from='client.? 172.21.15.117:0/973126100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi117-63923-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:31 smithi151 bash[23756]: audit 2024-03-04T15:58:30.866571+0000 mon.a (mon.0) 2336 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi117-63923-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:31 smithi151 bash[23756]: cluster 2024-03-04T15:58:30.922735+0000 mgr.y (mgr.24428) 355 : cluster 0 pgmap v488: 196 pgs: 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:31.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:31 smithi117 bash[21480]: audit 2024-03-04T15:58:30.403457+0000 mon.a (mon.0) 2332 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi117-63923-57"}]': finished 2024-03-04T15:58:31.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:31 smithi117 bash[21480]: cluster 2024-03-04T15:58:30.411057+0000 mon.a (mon.0) 2333 : cluster 0 osdmap e321: 8 total, 8 up, 8 in 2024-03-04T15:58:31.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:31 smithi117 bash[21480]: audit 2024-03-04T15:58:30.460410+0000 mon.c (mon.2) 509 : audit 1 from='client.? 172.21.15.117:0/973126100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:31.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:31 smithi117 bash[21480]: audit 2024-03-04T15:58:30.864080+0000 mon.a (mon.0) 2334 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:31.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:31 smithi117 bash[21480]: audit 2024-03-04T15:58:30.865077+0000 mon.c (mon.2) 510 : audit 1 from='client.? 172.21.15.117:0/973126100' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:31.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:31 smithi117 bash[21480]: audit 2024-03-04T15:58:30.865478+0000 mon.a (mon.0) 2335 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:31.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:31 smithi117 bash[21480]: audit 2024-03-04T15:58:30.866164+0000 mon.c (mon.2) 511 : audit 1 from='client.? 172.21.15.117:0/973126100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi117-63923-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:31.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:31 smithi117 bash[21480]: audit 2024-03-04T15:58:30.866571+0000 mon.a (mon.0) 2336 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi117-63923-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:31.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:31 smithi117 bash[21480]: cluster 2024-03-04T15:58:30.922735+0000 mgr.y (mgr.24428) 355 : cluster 0 pgmap v488: 196 pgs: 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:31 smithi117 bash[28843]: audit 2024-03-04T15:58:30.403457+0000 mon.a (mon.0) 2332 : audit 1 from='client.? 172.21.15.117:0/2863041524' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi117-63923-57"}]': finished 2024-03-04T15:58:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:31 smithi117 bash[28843]: cluster 2024-03-04T15:58:30.411057+0000 mon.a (mon.0) 2333 : cluster 0 osdmap e321: 8 total, 8 up, 8 in 2024-03-04T15:58:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:31 smithi117 bash[28843]: audit 2024-03-04T15:58:30.460410+0000 mon.c (mon.2) 509 : audit 1 from='client.? 172.21.15.117:0/973126100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:31 smithi117 bash[28843]: audit 2024-03-04T15:58:30.864080+0000 mon.a (mon.0) 2334 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:31 smithi117 bash[28843]: audit 2024-03-04T15:58:30.865077+0000 mon.c (mon.2) 510 : audit 1 from='client.? 172.21.15.117:0/973126100' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:31 smithi117 bash[28843]: audit 2024-03-04T15:58:30.865478+0000 mon.a (mon.0) 2335 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:31 smithi117 bash[28843]: audit 2024-03-04T15:58:30.866164+0000 mon.c (mon.2) 511 : audit 1 from='client.? 172.21.15.117:0/973126100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi117-63923-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:31 smithi117 bash[28843]: audit 2024-03-04T15:58:30.866571+0000 mon.a (mon.0) 2336 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi117-63923-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:31 smithi117 bash[28843]: cluster 2024-03-04T15:58:30.922735+0000 mgr.y (mgr.24428) 355 : cluster 0 pgmap v488: 196 pgs: 196 active+clean; 8.4 MiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:32 smithi151 bash[23756]: audit 2024-03-04T15:58:31.415134+0000 mon.a (mon.0) 2337 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi117-63923-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:32 smithi151 bash[23756]: cluster 2024-03-04T15:58:31.427094+0000 mon.a (mon.0) 2338 : cluster 0 osdmap e322: 8 total, 8 up, 8 in 2024-03-04T15:58:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:32 smithi151 bash[23756]: audit 2024-03-04T15:58:31.429005+0000 mon.c (mon.2) 512 : audit 1 from='client.? 172.21.15.117:0/973126100' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi117-63923-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:32 smithi151 bash[23756]: audit 2024-03-04T15:58:31.444093+0000 mon.c (mon.2) 513 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:32 smithi151 bash[23756]: audit 2024-03-04T15:58:31.448034+0000 mon.a (mon.0) 2339 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi117-63923-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:32 smithi151 bash[23756]: audit 2024-03-04T15:58:31.449712+0000 mon.a (mon.0) 2340 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:32.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:32 smithi117 bash[21480]: audit 2024-03-04T15:58:31.415134+0000 mon.a (mon.0) 2337 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi117-63923-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:32.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:32 smithi117 bash[21480]: cluster 2024-03-04T15:58:31.427094+0000 mon.a (mon.0) 2338 : cluster 0 osdmap e322: 8 total, 8 up, 8 in 2024-03-04T15:58:32.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:32 smithi117 bash[21480]: audit 2024-03-04T15:58:31.429005+0000 mon.c (mon.2) 512 : audit 1 from='client.? 172.21.15.117:0/973126100' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi117-63923-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:32.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:32 smithi117 bash[21480]: audit 2024-03-04T15:58:31.444093+0000 mon.c (mon.2) 513 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:32.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:32 smithi117 bash[21480]: audit 2024-03-04T15:58:31.448034+0000 mon.a (mon.0) 2339 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi117-63923-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:32.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:32 smithi117 bash[21480]: audit 2024-03-04T15:58:31.449712+0000 mon.a (mon.0) 2340 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:32.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:32 smithi117 bash[28843]: audit 2024-03-04T15:58:31.415134+0000 mon.a (mon.0) 2337 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi117-63923-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:32.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:32 smithi117 bash[28843]: cluster 2024-03-04T15:58:31.427094+0000 mon.a (mon.0) 2338 : cluster 0 osdmap e322: 8 total, 8 up, 8 in 2024-03-04T15:58:32.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:32 smithi117 bash[28843]: audit 2024-03-04T15:58:31.429005+0000 mon.c (mon.2) 512 : audit 1 from='client.? 172.21.15.117:0/973126100' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi117-63923-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:32.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:32 smithi117 bash[28843]: audit 2024-03-04T15:58:31.444093+0000 mon.c (mon.2) 513 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:32.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:32 smithi117 bash[28843]: audit 2024-03-04T15:58:31.448034+0000 mon.a (mon.0) 2339 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi117-63923-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:32.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:32 smithi117 bash[28843]: audit 2024-03-04T15:58:31.449712+0000 mon.a (mon.0) 2340 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:33.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:33 smithi151 bash[23756]: audit 2024-03-04T15:58:32.429618+0000 mon.a (mon.0) 2341 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:33.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:33 smithi151 bash[23756]: audit 2024-03-04T15:58:32.446534+0000 mon.c (mon.2) 514 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:33.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:33 smithi151 bash[23756]: cluster 2024-03-04T15:58:32.891566+0000 mon.a (mon.0) 2342 : cluster 0 osdmap e323: 8 total, 8 up, 8 in 2024-03-04T15:58:33.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:33 smithi151 bash[23756]: audit 2024-03-04T15:58:32.892400+0000 mon.a (mon.0) 2343 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:33.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:33 smithi151 bash[23756]: cluster 2024-03-04T15:58:32.923495+0000 mgr.y (mgr.24428) 356 : cluster 0 pgmap v491: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:33.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:33 smithi151 bash[23756]: cluster 2024-03-04T15:58:33.252228+0000 mon.a (mon.0) 2344 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:33.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:33 smithi117 bash[21480]: audit 2024-03-04T15:58:32.429618+0000 mon.a (mon.0) 2341 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:33.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:33 smithi117 bash[21480]: audit 2024-03-04T15:58:32.446534+0000 mon.c (mon.2) 514 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:33.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:33 smithi117 bash[21480]: cluster 2024-03-04T15:58:32.891566+0000 mon.a (mon.0) 2342 : cluster 0 osdmap e323: 8 total, 8 up, 8 in 2024-03-04T15:58:33.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:33 smithi117 bash[21480]: audit 2024-03-04T15:58:32.892400+0000 mon.a (mon.0) 2343 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:33.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:33 smithi117 bash[21480]: cluster 2024-03-04T15:58:32.923495+0000 mgr.y (mgr.24428) 356 : cluster 0 pgmap v491: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:33.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:33 smithi117 bash[21480]: cluster 2024-03-04T15:58:33.252228+0000 mon.a (mon.0) 2344 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:33.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:33 smithi117 bash[28843]: audit 2024-03-04T15:58:32.429618+0000 mon.a (mon.0) 2341 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:33.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:33 smithi117 bash[28843]: audit 2024-03-04T15:58:32.446534+0000 mon.c (mon.2) 514 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:33.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:33 smithi117 bash[28843]: cluster 2024-03-04T15:58:32.891566+0000 mon.a (mon.0) 2342 : cluster 0 osdmap e323: 8 total, 8 up, 8 in 2024-03-04T15:58:33.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:33 smithi117 bash[28843]: audit 2024-03-04T15:58:32.892400+0000 mon.a (mon.0) 2343 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:33.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:33 smithi117 bash[28843]: cluster 2024-03-04T15:58:32.923495+0000 mgr.y (mgr.24428) 356 : cluster 0 pgmap v491: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:33.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:33 smithi117 bash[28843]: cluster 2024-03-04T15:58:33.252228+0000 mon.a (mon.0) 2344 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:34.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:34 smithi117 bash[21480]: audit 2024-03-04T15:58:33.444625+0000 mon.a (mon.0) 2345 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi117-63923-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi117-63923-58"}]': finished 2024-03-04T15:58:34.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:34 smithi117 bash[21480]: audit 2024-03-04T15:58:33.444863+0000 mon.a (mon.0) 2346 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-45", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:58:34.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:34 smithi117 bash[21480]: cluster 2024-03-04T15:58:33.451465+0000 mon.a (mon.0) 2347 : cluster 0 osdmap e324: 8 total, 8 up, 8 in 2024-03-04T15:58:34.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:34 smithi117 bash[21480]: audit 2024-03-04T15:58:33.456596+0000 mon.c (mon.2) 515 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-04T15:58:34.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:34 smithi117 bash[21480]: audit 2024-03-04T15:58:33.457742+0000 mon.a (mon.0) 2348 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-04T15:58:34.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:34 smithi117 bash[28843]: audit 2024-03-04T15:58:33.444625+0000 mon.a (mon.0) 2345 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi117-63923-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi117-63923-58"}]': finished 2024-03-04T15:58:34.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:34 smithi117 bash[28843]: audit 2024-03-04T15:58:33.444863+0000 mon.a (mon.0) 2346 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-45", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:58:34.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:34 smithi117 bash[28843]: cluster 2024-03-04T15:58:33.451465+0000 mon.a (mon.0) 2347 : cluster 0 osdmap e324: 8 total, 8 up, 8 in 2024-03-04T15:58:34.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:34 smithi117 bash[28843]: audit 2024-03-04T15:58:33.456596+0000 mon.c (mon.2) 515 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-04T15:58:34.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:34 smithi117 bash[28843]: audit 2024-03-04T15:58:33.457742+0000 mon.a (mon.0) 2348 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-04T15:58:34.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:34 smithi151 bash[23756]: audit 2024-03-04T15:58:33.444625+0000 mon.a (mon.0) 2345 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi117-63923-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi117-63923-58"}]': finished 2024-03-04T15:58:34.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:34 smithi151 bash[23756]: audit 2024-03-04T15:58:33.444863+0000 mon.a (mon.0) 2346 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-45", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:58:34.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:34 smithi151 bash[23756]: cluster 2024-03-04T15:58:33.451465+0000 mon.a (mon.0) 2347 : cluster 0 osdmap e324: 8 total, 8 up, 8 in 2024-03-04T15:58:34.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:34 smithi151 bash[23756]: audit 2024-03-04T15:58:33.456596+0000 mon.c (mon.2) 515 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-04T15:58:34.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:34 smithi151 bash[23756]: audit 2024-03-04T15:58:33.457742+0000 mon.a (mon.0) 2348 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_count","val": "3"}]: dispatch 2024-03-04T15:58:35.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[21480]: audit 2024-03-04T15:58:34.449025+0000 mon.a (mon.0) 2349 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_count","val": "3"}]': finished 2024-03-04T15:58:35.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[21480]: cluster 2024-03-04T15:58:34.457638+0000 mon.a (mon.0) 2350 : cluster 0 osdmap e325: 8 total, 8 up, 8 in 2024-03-04T15:58:35.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[21480]: audit 2024-03-04T15:58:34.465002+0000 mon.c (mon.2) 516 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-04T15:58:35.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[21480]: audit 2024-03-04T15:58:34.467836+0000 mon.a (mon.0) 2351 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-04T15:58:35.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[21480]: cluster 2024-03-04T15:58:34.924354+0000 mgr.y (mgr.24428) 357 : cluster 0 pgmap v494: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:35.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[21480]: audit 2024-03-04T15:58:35.452047+0000 mon.a (mon.0) 2352 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_period","val": "3"}]': finished 2024-03-04T15:58:35.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[21480]: cluster 2024-03-04T15:58:35.460233+0000 mon.a (mon.0) 2353 : cluster 0 osdmap e326: 8 total, 8 up, 8 in 2024-03-04T15:58:35.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[21480]: audit 2024-03-04T15:58:35.462857+0000 mon.c (mon.2) 517 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T15:58:35.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[21480]: audit 2024-03-04T15:58:35.466415+0000 mon.c (mon.2) 518 : audit 1 from='client.? 172.21.15.117:0/973126100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:35.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[21480]: audit 2024-03-04T15:58:35.467399+0000 mon.a (mon.0) 2354 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T15:58:35.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[21480]: audit 2024-03-04T15:58:35.467917+0000 mon.a (mon.0) 2355 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:35.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[28843]: audit 2024-03-04T15:58:34.449025+0000 mon.a (mon.0) 2349 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_count","val": "3"}]': finished 2024-03-04T15:58:35.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[28843]: cluster 2024-03-04T15:58:34.457638+0000 mon.a (mon.0) 2350 : cluster 0 osdmap e325: 8 total, 8 up, 8 in 2024-03-04T15:58:35.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[28843]: audit 2024-03-04T15:58:34.465002+0000 mon.c (mon.2) 516 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-04T15:58:35.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[28843]: audit 2024-03-04T15:58:34.467836+0000 mon.a (mon.0) 2351 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-04T15:58:35.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[28843]: cluster 2024-03-04T15:58:34.924354+0000 mgr.y (mgr.24428) 357 : cluster 0 pgmap v494: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:35.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[28843]: audit 2024-03-04T15:58:35.452047+0000 mon.a (mon.0) 2352 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_period","val": "3"}]': finished 2024-03-04T15:58:35.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[28843]: cluster 2024-03-04T15:58:35.460233+0000 mon.a (mon.0) 2353 : cluster 0 osdmap e326: 8 total, 8 up, 8 in 2024-03-04T15:58:35.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[28843]: audit 2024-03-04T15:58:35.462857+0000 mon.c (mon.2) 517 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T15:58:35.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[28843]: audit 2024-03-04T15:58:35.466415+0000 mon.c (mon.2) 518 : audit 1 from='client.? 172.21.15.117:0/973126100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:35.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[28843]: audit 2024-03-04T15:58:35.467399+0000 mon.a (mon.0) 2354 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T15:58:35.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:35 smithi117 bash[28843]: audit 2024-03-04T15:58:35.467917+0000 mon.a (mon.0) 2355 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:35.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:35 smithi151 bash[23756]: audit 2024-03-04T15:58:34.449025+0000 mon.a (mon.0) 2349 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_count","val": "3"}]': finished 2024-03-04T15:58:35.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:35 smithi151 bash[23756]: cluster 2024-03-04T15:58:34.457638+0000 mon.a (mon.0) 2350 : cluster 0 osdmap e325: 8 total, 8 up, 8 in 2024-03-04T15:58:35.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:35 smithi151 bash[23756]: audit 2024-03-04T15:58:34.465002+0000 mon.c (mon.2) 516 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-04T15:58:35.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:35 smithi151 bash[23756]: audit 2024-03-04T15:58:34.467836+0000 mon.a (mon.0) 2351 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_period","val": "3"}]: dispatch 2024-03-04T15:58:35.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:35 smithi151 bash[23756]: cluster 2024-03-04T15:58:34.924354+0000 mgr.y (mgr.24428) 357 : cluster 0 pgmap v494: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:35.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:35 smithi151 bash[23756]: audit 2024-03-04T15:58:35.452047+0000 mon.a (mon.0) 2352 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_period","val": "3"}]': finished 2024-03-04T15:58:35.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:35 smithi151 bash[23756]: cluster 2024-03-04T15:58:35.460233+0000 mon.a (mon.0) 2353 : cluster 0 osdmap e326: 8 total, 8 up, 8 in 2024-03-04T15:58:35.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:35 smithi151 bash[23756]: audit 2024-03-04T15:58:35.462857+0000 mon.c (mon.2) 517 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T15:58:35.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:35 smithi151 bash[23756]: audit 2024-03-04T15:58:35.466415+0000 mon.c (mon.2) 518 : audit 1 from='client.? 172.21.15.117:0/973126100' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:35.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:35 smithi151 bash[23756]: audit 2024-03-04T15:58:35.467399+0000 mon.a (mon.0) 2354 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T15:58:35.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:35 smithi151 bash[23756]: audit 2024-03-04T15:58:35.467917+0000 mon.a (mon.0) 2355 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:36.567 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:58:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:58:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:58:37.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:58:36 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:58:37.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:37 smithi117 bash[28843]: audit 2024-03-04T15:58:36.454739+0000 mon.a (mon.0) 2356 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T15:58:37.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:37 smithi117 bash[28843]: audit 2024-03-04T15:58:36.455000+0000 mon.a (mon.0) 2357 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi117-63923-58"}]': finished 2024-03-04T15:58:37.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:37 smithi117 bash[28843]: cluster 2024-03-04T15:58:36.462756+0000 mon.a (mon.0) 2358 : cluster 0 osdmap e327: 8 total, 8 up, 8 in 2024-03-04T15:58:37.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:37 smithi117 bash[28843]: audit 2024-03-04T15:58:36.469722+0000 mon.c (mon.2) 519 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-04T15:58:37.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:37 smithi117 bash[28843]: audit 2024-03-04T15:58:36.470427+0000 mon.c (mon.2) 520 : audit 1 from='client.? 172.21.15.117:0/973126100' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:37.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:37 smithi117 bash[28843]: audit 2024-03-04T15:58:36.470954+0000 mon.a (mon.0) 2359 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-04T15:58:37.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:37 smithi117 bash[28843]: audit 2024-03-04T15:58:36.471916+0000 mon.a (mon.0) 2360 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:37.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:37 smithi117 bash[28843]: audit 2024-03-04T15:58:36.697961+0000 mgr.y (mgr.24428) 358 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:58:37.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:37 smithi117 bash[28843]: cluster 2024-03-04T15:58:36.925203+0000 mgr.y (mgr.24428) 359 : cluster 0 pgmap v497: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:37.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:37 smithi117 bash[21480]: audit 2024-03-04T15:58:36.454739+0000 mon.a (mon.0) 2356 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T15:58:37.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:37 smithi117 bash[21480]: audit 2024-03-04T15:58:36.455000+0000 mon.a (mon.0) 2357 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi117-63923-58"}]': finished 2024-03-04T15:58:37.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:37 smithi117 bash[21480]: cluster 2024-03-04T15:58:36.462756+0000 mon.a (mon.0) 2358 : cluster 0 osdmap e327: 8 total, 8 up, 8 in 2024-03-04T15:58:37.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:37 smithi117 bash[21480]: audit 2024-03-04T15:58:36.469722+0000 mon.c (mon.2) 519 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-04T15:58:37.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:37 smithi117 bash[21480]: audit 2024-03-04T15:58:36.470427+0000 mon.c (mon.2) 520 : audit 1 from='client.? 172.21.15.117:0/973126100' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:37.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:37 smithi117 bash[21480]: audit 2024-03-04T15:58:36.470954+0000 mon.a (mon.0) 2359 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-04T15:58:37.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:37 smithi117 bash[21480]: audit 2024-03-04T15:58:36.471916+0000 mon.a (mon.0) 2360 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:37.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:37 smithi117 bash[21480]: audit 2024-03-04T15:58:36.697961+0000 mgr.y (mgr.24428) 358 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:58:37.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:37 smithi117 bash[21480]: cluster 2024-03-04T15:58:36.925203+0000 mgr.y (mgr.24428) 359 : cluster 0 pgmap v497: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:37.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:37 smithi151 bash[23756]: audit 2024-03-04T15:58:36.454739+0000 mon.a (mon.0) 2356 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T15:58:37.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:37 smithi151 bash[23756]: audit 2024-03-04T15:58:36.455000+0000 mon.a (mon.0) 2357 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi117-63923-58"}]': finished 2024-03-04T15:58:37.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:37 smithi151 bash[23756]: cluster 2024-03-04T15:58:36.462756+0000 mon.a (mon.0) 2358 : cluster 0 osdmap e327: 8 total, 8 up, 8 in 2024-03-04T15:58:37.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:37 smithi151 bash[23756]: audit 2024-03-04T15:58:36.469722+0000 mon.c (mon.2) 519 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-04T15:58:37.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:37 smithi151 bash[23756]: audit 2024-03-04T15:58:36.470427+0000 mon.c (mon.2) 520 : audit 1 from='client.? 172.21.15.117:0/973126100' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:37.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:37 smithi151 bash[23756]: audit 2024-03-04T15:58:36.470954+0000 mon.a (mon.0) 2359 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-04T15:58:37.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:37 smithi151 bash[23756]: audit 2024-03-04T15:58:36.471916+0000 mon.a (mon.0) 2360 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi117-63923-58"}]: dispatch 2024-03-04T15:58:37.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:37 smithi151 bash[23756]: audit 2024-03-04T15:58:36.697961+0000 mgr.y (mgr.24428) 358 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:58:37.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:37 smithi151 bash[23756]: cluster 2024-03-04T15:58:36.925203+0000 mgr.y (mgr.24428) 359 : cluster 0 pgmap v497: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:38.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:38 smithi117 bash[28843]: audit 2024-03-04T15:58:37.466157+0000 mon.a (mon.0) 2361 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-04T15:58:38.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:38 smithi117 bash[28843]: audit 2024-03-04T15:58:37.466398+0000 mon.a (mon.0) 2362 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi117-63923-58"}]': finished 2024-03-04T15:58:38.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:38 smithi117 bash[28843]: cluster 2024-03-04T15:58:37.476885+0000 mon.a (mon.0) 2363 : cluster 0 osdmap e328: 8 total, 8 up, 8 in 2024-03-04T15:58:38.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:38 smithi117 bash[28843]: audit 2024-03-04T15:58:37.507937+0000 mon.a (mon.0) 2364 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi117-63923-59"}]: dispatch 2024-03-04T15:58:38.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:38 smithi117 bash[28843]: audit 2024-03-04T15:58:37.508870+0000 mon.a (mon.0) 2365 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi117-63923-59"}]: dispatch 2024-03-04T15:58:38.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:38 smithi117 bash[28843]: audit 2024-03-04T15:58:37.509342+0000 mon.a (mon.0) 2366 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi117-63923-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:38.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:38 smithi117 bash[21480]: audit 2024-03-04T15:58:37.466157+0000 mon.a (mon.0) 2361 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-04T15:58:38.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:38 smithi117 bash[21480]: audit 2024-03-04T15:58:37.466398+0000 mon.a (mon.0) 2362 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi117-63923-58"}]': finished 2024-03-04T15:58:38.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:38 smithi117 bash[21480]: cluster 2024-03-04T15:58:37.476885+0000 mon.a (mon.0) 2363 : cluster 0 osdmap e328: 8 total, 8 up, 8 in 2024-03-04T15:58:38.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:38 smithi117 bash[21480]: audit 2024-03-04T15:58:37.507937+0000 mon.a (mon.0) 2364 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi117-63923-59"}]: dispatch 2024-03-04T15:58:38.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:38 smithi117 bash[21480]: audit 2024-03-04T15:58:37.508870+0000 mon.a (mon.0) 2365 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi117-63923-59"}]: dispatch 2024-03-04T15:58:38.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:38 smithi117 bash[21480]: audit 2024-03-04T15:58:37.509342+0000 mon.a (mon.0) 2366 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi117-63923-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:38.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:38 smithi151 bash[23756]: audit 2024-03-04T15:58:37.466157+0000 mon.a (mon.0) 2361 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-04T15:58:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:38 smithi151 bash[23756]: audit 2024-03-04T15:58:37.466398+0000 mon.a (mon.0) 2362 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi117-63923-58"}]': finished 2024-03-04T15:58:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:38 smithi151 bash[23756]: cluster 2024-03-04T15:58:37.476885+0000 mon.a (mon.0) 2363 : cluster 0 osdmap e328: 8 total, 8 up, 8 in 2024-03-04T15:58:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:38 smithi151 bash[23756]: audit 2024-03-04T15:58:37.507937+0000 mon.a (mon.0) 2364 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi117-63923-59"}]: dispatch 2024-03-04T15:58:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:38 smithi151 bash[23756]: audit 2024-03-04T15:58:37.508870+0000 mon.a (mon.0) 2365 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi117-63923-59"}]: dispatch 2024-03-04T15:58:38.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:38 smithi151 bash[23756]: audit 2024-03-04T15:58:37.509342+0000 mon.a (mon.0) 2366 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi117-63923-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:40.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:39 smithi117 bash[28843]: audit 2024-03-04T15:58:38.468824+0000 mon.a (mon.0) 2367 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi117-63923-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:40.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:39 smithi117 bash[28843]: cluster 2024-03-04T15:58:38.473940+0000 mon.a (mon.0) 2368 : cluster 0 osdmap e329: 8 total, 8 up, 8 in 2024-03-04T15:58:40.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:39 smithi117 bash[28843]: audit 2024-03-04T15:58:38.475273+0000 mon.a (mon.0) 2369 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi117-63923-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi117-63923-59"}]: dispatch 2024-03-04T15:58:40.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:39 smithi117 bash[28843]: cluster 2024-03-04T15:58:38.925989+0000 mgr.y (mgr.24428) 360 : cluster 0 pgmap v500: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:40.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:39 smithi117 bash[21480]: audit 2024-03-04T15:58:38.468824+0000 mon.a (mon.0) 2367 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi117-63923-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:40.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:39 smithi117 bash[21480]: cluster 2024-03-04T15:58:38.473940+0000 mon.a (mon.0) 2368 : cluster 0 osdmap e329: 8 total, 8 up, 8 in 2024-03-04T15:58:40.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:39 smithi117 bash[21480]: audit 2024-03-04T15:58:38.475273+0000 mon.a (mon.0) 2369 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi117-63923-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi117-63923-59"}]: dispatch 2024-03-04T15:58:40.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:39 smithi117 bash[21480]: cluster 2024-03-04T15:58:38.925989+0000 mgr.y (mgr.24428) 360 : cluster 0 pgmap v500: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:40.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:39 smithi151 bash[23756]: audit 2024-03-04T15:58:38.468824+0000 mon.a (mon.0) 2367 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi117-63923-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:40.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:39 smithi151 bash[23756]: cluster 2024-03-04T15:58:38.473940+0000 mon.a (mon.0) 2368 : cluster 0 osdmap e329: 8 total, 8 up, 8 in 2024-03-04T15:58:40.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:39 smithi151 bash[23756]: audit 2024-03-04T15:58:38.475273+0000 mon.a (mon.0) 2369 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi117-63923-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi117-63923-59"}]: dispatch 2024-03-04T15:58:40.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:39 smithi151 bash[23756]: cluster 2024-03-04T15:58:38.925989+0000 mgr.y (mgr.24428) 360 : cluster 0 pgmap v500: 228 pgs: 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:41.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:40 smithi117 bash[21480]: audit 2024-03-04T15:58:39.580896+0000 mon.c (mon.2) 521 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:58:41.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:40 smithi117 bash[21480]: cluster 2024-03-04T15:58:39.935954+0000 mon.a (mon.0) 2370 : cluster 0 osdmap e330: 8 total, 8 up, 8 in 2024-03-04T15:58:41.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:40 smithi117 bash[21480]: audit 2024-03-04T15:58:40.311736+0000 mon.c (mon.2) 522 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:58:41.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:40 smithi117 bash[21480]: audit 2024-03-04T15:58:40.313482+0000 mon.c (mon.2) 523 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:58:41.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:40 smithi117 bash[21480]: audit 2024-03-04T15:58:40.325592+0000 mon.a (mon.0) 2371 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:58:41.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:40 smithi117 bash[21480]: audit 2024-03-04T15:58:40.476009+0000 mon.a (mon.0) 2372 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi117-63923-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi117-63923-59"}]': finished 2024-03-04T15:58:41.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:40 smithi117 bash[21480]: cluster 2024-03-04T15:58:40.483088+0000 mon.a (mon.0) 2373 : cluster 0 osdmap e331: 8 total, 8 up, 8 in 2024-03-04T15:58:41.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:40 smithi117 bash[28843]: audit 2024-03-04T15:58:39.580896+0000 mon.c (mon.2) 521 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:58:41.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:40 smithi117 bash[28843]: cluster 2024-03-04T15:58:39.935954+0000 mon.a (mon.0) 2370 : cluster 0 osdmap e330: 8 total, 8 up, 8 in 2024-03-04T15:58:41.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:40 smithi117 bash[28843]: audit 2024-03-04T15:58:40.311736+0000 mon.c (mon.2) 522 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:58:41.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:40 smithi117 bash[28843]: audit 2024-03-04T15:58:40.313482+0000 mon.c (mon.2) 523 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:58:41.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:40 smithi117 bash[28843]: audit 2024-03-04T15:58:40.325592+0000 mon.a (mon.0) 2371 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:58:41.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:40 smithi117 bash[28843]: audit 2024-03-04T15:58:40.476009+0000 mon.a (mon.0) 2372 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi117-63923-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi117-63923-59"}]': finished 2024-03-04T15:58:41.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:40 smithi117 bash[28843]: cluster 2024-03-04T15:58:40.483088+0000 mon.a (mon.0) 2373 : cluster 0 osdmap e331: 8 total, 8 up, 8 in 2024-03-04T15:58:41.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:40 smithi151 bash[23756]: audit 2024-03-04T15:58:39.580896+0000 mon.c (mon.2) 521 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:58:41.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:40 smithi151 bash[23756]: cluster 2024-03-04T15:58:39.935954+0000 mon.a (mon.0) 2370 : cluster 0 osdmap e330: 8 total, 8 up, 8 in 2024-03-04T15:58:41.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:40 smithi151 bash[23756]: audit 2024-03-04T15:58:40.311736+0000 mon.c (mon.2) 522 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:58:41.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:40 smithi151 bash[23756]: audit 2024-03-04T15:58:40.313482+0000 mon.c (mon.2) 523 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:58:41.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:40 smithi151 bash[23756]: audit 2024-03-04T15:58:40.325592+0000 mon.a (mon.0) 2371 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:58:41.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:40 smithi151 bash[23756]: audit 2024-03-04T15:58:40.476009+0000 mon.a (mon.0) 2372 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi117-63923-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi117-63923-59"}]': finished 2024-03-04T15:58:41.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:40 smithi151 bash[23756]: cluster 2024-03-04T15:58:40.483088+0000 mon.a (mon.0) 2373 : cluster 0 osdmap e331: 8 total, 8 up, 8 in 2024-03-04T15:58:42.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:41 smithi117 bash[28843]: cluster 2024-03-04T15:58:40.926834+0000 mgr.y (mgr.24428) 361 : cluster 0 pgmap v503: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:42.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:41 smithi117 bash[28843]: cluster 2024-03-04T15:58:41.475923+0000 mon.a (mon.0) 2374 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:42.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:41 smithi117 bash[28843]: cluster 2024-03-04T15:58:41.494596+0000 mon.a (mon.0) 2375 : cluster 0 osdmap e332: 8 total, 8 up, 8 in 2024-03-04T15:58:42.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:41 smithi117 bash[21480]: cluster 2024-03-04T15:58:40.926834+0000 mgr.y (mgr.24428) 361 : cluster 0 pgmap v503: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:42.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:41 smithi117 bash[21480]: cluster 2024-03-04T15:58:41.475923+0000 mon.a (mon.0) 2374 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:42.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:41 smithi117 bash[21480]: cluster 2024-03-04T15:58:41.494596+0000 mon.a (mon.0) 2375 : cluster 0 osdmap e332: 8 total, 8 up, 8 in 2024-03-04T15:58:42.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:41 smithi151 bash[23756]: cluster 2024-03-04T15:58:40.926834+0000 mgr.y (mgr.24428) 361 : cluster 0 pgmap v503: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 626 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:42.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:41 smithi151 bash[23756]: cluster 2024-03-04T15:58:41.475923+0000 mon.a (mon.0) 2374 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:42.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:41 smithi151 bash[23756]: cluster 2024-03-04T15:58:41.494596+0000 mon.a (mon.0) 2375 : cluster 0 osdmap e332: 8 total, 8 up, 8 in 2024-03-04T15:58:43.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:43 smithi117 bash[28843]: cluster 2024-03-04T15:58:42.498040+0000 mon.a (mon.0) 2376 : cluster 0 osdmap e333: 8 total, 8 up, 8 in 2024-03-04T15:58:43.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:43 smithi117 bash[28843]: audit 2024-03-04T15:58:42.503720+0000 mon.a (mon.0) 2377 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi117-63923-59"}]: dispatch 2024-03-04T15:58:43.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:43 smithi117 bash[28843]: cluster 2024-03-04T15:58:42.927632+0000 mgr.y (mgr.24428) 362 : cluster 0 pgmap v506: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T15:58:43.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:43 smithi117 bash[21480]: cluster 2024-03-04T15:58:42.498040+0000 mon.a (mon.0) 2376 : cluster 0 osdmap e333: 8 total, 8 up, 8 in 2024-03-04T15:58:43.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:43 smithi117 bash[21480]: audit 2024-03-04T15:58:42.503720+0000 mon.a (mon.0) 2377 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi117-63923-59"}]: dispatch 2024-03-04T15:58:43.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:43 smithi117 bash[21480]: cluster 2024-03-04T15:58:42.927632+0000 mgr.y (mgr.24428) 362 : cluster 0 pgmap v506: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T15:58:43.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:43 smithi151 bash[23756]: cluster 2024-03-04T15:58:42.498040+0000 mon.a (mon.0) 2376 : cluster 0 osdmap e333: 8 total, 8 up, 8 in 2024-03-04T15:58:43.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:43 smithi151 bash[23756]: audit 2024-03-04T15:58:42.503720+0000 mon.a (mon.0) 2377 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi117-63923-59"}]: dispatch 2024-03-04T15:58:43.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:43 smithi151 bash[23756]: cluster 2024-03-04T15:58:42.927632+0000 mgr.y (mgr.24428) 362 : cluster 0 pgmap v506: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T15:58:44.522 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:7462ddf6:::io.RoundTripAppendPP (3016 ms) 2024-03-04T15:58:44.522 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-03-04T15:58:44.522 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3045 ms) 2024-03-04T15:58:44.523 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-03-04T15:58:44.523 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3005 ms) 2024-03-04T15:58:44.523 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-03-04T15:58:44.523 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3391 ms) 2024-03-04T15:58:44.523 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-03-04T15:58:44.523 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: setting pool EIO 2024-03-04T15:58:44.523 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: max_success 98, min_failed 99 2024-03-04T15:58:44.523 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (3922 ms) 2024-03-04T15:58:44.523 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-03-04T15:58:44.523 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3456 ms) 2024-03-04T15:58:44.523 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (157713 ms total) 2024-03-04T15:58:44.523 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: 2024-03-04T15:58:44.523 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-03-04T15:58:44.523 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-03-04T15:58:44.524 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (2562 ms) 2024-03-04T15:58:44.524 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-03-04T15:58:44.524 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9055 ms) 2024-03-04T15:58:44.524 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-03-04T15:58:44.524 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14939 ms) 2024-03-04T15:58:44.524 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-03-04T15:58:44.524 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3007 ms) 2024-03-04T15:58:44.524 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29563 ms total) 2024-03-04T15:58:44.524 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: 2024-03-04T15:58:44.524 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-03-04T15:58:44.524 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-03-04T15:58:44.524 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3019 ms) 2024-03-04T15:58:44.524 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3019 ms total) 2024-03-04T15:58:44.525 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: 2024-03-04T15:58:44.525 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-03-04T15:58:44.525 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-03-04T15:58:44.525 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13984 ms) 2024-03-04T15:58:44.525 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-03-04T15:58:44.525 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (6958 ms) 2024-03-04T15:58:44.525 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-03-04T15:58:44.525 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (7068 ms) 2024-03-04T15:58:44.525 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-03-04T15:58:44.525 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7085 ms) 2024-03-04T15:58:44.525 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-03-04T15:58:44.525 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3020 ms) 2024-03-04T15:58:44.525 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-03-04T15:58:44.525 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7044 ms) 2024-03-04T15:58:44.525 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-03-04T15:58:44.526 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (7047 ms) 2024-03-04T15:58:44.526 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-03-04T15:58:44.526 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (6928 ms) 2024-03-04T15:58:44.526 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-03-04T15:58:44.526 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7036 ms) 2024-03-04T15:58:44.526 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-03-04T15:58:44.526 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7052 ms) 2024-03-04T15:58:44.526 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-03-04T15:58:44.526 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7013 ms) 2024-03-04T15:58:44.526 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-03-04T15:58:44.526 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7066 ms) 2024-03-04T15:58:44.526 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-03-04T15:58:44.526 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7043 ms) 2024-03-04T15:58:44.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:44 smithi117 bash[21480]: audit 2024-03-04T15:58:43.501692+0000 mon.a (mon.0) 2378 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi117-63923-59"}]': finished 2024-03-04T15:58:44.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:44 smithi117 bash[21480]: cluster 2024-03-04T15:58:43.511755+0000 mon.a (mon.0) 2379 : cluster 0 osdmap e334: 8 total, 8 up, 8 in 2024-03-04T15:58:44.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:44 smithi117 bash[21480]: audit 2024-03-04T15:58:43.515219+0000 mon.a (mon.0) 2380 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi117-63923-59"}]: dispatch 2024-03-04T15:58:44.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:44 smithi117 bash[21480]: audit 2024-03-04T15:58:43.831964+0000 mon.c (mon.2) 524 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:58:44.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:44 smithi117 bash[28843]: audit 2024-03-04T15:58:43.501692+0000 mon.a (mon.0) 2378 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi117-63923-59"}]': finished 2024-03-04T15:58:44.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:44 smithi117 bash[28843]: cluster 2024-03-04T15:58:43.511755+0000 mon.a (mon.0) 2379 : cluster 0 osdmap e334: 8 total, 8 up, 8 in 2024-03-04T15:58:44.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:44 smithi117 bash[28843]: audit 2024-03-04T15:58:43.515219+0000 mon.a (mon.0) 2380 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi117-63923-59"}]: dispatch 2024-03-04T15:58:44.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:44 smithi117 bash[28843]: audit 2024-03-04T15:58:43.831964+0000 mon.c (mon.2) 524 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:58:44.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:44 smithi151 bash[23756]: audit 2024-03-04T15:58:43.501692+0000 mon.a (mon.0) 2378 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi117-63923-59"}]': finished 2024-03-04T15:58:44.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:44 smithi151 bash[23756]: cluster 2024-03-04T15:58:43.511755+0000 mon.a (mon.0) 2379 : cluster 0 osdmap e334: 8 total, 8 up, 8 in 2024-03-04T15:58:44.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:44 smithi151 bash[23756]: audit 2024-03-04T15:58:43.515219+0000 mon.a (mon.0) 2380 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi117-63923-59"}]: dispatch 2024-03-04T15:58:44.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:44 smithi151 bash[23756]: audit 2024-03-04T15:58:43.831964+0000 mon.c (mon.2) 524 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:58:45.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:45 smithi117 bash[21480]: audit 2024-03-04T15:58:44.512145+0000 mon.a (mon.0) 2381 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi117-63923-59"}]': finished 2024-03-04T15:58:45.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:45 smithi117 bash[21480]: cluster 2024-03-04T15:58:44.520353+0000 mon.a (mon.0) 2382 : cluster 0 osdmap e335: 8 total, 8 up, 8 in 2024-03-04T15:58:45.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:45 smithi117 bash[21480]: cluster 2024-03-04T15:58:44.928421+0000 mgr.y (mgr.24428) 363 : cluster 0 pgmap v509: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T15:58:45.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:45 smithi117 bash[28843]: audit 2024-03-04T15:58:44.512145+0000 mon.a (mon.0) 2381 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi117-63923-59"}]': finished 2024-03-04T15:58:45.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:45 smithi117 bash[28843]: cluster 2024-03-04T15:58:44.520353+0000 mon.a (mon.0) 2382 : cluster 0 osdmap e335: 8 total, 8 up, 8 in 2024-03-04T15:58:45.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:45 smithi117 bash[28843]: cluster 2024-03-04T15:58:44.928421+0000 mgr.y (mgr.24428) 363 : cluster 0 pgmap v509: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T15:58:45.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:45 smithi151 bash[23756]: audit 2024-03-04T15:58:44.512145+0000 mon.a (mon.0) 2381 : audit 1 from='client.? 172.21.15.117:0/2585910609' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi117-63923-59"}]': finished 2024-03-04T15:58:45.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:45 smithi151 bash[23756]: cluster 2024-03-04T15:58:44.520353+0000 mon.a (mon.0) 2382 : cluster 0 osdmap e335: 8 total, 8 up, 8 in 2024-03-04T15:58:45.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:45 smithi151 bash[23756]: cluster 2024-03-04T15:58:44.928421+0000 mgr.y (mgr.24428) 363 : cluster 0 pgmap v509: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T15:58:46.540 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:58:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:58:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:58:46.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:46 smithi117 bash[21480]: cluster 2024-03-04T15:58:45.536578+0000 mon.a (mon.0) 2383 : cluster 0 osdmap e336: 8 total, 8 up, 8 in 2024-03-04T15:58:46.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:46 smithi117 bash[21480]: audit 2024-03-04T15:58:45.546290+0000 mon.c (mon.2) 525 : audit 1 from='client.? 172.21.15.117:0/1447886577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi117-63923-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:46.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:46 smithi117 bash[21480]: audit 2024-03-04T15:58:45.548444+0000 mon.a (mon.0) 2384 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi117-63923-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:46.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:46 smithi117 bash[28843]: cluster 2024-03-04T15:58:45.536578+0000 mon.a (mon.0) 2383 : cluster 0 osdmap e336: 8 total, 8 up, 8 in 2024-03-04T15:58:46.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:46 smithi117 bash[28843]: audit 2024-03-04T15:58:45.546290+0000 mon.c (mon.2) 525 : audit 1 from='client.? 172.21.15.117:0/1447886577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi117-63923-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:46.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:46 smithi117 bash[28843]: audit 2024-03-04T15:58:45.548444+0000 mon.a (mon.0) 2384 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi117-63923-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:46.931 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:58:46 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:58:46.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:46 smithi151 bash[23756]: cluster 2024-03-04T15:58:45.536578+0000 mon.a (mon.0) 2383 : cluster 0 osdmap e336: 8 total, 8 up, 8 in 2024-03-04T15:58:46.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:46 smithi151 bash[23756]: audit 2024-03-04T15:58:45.546290+0000 mon.c (mon.2) 525 : audit 1 from='client.? 172.21.15.117:0/1447886577' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi117-63923-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:46.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:46 smithi151 bash[23756]: audit 2024-03-04T15:58:45.548444+0000 mon.a (mon.0) 2384 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi117-63923-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:47.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:47 smithi117 bash[21480]: audit 2024-03-04T15:58:46.527140+0000 mon.a (mon.0) 2385 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi117-63923-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:47.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:47 smithi117 bash[21480]: cluster 2024-03-04T15:58:46.535827+0000 mon.a (mon.0) 2386 : cluster 0 osdmap e337: 8 total, 8 up, 8 in 2024-03-04T15:58:47.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:47 smithi117 bash[21480]: audit 2024-03-04T15:58:46.709430+0000 mgr.y (mgr.24428) 364 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:58:47.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:47 smithi117 bash[21480]: cluster 2024-03-04T15:58:46.929306+0000 mgr.y (mgr.24428) 365 : cluster 0 pgmap v512: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-04T15:58:47.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:47 smithi117 bash[28843]: audit 2024-03-04T15:58:46.527140+0000 mon.a (mon.0) 2385 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi117-63923-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:47.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:47 smithi117 bash[28843]: cluster 2024-03-04T15:58:46.535827+0000 mon.a (mon.0) 2386 : cluster 0 osdmap e337: 8 total, 8 up, 8 in 2024-03-04T15:58:47.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:47 smithi117 bash[28843]: audit 2024-03-04T15:58:46.709430+0000 mgr.y (mgr.24428) 364 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:58:47.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:47 smithi117 bash[28843]: cluster 2024-03-04T15:58:46.929306+0000 mgr.y (mgr.24428) 365 : cluster 0 pgmap v512: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-04T15:58:47.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:47 smithi151 bash[23756]: audit 2024-03-04T15:58:46.527140+0000 mon.a (mon.0) 2385 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi117-63923-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:47.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:47 smithi151 bash[23756]: cluster 2024-03-04T15:58:46.535827+0000 mon.a (mon.0) 2386 : cluster 0 osdmap e337: 8 total, 8 up, 8 in 2024-03-04T15:58:47.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:47 smithi151 bash[23756]: audit 2024-03-04T15:58:46.709430+0000 mgr.y (mgr.24428) 364 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:58:47.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:47 smithi151 bash[23756]: cluster 2024-03-04T15:58:46.929306+0000 mgr.y (mgr.24428) 365 : cluster 0 pgmap v512: 260 pgs: 32 creating+peering, 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-04T15:58:48.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:48 smithi117 bash[21480]: cluster 2024-03-04T15:58:47.552960+0000 mon.a (mon.0) 2387 : cluster 0 osdmap e338: 8 total, 8 up, 8 in 2024-03-04T15:58:48.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:48 smithi117 bash[21480]: audit 2024-03-04T15:58:47.585161+0000 mon.a (mon.0) 2388 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi117-63923-61"}]: dispatch 2024-03-04T15:58:48.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:48 smithi117 bash[21480]: audit 2024-03-04T15:58:47.586242+0000 mon.a (mon.0) 2389 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi117-63923-61"}]: dispatch 2024-03-04T15:58:48.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:48 smithi117 bash[21480]: audit 2024-03-04T15:58:47.586715+0000 mon.a (mon.0) 2390 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi117-63923-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:48.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:48 smithi117 bash[21480]: cluster 2024-03-04T15:58:48.256994+0000 mon.a (mon.0) 2391 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:48.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:48 smithi117 bash[28843]: cluster 2024-03-04T15:58:47.552960+0000 mon.a (mon.0) 2387 : cluster 0 osdmap e338: 8 total, 8 up, 8 in 2024-03-04T15:58:48.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:48 smithi117 bash[28843]: audit 2024-03-04T15:58:47.585161+0000 mon.a (mon.0) 2388 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi117-63923-61"}]: dispatch 2024-03-04T15:58:48.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:48 smithi117 bash[28843]: audit 2024-03-04T15:58:47.586242+0000 mon.a (mon.0) 2389 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi117-63923-61"}]: dispatch 2024-03-04T15:58:48.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:48 smithi117 bash[28843]: audit 2024-03-04T15:58:47.586715+0000 mon.a (mon.0) 2390 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi117-63923-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:48.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:48 smithi117 bash[28843]: cluster 2024-03-04T15:58:48.256994+0000 mon.a (mon.0) 2391 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:48.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:48 smithi151 bash[23756]: cluster 2024-03-04T15:58:47.552960+0000 mon.a (mon.0) 2387 : cluster 0 osdmap e338: 8 total, 8 up, 8 in 2024-03-04T15:58:48.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:48 smithi151 bash[23756]: audit 2024-03-04T15:58:47.585161+0000 mon.a (mon.0) 2388 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi117-63923-61"}]: dispatch 2024-03-04T15:58:48.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:48 smithi151 bash[23756]: audit 2024-03-04T15:58:47.586242+0000 mon.a (mon.0) 2389 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi117-63923-61"}]: dispatch 2024-03-04T15:58:48.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:48 smithi151 bash[23756]: audit 2024-03-04T15:58:47.586715+0000 mon.a (mon.0) 2390 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi117-63923-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:48.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:48 smithi151 bash[23756]: cluster 2024-03-04T15:58:48.256994+0000 mon.a (mon.0) 2391 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:49.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:49 smithi151 bash[23756]: audit 2024-03-04T15:58:48.573017+0000 mon.a (mon.0) 2392 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi117-63923-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:49.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:49 smithi151 bash[23756]: cluster 2024-03-04T15:58:48.581857+0000 mon.a (mon.0) 2393 : cluster 0 osdmap e339: 8 total, 8 up, 8 in 2024-03-04T15:58:49.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:49 smithi151 bash[23756]: audit 2024-03-04T15:58:48.587971+0000 mon.a (mon.0) 2394 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi117-63923-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi117-63923-61"}]: dispatch 2024-03-04T15:58:49.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:49 smithi151 bash[23756]: cluster 2024-03-04T15:58:48.930091+0000 mgr.y (mgr.24428) 366 : cluster 0 pgmap v515: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-04T15:58:50.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:49 smithi117 bash[21480]: audit 2024-03-04T15:58:48.573017+0000 mon.a (mon.0) 2392 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi117-63923-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:50.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:49 smithi117 bash[21480]: cluster 2024-03-04T15:58:48.581857+0000 mon.a (mon.0) 2393 : cluster 0 osdmap e339: 8 total, 8 up, 8 in 2024-03-04T15:58:50.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:49 smithi117 bash[21480]: audit 2024-03-04T15:58:48.587971+0000 mon.a (mon.0) 2394 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi117-63923-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi117-63923-61"}]: dispatch 2024-03-04T15:58:50.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:49 smithi117 bash[21480]: cluster 2024-03-04T15:58:48.930091+0000 mgr.y (mgr.24428) 366 : cluster 0 pgmap v515: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-04T15:58:50.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:49 smithi117 bash[28843]: audit 2024-03-04T15:58:48.573017+0000 mon.a (mon.0) 2392 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi117-63923-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:50.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:49 smithi117 bash[28843]: cluster 2024-03-04T15:58:48.581857+0000 mon.a (mon.0) 2393 : cluster 0 osdmap e339: 8 total, 8 up, 8 in 2024-03-04T15:58:50.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:49 smithi117 bash[28843]: audit 2024-03-04T15:58:48.587971+0000 mon.a (mon.0) 2394 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi117-63923-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi117-63923-61"}]: dispatch 2024-03-04T15:58:50.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:49 smithi117 bash[28843]: cluster 2024-03-04T15:58:48.930091+0000 mgr.y (mgr.24428) 366 : cluster 0 pgmap v515: 228 pgs: 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-04T15:58:50.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:50 smithi151 bash[23756]: audit 2024-03-04T15:58:49.651324+0000 mon.c (mon.2) 526 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:50 smithi151 bash[23756]: cluster 2024-03-04T15:58:50.040123+0000 mon.a (mon.0) 2395 : cluster 0 osdmap e340: 8 total, 8 up, 8 in 2024-03-04T15:58:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:50 smithi151 bash[23756]: audit 2024-03-04T15:58:50.040879+0000 mon.a (mon.0) 2396 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:50 smithi151 bash[23756]: audit 2024-03-04T15:58:50.041747+0000 mon.c (mon.2) 527 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-45"}]: dispatch 2024-03-04T15:58:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:50 smithi151 bash[23756]: audit 2024-03-04T15:58:50.042242+0000 mon.a (mon.0) 2397 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-45"}]: dispatch 2024-03-04T15:58:51.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:50 smithi117 bash[21480]: audit 2024-03-04T15:58:49.651324+0000 mon.c (mon.2) 526 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:51.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:50 smithi117 bash[21480]: cluster 2024-03-04T15:58:50.040123+0000 mon.a (mon.0) 2395 : cluster 0 osdmap e340: 8 total, 8 up, 8 in 2024-03-04T15:58:51.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:50 smithi117 bash[21480]: audit 2024-03-04T15:58:50.040879+0000 mon.a (mon.0) 2396 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:51.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:50 smithi117 bash[21480]: audit 2024-03-04T15:58:50.041747+0000 mon.c (mon.2) 527 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-45"}]: dispatch 2024-03-04T15:58:51.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:50 smithi117 bash[21480]: audit 2024-03-04T15:58:50.042242+0000 mon.a (mon.0) 2397 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-45"}]: dispatch 2024-03-04T15:58:51.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:50 smithi117 bash[28843]: audit 2024-03-04T15:58:49.651324+0000 mon.c (mon.2) 526 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:51.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:50 smithi117 bash[28843]: cluster 2024-03-04T15:58:50.040123+0000 mon.a (mon.0) 2395 : cluster 0 osdmap e340: 8 total, 8 up, 8 in 2024-03-04T15:58:51.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:50 smithi117 bash[28843]: audit 2024-03-04T15:58:50.040879+0000 mon.a (mon.0) 2396 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:58:51.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:50 smithi117 bash[28843]: audit 2024-03-04T15:58:50.041747+0000 mon.c (mon.2) 527 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-45"}]: dispatch 2024-03-04T15:58:51.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:50 smithi117 bash[28843]: audit 2024-03-04T15:58:50.042242+0000 mon.a (mon.0) 2397 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-45"}]: dispatch 2024-03-04T15:58:51.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:51 smithi151 bash[23756]: audit 2024-03-04T15:58:50.597811+0000 mon.a (mon.0) 2398 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi117-63923-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi117-63923-61"}]': finished 2024-03-04T15:58:51.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:51 smithi151 bash[23756]: audit 2024-03-04T15:58:50.598034+0000 mon.a (mon.0) 2399 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-45"}]': finished 2024-03-04T15:58:51.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:51 smithi151 bash[23756]: cluster 2024-03-04T15:58:50.605079+0000 mon.a (mon.0) 2400 : cluster 0 osdmap e341: 8 total, 8 up, 8 in 2024-03-04T15:58:51.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:51 smithi151 bash[23756]: cluster 2024-03-04T15:58:50.930924+0000 mgr.y (mgr.24428) 367 : cluster 0 pgmap v518: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:52.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:51 smithi117 bash[21480]: audit 2024-03-04T15:58:50.597811+0000 mon.a (mon.0) 2398 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi117-63923-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi117-63923-61"}]': finished 2024-03-04T15:58:52.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:51 smithi117 bash[21480]: audit 2024-03-04T15:58:50.598034+0000 mon.a (mon.0) 2399 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-45"}]': finished 2024-03-04T15:58:52.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:51 smithi117 bash[21480]: cluster 2024-03-04T15:58:50.605079+0000 mon.a (mon.0) 2400 : cluster 0 osdmap e341: 8 total, 8 up, 8 in 2024-03-04T15:58:52.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:51 smithi117 bash[21480]: cluster 2024-03-04T15:58:50.930924+0000 mgr.y (mgr.24428) 367 : cluster 0 pgmap v518: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:52.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:51 smithi117 bash[28843]: audit 2024-03-04T15:58:50.597811+0000 mon.a (mon.0) 2398 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi117-63923-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi117-63923-61"}]': finished 2024-03-04T15:58:52.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:51 smithi117 bash[28843]: audit 2024-03-04T15:58:50.598034+0000 mon.a (mon.0) 2399 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-45"}]': finished 2024-03-04T15:58:52.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:51 smithi117 bash[28843]: cluster 2024-03-04T15:58:50.605079+0000 mon.a (mon.0) 2400 : cluster 0 osdmap e341: 8 total, 8 up, 8 in 2024-03-04T15:58:52.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:51 smithi117 bash[28843]: cluster 2024-03-04T15:58:50.930924+0000 mgr.y (mgr.24428) 367 : cluster 0 pgmap v518: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 627 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:58:52.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:52 smithi151 bash[23756]: cluster 2024-03-04T15:58:51.619167+0000 mon.a (mon.0) 2401 : cluster 0 osdmap e342: 8 total, 8 up, 8 in 2024-03-04T15:58:53.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:52 smithi117 bash[21480]: cluster 2024-03-04T15:58:51.619167+0000 mon.a (mon.0) 2401 : cluster 0 osdmap e342: 8 total, 8 up, 8 in 2024-03-04T15:58:53.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:52 smithi117 bash[28843]: cluster 2024-03-04T15:58:51.619167+0000 mon.a (mon.0) 2401 : cluster 0 osdmap e342: 8 total, 8 up, 8 in 2024-03-04T15:58:53.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:53 smithi151 bash[23756]: cluster 2024-03-04T15:58:52.626048+0000 mon.a (mon.0) 2402 : cluster 0 osdmap e343: 8 total, 8 up, 8 in 2024-03-04T15:58:53.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:53 smithi151 bash[23756]: audit 2024-03-04T15:58:52.628325+0000 mon.c (mon.2) 528 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:53.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:53 smithi151 bash[23756]: audit 2024-03-04T15:58:52.633473+0000 mon.a (mon.0) 2403 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:53.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:53 smithi151 bash[23756]: audit 2024-03-04T15:58:52.634580+0000 mon.a (mon.0) 2404 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi117-63923-61"}]: dispatch 2024-03-04T15:58:53.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:53 smithi151 bash[23756]: cluster 2024-03-04T15:58:52.931752+0000 mgr.y (mgr.24428) 368 : cluster 0 pgmap v521: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:53.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:53 smithi151 bash[23756]: cluster 2024-03-04T15:58:53.258749+0000 mon.a (mon.0) 2405 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:53.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:53 smithi151 bash[23756]: audit 2024-03-04T15:58:53.614739+0000 mon.a (mon.0) 2406 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:53.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:53 smithi151 bash[23756]: audit 2024-03-04T15:58:53.614951+0000 mon.a (mon.0) 2407 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi117-63923-61"}]': finished 2024-03-04T15:58:53.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:53 smithi151 bash[23756]: cluster 2024-03-04T15:58:53.623527+0000 mon.a (mon.0) 2408 : cluster 0 osdmap e344: 8 total, 8 up, 8 in 2024-03-04T15:58:53.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:53 smithi151 bash[23756]: audit 2024-03-04T15:58:53.624509+0000 mon.a (mon.0) 2409 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi117-63923-61"}]: dispatch 2024-03-04T15:58:54.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:53 smithi117 bash[21480]: cluster 2024-03-04T15:58:52.626048+0000 mon.a (mon.0) 2402 : cluster 0 osdmap e343: 8 total, 8 up, 8 in 2024-03-04T15:58:54.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:53 smithi117 bash[21480]: audit 2024-03-04T15:58:52.628325+0000 mon.c (mon.2) 528 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:54.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:53 smithi117 bash[21480]: audit 2024-03-04T15:58:52.633473+0000 mon.a (mon.0) 2403 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:54.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:53 smithi117 bash[21480]: audit 2024-03-04T15:58:52.634580+0000 mon.a (mon.0) 2404 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi117-63923-61"}]: dispatch 2024-03-04T15:58:54.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:53 smithi117 bash[21480]: cluster 2024-03-04T15:58:52.931752+0000 mgr.y (mgr.24428) 368 : cluster 0 pgmap v521: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:54.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:53 smithi117 bash[21480]: cluster 2024-03-04T15:58:53.258749+0000 mon.a (mon.0) 2405 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:54.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:53 smithi117 bash[21480]: audit 2024-03-04T15:58:53.614739+0000 mon.a (mon.0) 2406 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:54.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:53 smithi117 bash[21480]: audit 2024-03-04T15:58:53.614951+0000 mon.a (mon.0) 2407 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi117-63923-61"}]': finished 2024-03-04T15:58:54.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:53 smithi117 bash[21480]: cluster 2024-03-04T15:58:53.623527+0000 mon.a (mon.0) 2408 : cluster 0 osdmap e344: 8 total, 8 up, 8 in 2024-03-04T15:58:54.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:53 smithi117 bash[21480]: audit 2024-03-04T15:58:53.624509+0000 mon.a (mon.0) 2409 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi117-63923-61"}]: dispatch 2024-03-04T15:58:54.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:53 smithi117 bash[28843]: cluster 2024-03-04T15:58:52.626048+0000 mon.a (mon.0) 2402 : cluster 0 osdmap e343: 8 total, 8 up, 8 in 2024-03-04T15:58:54.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:53 smithi117 bash[28843]: audit 2024-03-04T15:58:52.628325+0000 mon.c (mon.2) 528 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:54.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:53 smithi117 bash[28843]: audit 2024-03-04T15:58:52.633473+0000 mon.a (mon.0) 2403 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:58:54.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:53 smithi117 bash[28843]: audit 2024-03-04T15:58:52.634580+0000 mon.a (mon.0) 2404 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi117-63923-61"}]: dispatch 2024-03-04T15:58:54.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:53 smithi117 bash[28843]: cluster 2024-03-04T15:58:52.931752+0000 mgr.y (mgr.24428) 368 : cluster 0 pgmap v521: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:54.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:53 smithi117 bash[28843]: cluster 2024-03-04T15:58:53.258749+0000 mon.a (mon.0) 2405 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:54.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:53 smithi117 bash[28843]: audit 2024-03-04T15:58:53.614739+0000 mon.a (mon.0) 2406 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:58:54.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:53 smithi117 bash[28843]: audit 2024-03-04T15:58:53.614951+0000 mon.a (mon.0) 2407 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi117-63923-61"}]': finished 2024-03-04T15:58:54.073 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:53 smithi117 bash[28843]: cluster 2024-03-04T15:58:53.623527+0000 mon.a (mon.0) 2408 : cluster 0 osdmap e344: 8 total, 8 up, 8 in 2024-03-04T15:58:54.074 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:53 smithi117 bash[28843]: audit 2024-03-04T15:58:53.624509+0000 mon.a (mon.0) 2409 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi117-63923-61"}]: dispatch 2024-03-04T15:58:54.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:54 smithi151 bash[23756]: audit 2024-03-04T15:58:53.712985+0000 mon.c (mon.2) 529 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:54.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:54 smithi151 bash[23756]: audit 2024-03-04T15:58:53.713720+0000 mon.a (mon.0) 2410 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:54.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:54 smithi151 bash[23756]: audit 2024-03-04T15:58:54.618479+0000 mon.a (mon.0) 2411 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi117-63923-61"}]': finished 2024-03-04T15:58:54.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:54 smithi151 bash[23756]: audit 2024-03-04T15:58:54.618605+0000 mon.a (mon.0) 2412 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:58:54.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:54 smithi151 bash[23756]: cluster 2024-03-04T15:58:54.625791+0000 mon.a (mon.0) 2413 : cluster 0 osdmap e345: 8 total, 8 up, 8 in 2024-03-04T15:58:54.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:54 smithi151 bash[23756]: audit 2024-03-04T15:58:54.628208+0000 mon.c (mon.2) 530 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-47"}]: dispatch 2024-03-04T15:58:54.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:54 smithi151 bash[23756]: audit 2024-03-04T15:58:54.629910+0000 mon.a (mon.0) 2414 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-47"}]: dispatch 2024-03-04T15:58:54.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:54 smithi151 bash[23756]: audit 2024-03-04T15:58:54.659739+0000 mon.c (mon.2) 531 : audit 1 from='client.? 172.21.15.117:0/988106252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:58:54.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:54 smithi151 bash[23756]: audit 2024-03-04T15:58:54.660132+0000 mon.a (mon.0) 2415 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:58:54.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:54 smithi151 bash[23756]: audit 2024-03-04T15:58:54.660944+0000 mon.c (mon.2) 532 : audit 1 from='client.? 172.21.15.117:0/988106252' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:58:54.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:54 smithi151 bash[23756]: audit 2024-03-04T15:58:54.661342+0000 mon.a (mon.0) 2416 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:58:54.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:54 smithi151 bash[23756]: audit 2024-03-04T15:58:54.661908+0000 mon.c (mon.2) 533 : audit 1 from='client.? 172.21.15.117:0/988106252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:54.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:54 smithi151 bash[23756]: audit 2024-03-04T15:58:54.662268+0000 mon.a (mon.0) 2417 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:55.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[21480]: audit 2024-03-04T15:58:53.712985+0000 mon.c (mon.2) 529 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:55.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[21480]: audit 2024-03-04T15:58:53.713720+0000 mon.a (mon.0) 2410 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:55.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[21480]: audit 2024-03-04T15:58:54.618479+0000 mon.a (mon.0) 2411 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi117-63923-61"}]': finished 2024-03-04T15:58:55.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[21480]: audit 2024-03-04T15:58:54.618605+0000 mon.a (mon.0) 2412 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:58:55.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[21480]: cluster 2024-03-04T15:58:54.625791+0000 mon.a (mon.0) 2413 : cluster 0 osdmap e345: 8 total, 8 up, 8 in 2024-03-04T15:58:55.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[21480]: audit 2024-03-04T15:58:54.628208+0000 mon.c (mon.2) 530 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-47"}]: dispatch 2024-03-04T15:58:55.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[21480]: audit 2024-03-04T15:58:54.629910+0000 mon.a (mon.0) 2414 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-47"}]: dispatch 2024-03-04T15:58:55.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[21480]: audit 2024-03-04T15:58:54.659739+0000 mon.c (mon.2) 531 : audit 1 from='client.? 172.21.15.117:0/988106252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:58:55.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[21480]: audit 2024-03-04T15:58:54.660132+0000 mon.a (mon.0) 2415 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:58:55.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[21480]: audit 2024-03-04T15:58:54.660944+0000 mon.c (mon.2) 532 : audit 1 from='client.? 172.21.15.117:0/988106252' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:58:55.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[21480]: audit 2024-03-04T15:58:54.661342+0000 mon.a (mon.0) 2416 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:58:55.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[21480]: audit 2024-03-04T15:58:54.661908+0000 mon.c (mon.2) 533 : audit 1 from='client.? 172.21.15.117:0/988106252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:55.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[21480]: audit 2024-03-04T15:58:54.662268+0000 mon.a (mon.0) 2417 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:55.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[28843]: audit 2024-03-04T15:58:53.712985+0000 mon.c (mon.2) 529 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:55.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[28843]: audit 2024-03-04T15:58:53.713720+0000 mon.a (mon.0) 2410 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:58:55.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[28843]: audit 2024-03-04T15:58:54.618479+0000 mon.a (mon.0) 2411 : audit 1 from='client.? 172.21.15.117:0/3918408739' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi117-63923-61"}]': finished 2024-03-04T15:58:55.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[28843]: audit 2024-03-04T15:58:54.618605+0000 mon.a (mon.0) 2412 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:58:55.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[28843]: cluster 2024-03-04T15:58:54.625791+0000 mon.a (mon.0) 2413 : cluster 0 osdmap e345: 8 total, 8 up, 8 in 2024-03-04T15:58:55.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[28843]: audit 2024-03-04T15:58:54.628208+0000 mon.c (mon.2) 530 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-47"}]: dispatch 2024-03-04T15:58:55.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[28843]: audit 2024-03-04T15:58:54.629910+0000 mon.a (mon.0) 2414 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-47"}]: dispatch 2024-03-04T15:58:55.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[28843]: audit 2024-03-04T15:58:54.659739+0000 mon.c (mon.2) 531 : audit 1 from='client.? 172.21.15.117:0/988106252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:58:55.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[28843]: audit 2024-03-04T15:58:54.660132+0000 mon.a (mon.0) 2415 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:58:55.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[28843]: audit 2024-03-04T15:58:54.660944+0000 mon.c (mon.2) 532 : audit 1 from='client.? 172.21.15.117:0/988106252' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:58:55.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[28843]: audit 2024-03-04T15:58:54.661342+0000 mon.a (mon.0) 2416 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:58:55.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[28843]: audit 2024-03-04T15:58:54.661908+0000 mon.c (mon.2) 533 : audit 1 from='client.? 172.21.15.117:0/988106252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:55.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:54 smithi117 bash[28843]: audit 2024-03-04T15:58:54.662268+0000 mon.a (mon.0) 2417 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:58:56.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:55 smithi117 bash[21480]: cluster 2024-03-04T15:58:54.932682+0000 mgr.y (mgr.24428) 369 : cluster 0 pgmap v524: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:56.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:55 smithi117 bash[21480]: audit 2024-03-04T15:58:55.621481+0000 mon.a (mon.0) 2418 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-47"}]': finished 2024-03-04T15:58:56.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:55 smithi117 bash[21480]: audit 2024-03-04T15:58:55.621753+0000 mon.a (mon.0) 2419 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:56.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:55 smithi117 bash[21480]: cluster 2024-03-04T15:58:55.630242+0000 mon.a (mon.0) 2420 : cluster 0 osdmap e346: 8 total, 8 up, 8 in 2024-03-04T15:58:56.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:55 smithi117 bash[21480]: audit 2024-03-04T15:58:55.631498+0000 mon.c (mon.2) 534 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-47", "mode": "writeback"}]: dispatch 2024-03-04T15:58:56.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:55 smithi117 bash[21480]: audit 2024-03-04T15:58:55.632324+0000 mon.c (mon.2) 535 : audit 1 from='client.? 172.21.15.117:0/988106252' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi117-63923-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:58:56.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:55 smithi117 bash[21480]: audit 2024-03-04T15:58:55.634311+0000 mon.a (mon.0) 2421 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-47", "mode": "writeback"}]: dispatch 2024-03-04T15:58:56.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:55 smithi117 bash[21480]: audit 2024-03-04T15:58:55.635108+0000 mon.a (mon.0) 2422 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi117-63923-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:58:56.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:55 smithi117 bash[28843]: cluster 2024-03-04T15:58:54.932682+0000 mgr.y (mgr.24428) 369 : cluster 0 pgmap v524: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:56.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:55 smithi117 bash[28843]: audit 2024-03-04T15:58:55.621481+0000 mon.a (mon.0) 2418 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-47"}]': finished 2024-03-04T15:58:56.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:55 smithi117 bash[28843]: audit 2024-03-04T15:58:55.621753+0000 mon.a (mon.0) 2419 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:56.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:55 smithi117 bash[28843]: cluster 2024-03-04T15:58:55.630242+0000 mon.a (mon.0) 2420 : cluster 0 osdmap e346: 8 total, 8 up, 8 in 2024-03-04T15:58:56.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:55 smithi117 bash[28843]: audit 2024-03-04T15:58:55.631498+0000 mon.c (mon.2) 534 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-47", "mode": "writeback"}]: dispatch 2024-03-04T15:58:56.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:55 smithi117 bash[28843]: audit 2024-03-04T15:58:55.632324+0000 mon.c (mon.2) 535 : audit 1 from='client.? 172.21.15.117:0/988106252' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi117-63923-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:58:56.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:55 smithi117 bash[28843]: audit 2024-03-04T15:58:55.634311+0000 mon.a (mon.0) 2421 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-47", "mode": "writeback"}]: dispatch 2024-03-04T15:58:56.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:55 smithi117 bash[28843]: audit 2024-03-04T15:58:55.635108+0000 mon.a (mon.0) 2422 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi117-63923-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:58:56.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:55 smithi151 bash[23756]: cluster 2024-03-04T15:58:54.932682+0000 mgr.y (mgr.24428) 369 : cluster 0 pgmap v524: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:58:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:55 smithi151 bash[23756]: audit 2024-03-04T15:58:55.621481+0000 mon.a (mon.0) 2418 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-47"}]': finished 2024-03-04T15:58:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:55 smithi151 bash[23756]: audit 2024-03-04T15:58:55.621753+0000 mon.a (mon.0) 2419 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:58:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:55 smithi151 bash[23756]: cluster 2024-03-04T15:58:55.630242+0000 mon.a (mon.0) 2420 : cluster 0 osdmap e346: 8 total, 8 up, 8 in 2024-03-04T15:58:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:55 smithi151 bash[23756]: audit 2024-03-04T15:58:55.631498+0000 mon.c (mon.2) 534 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-47", "mode": "writeback"}]: dispatch 2024-03-04T15:58:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:55 smithi151 bash[23756]: audit 2024-03-04T15:58:55.632324+0000 mon.c (mon.2) 535 : audit 1 from='client.? 172.21.15.117:0/988106252' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi117-63923-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:58:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:55 smithi151 bash[23756]: audit 2024-03-04T15:58:55.634311+0000 mon.a (mon.0) 2421 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-47", "mode": "writeback"}]: dispatch 2024-03-04T15:58:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:55 smithi151 bash[23756]: audit 2024-03-04T15:58:55.635108+0000 mon.a (mon.0) 2422 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi117-63923-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:58:56.567 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:58:56 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:58:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:58:57.084 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:58:56 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:58:57.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:57 smithi151 bash[23756]: cluster 2024-03-04T15:58:56.622330+0000 mon.a (mon.0) 2423 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:58:57.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:57 smithi151 bash[23756]: audit 2024-03-04T15:58:56.624703+0000 mon.a (mon.0) 2424 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-47", "mode": "writeback"}]': finished 2024-03-04T15:58:57.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:57 smithi117 bash[21480]: cluster 2024-03-04T15:58:56.622330+0000 mon.a (mon.0) 2423 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:58:57.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:57 smithi117 bash[21480]: audit 2024-03-04T15:58:56.624703+0000 mon.a (mon.0) 2424 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-47", "mode": "writeback"}]': finished 2024-03-04T15:58:57.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:57 smithi117 bash[28843]: cluster 2024-03-04T15:58:56.622330+0000 mon.a (mon.0) 2423 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:58:57.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:57 smithi117 bash[28843]: audit 2024-03-04T15:58:56.624703+0000 mon.a (mon.0) 2424 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-47", "mode": "writeback"}]': finished 2024-03-04T15:58:58.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:58 smithi151 bash[23756]: audit 2024-03-04T15:58:56.638281+0000 mon.c (mon.2) 536 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T15:58:58.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:58 smithi151 bash[23756]: audit 2024-03-04T15:58:56.720795+0000 mgr.y (mgr.24428) 370 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:58:58.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:58 smithi151 bash[23756]: cluster 2024-03-04T15:58:56.933483+0000 mgr.y (mgr.24428) 371 : cluster 0 pgmap v527: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-04T15:58:58.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:58 smithi151 bash[23756]: cluster 2024-03-04T15:58:57.083033+0000 mon.a (mon.0) 2425 : cluster 0 osdmap e347: 8 total, 8 up, 8 in 2024-03-04T15:58:58.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:58 smithi151 bash[23756]: audit 2024-03-04T15:58:57.084882+0000 mon.a (mon.0) 2426 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T15:58:58.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:58 smithi151 bash[23756]: audit 2024-03-04T15:58:57.627942+0000 mon.a (mon.0) 2427 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi117-63923-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi117-63923-62"}]': finished 2024-03-04T15:58:58.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:58 smithi151 bash[23756]: audit 2024-03-04T15:58:57.628148+0000 mon.a (mon.0) 2428 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_count","val": "2"}]': finished 2024-03-04T15:58:58.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:58 smithi151 bash[23756]: cluster 2024-03-04T15:58:57.635612+0000 mon.a (mon.0) 2429 : cluster 0 osdmap e348: 8 total, 8 up, 8 in 2024-03-04T15:58:58.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:58 smithi151 bash[23756]: audit 2024-03-04T15:58:57.637714+0000 mon.c (mon.2) 537 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:58:58.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:58 smithi151 bash[23756]: audit 2024-03-04T15:58:57.643735+0000 mon.a (mon.0) 2430 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:58:58.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:58 smithi117 bash[21480]: audit 2024-03-04T15:58:56.638281+0000 mon.c (mon.2) 536 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T15:58:58.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:58 smithi117 bash[21480]: audit 2024-03-04T15:58:56.720795+0000 mgr.y (mgr.24428) 370 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:58:58.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:58 smithi117 bash[21480]: cluster 2024-03-04T15:58:56.933483+0000 mgr.y (mgr.24428) 371 : cluster 0 pgmap v527: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-04T15:58:58.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:58 smithi117 bash[21480]: cluster 2024-03-04T15:58:57.083033+0000 mon.a (mon.0) 2425 : cluster 0 osdmap e347: 8 total, 8 up, 8 in 2024-03-04T15:58:58.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:58 smithi117 bash[21480]: audit 2024-03-04T15:58:57.084882+0000 mon.a (mon.0) 2426 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T15:58:58.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:58 smithi117 bash[21480]: audit 2024-03-04T15:58:57.627942+0000 mon.a (mon.0) 2427 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi117-63923-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi117-63923-62"}]': finished 2024-03-04T15:58:58.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:58 smithi117 bash[21480]: audit 2024-03-04T15:58:57.628148+0000 mon.a (mon.0) 2428 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_count","val": "2"}]': finished 2024-03-04T15:58:58.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:58 smithi117 bash[21480]: cluster 2024-03-04T15:58:57.635612+0000 mon.a (mon.0) 2429 : cluster 0 osdmap e348: 8 total, 8 up, 8 in 2024-03-04T15:58:58.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:58 smithi117 bash[21480]: audit 2024-03-04T15:58:57.637714+0000 mon.c (mon.2) 537 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:58:58.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:58 smithi117 bash[21480]: audit 2024-03-04T15:58:57.643735+0000 mon.a (mon.0) 2430 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:58:58.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:58 smithi117 bash[28843]: audit 2024-03-04T15:58:56.638281+0000 mon.c (mon.2) 536 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T15:58:58.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:58 smithi117 bash[28843]: audit 2024-03-04T15:58:56.720795+0000 mgr.y (mgr.24428) 370 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:58:58.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:58 smithi117 bash[28843]: cluster 2024-03-04T15:58:56.933483+0000 mgr.y (mgr.24428) 371 : cluster 0 pgmap v527: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-04T15:58:58.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:58 smithi117 bash[28843]: cluster 2024-03-04T15:58:57.083033+0000 mon.a (mon.0) 2425 : cluster 0 osdmap e347: 8 total, 8 up, 8 in 2024-03-04T15:58:58.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:58 smithi117 bash[28843]: audit 2024-03-04T15:58:57.084882+0000 mon.a (mon.0) 2426 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T15:58:58.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:58 smithi117 bash[28843]: audit 2024-03-04T15:58:57.627942+0000 mon.a (mon.0) 2427 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi117-63923-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi117-63923-62"}]': finished 2024-03-04T15:58:58.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:58 smithi117 bash[28843]: audit 2024-03-04T15:58:57.628148+0000 mon.a (mon.0) 2428 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_count","val": "2"}]': finished 2024-03-04T15:58:58.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:58 smithi117 bash[28843]: cluster 2024-03-04T15:58:57.635612+0000 mon.a (mon.0) 2429 : cluster 0 osdmap e348: 8 total, 8 up, 8 in 2024-03-04T15:58:58.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:58 smithi117 bash[28843]: audit 2024-03-04T15:58:57.637714+0000 mon.c (mon.2) 537 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:58:58.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:58 smithi117 bash[28843]: audit 2024-03-04T15:58:57.643735+0000 mon.a (mon.0) 2430 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:58:59.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:59 smithi151 bash[23756]: cluster 2024-03-04T15:58:58.260131+0000 mon.a (mon.0) 2431 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:59.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:59 smithi151 bash[23756]: audit 2024-03-04T15:58:58.630718+0000 mon.a (mon.0) 2432 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_period","val": "600"}]': finished 2024-03-04T15:58:59.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:59 smithi151 bash[23756]: cluster 2024-03-04T15:58:58.639023+0000 mon.a (mon.0) 2433 : cluster 0 osdmap e349: 8 total, 8 up, 8 in 2024-03-04T15:58:59.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:59 smithi151 bash[23756]: audit 2024-03-04T15:58:58.640157+0000 mon.c (mon.2) 538 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T15:58:59.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:59 smithi151 bash[23756]: audit 2024-03-04T15:58:58.641245+0000 mon.a (mon.0) 2434 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T15:58:59.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:59 smithi151 bash[23756]: audit 2024-03-04T15:58:58.872493+0000 mon.a (mon.0) 2435 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:58:59.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:58:59 smithi151 bash[23756]: audit 2024-03-04T15:58:58.875547+0000 mon.c (mon.2) 539 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:58:59.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:59 smithi117 bash[21480]: cluster 2024-03-04T15:58:58.260131+0000 mon.a (mon.0) 2431 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:59.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:59 smithi117 bash[21480]: audit 2024-03-04T15:58:58.630718+0000 mon.a (mon.0) 2432 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_period","val": "600"}]': finished 2024-03-04T15:58:59.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:59 smithi117 bash[21480]: cluster 2024-03-04T15:58:58.639023+0000 mon.a (mon.0) 2433 : cluster 0 osdmap e349: 8 total, 8 up, 8 in 2024-03-04T15:58:59.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:59 smithi117 bash[21480]: audit 2024-03-04T15:58:58.640157+0000 mon.c (mon.2) 538 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T15:58:59.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:59 smithi117 bash[21480]: audit 2024-03-04T15:58:58.641245+0000 mon.a (mon.0) 2434 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T15:58:59.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:59 smithi117 bash[21480]: audit 2024-03-04T15:58:58.872493+0000 mon.a (mon.0) 2435 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:58:59.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:58:59 smithi117 bash[21480]: audit 2024-03-04T15:58:58.875547+0000 mon.c (mon.2) 539 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:58:59.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:59 smithi117 bash[28843]: cluster 2024-03-04T15:58:58.260131+0000 mon.a (mon.0) 2431 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:58:59.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:59 smithi117 bash[28843]: audit 2024-03-04T15:58:58.630718+0000 mon.a (mon.0) 2432 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_period","val": "600"}]': finished 2024-03-04T15:58:59.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:59 smithi117 bash[28843]: cluster 2024-03-04T15:58:58.639023+0000 mon.a (mon.0) 2433 : cluster 0 osdmap e349: 8 total, 8 up, 8 in 2024-03-04T15:58:59.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:59 smithi117 bash[28843]: audit 2024-03-04T15:58:58.640157+0000 mon.c (mon.2) 538 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T15:58:59.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:59 smithi117 bash[28843]: audit 2024-03-04T15:58:58.641245+0000 mon.a (mon.0) 2434 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T15:58:59.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:59 smithi117 bash[28843]: audit 2024-03-04T15:58:58.872493+0000 mon.a (mon.0) 2435 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:58:59.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:58:59 smithi117 bash[28843]: audit 2024-03-04T15:58:58.875547+0000 mon.c (mon.2) 539 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:59:00.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:00 smithi151 bash[23756]: cluster 2024-03-04T15:58:58.934301+0000 mgr.y (mgr.24428) 372 : cluster 0 pgmap v530: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-04T15:59:00.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:00 smithi151 bash[23756]: cluster 2024-03-04T15:58:59.631068+0000 mon.a (mon.0) 2436 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:59:00.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:00 smithi151 bash[23756]: audit 2024-03-04T15:58:59.633494+0000 mon.a (mon.0) 2437 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T15:59:00.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:00 smithi151 bash[23756]: cluster 2024-03-04T15:58:59.640946+0000 mon.a (mon.0) 2438 : cluster 0 osdmap e350: 8 total, 8 up, 8 in 2024-03-04T15:59:00.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:00 smithi151 bash[23756]: audit 2024-03-04T15:58:59.643607+0000 mon.c (mon.2) 540 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-04T15:59:00.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:00 smithi151 bash[23756]: audit 2024-03-04T15:58:59.645031+0000 mon.a (mon.0) 2439 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-04T15:59:00.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:00 smithi151 bash[23756]: audit 2024-03-04T15:58:59.645388+0000 mon.c (mon.2) 541 : audit 1 from='client.? 172.21.15.117:0/988106252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:59:00.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:00 smithi151 bash[23756]: audit 2024-03-04T15:58:59.646399+0000 mon.a (mon.0) 2440 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:59:00.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:00 smithi117 bash[21480]: cluster 2024-03-04T15:58:58.934301+0000 mgr.y (mgr.24428) 372 : cluster 0 pgmap v530: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-04T15:59:00.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:00 smithi117 bash[21480]: cluster 2024-03-04T15:58:59.631068+0000 mon.a (mon.0) 2436 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:59:00.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:00 smithi117 bash[21480]: audit 2024-03-04T15:58:59.633494+0000 mon.a (mon.0) 2437 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T15:59:00.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:00 smithi117 bash[21480]: cluster 2024-03-04T15:58:59.640946+0000 mon.a (mon.0) 2438 : cluster 0 osdmap e350: 8 total, 8 up, 8 in 2024-03-04T15:59:00.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:00 smithi117 bash[21480]: audit 2024-03-04T15:58:59.643607+0000 mon.c (mon.2) 540 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-04T15:59:00.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:00 smithi117 bash[21480]: audit 2024-03-04T15:58:59.645031+0000 mon.a (mon.0) 2439 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-04T15:59:00.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:00 smithi117 bash[21480]: audit 2024-03-04T15:58:59.645388+0000 mon.c (mon.2) 541 : audit 1 from='client.? 172.21.15.117:0/988106252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:59:00.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:00 smithi117 bash[21480]: audit 2024-03-04T15:58:59.646399+0000 mon.a (mon.0) 2440 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:59:00.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:00 smithi117 bash[28843]: cluster 2024-03-04T15:58:58.934301+0000 mgr.y (mgr.24428) 372 : cluster 0 pgmap v530: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-04T15:59:00.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:00 smithi117 bash[28843]: cluster 2024-03-04T15:58:59.631068+0000 mon.a (mon.0) 2436 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:59:00.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:00 smithi117 bash[28843]: audit 2024-03-04T15:58:59.633494+0000 mon.a (mon.0) 2437 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T15:59:00.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:00 smithi117 bash[28843]: cluster 2024-03-04T15:58:59.640946+0000 mon.a (mon.0) 2438 : cluster 0 osdmap e350: 8 total, 8 up, 8 in 2024-03-04T15:59:00.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:00 smithi117 bash[28843]: audit 2024-03-04T15:58:59.643607+0000 mon.c (mon.2) 540 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-04T15:59:00.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:00 smithi117 bash[28843]: audit 2024-03-04T15:58:59.645031+0000 mon.a (mon.0) 2439 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-04T15:59:00.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:00 smithi117 bash[28843]: audit 2024-03-04T15:58:59.645388+0000 mon.c (mon.2) 541 : audit 1 from='client.? 172.21.15.117:0/988106252' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:59:00.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:00 smithi117 bash[28843]: audit 2024-03-04T15:58:59.646399+0000 mon.a (mon.0) 2440 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:59:01.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:01 smithi151 bash[23756]: audit 2024-03-04T15:59:00.636591+0000 mon.a (mon.0) 2441 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-04T15:59:01.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:01 smithi151 bash[23756]: audit 2024-03-04T15:59:00.636754+0000 mon.a (mon.0) 2442 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62"}]': finished 2024-03-04T15:59:01.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:01 smithi151 bash[23756]: cluster 2024-03-04T15:59:00.644228+0000 mon.a (mon.0) 2443 : cluster 0 osdmap e351: 8 total, 8 up, 8 in 2024-03-04T15:59:01.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:01 smithi151 bash[23756]: audit 2024-03-04T15:59:00.646589+0000 mon.c (mon.2) 542 : audit 1 from='client.? 172.21.15.117:0/988106252' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:59:01.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:01 smithi151 bash[23756]: audit 2024-03-04T15:59:00.650519+0000 mon.c (mon.2) 543 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-04T15:59:01.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:01 smithi151 bash[23756]: audit 2024-03-04T15:59:00.650772+0000 mon.a (mon.0) 2444 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:59:01.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:01 smithi151 bash[23756]: audit 2024-03-04T15:59:00.651612+0000 mon.a (mon.0) 2445 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-04T15:59:01.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:01 smithi151 bash[23756]: cluster 2024-03-04T15:59:00.935256+0000 mgr.y (mgr.24428) 373 : cluster 0 pgmap v533: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:59:02.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:01 smithi117 bash[21480]: audit 2024-03-04T15:59:00.636591+0000 mon.a (mon.0) 2441 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-04T15:59:02.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:01 smithi117 bash[21480]: audit 2024-03-04T15:59:00.636754+0000 mon.a (mon.0) 2442 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62"}]': finished 2024-03-04T15:59:02.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:01 smithi117 bash[21480]: cluster 2024-03-04T15:59:00.644228+0000 mon.a (mon.0) 2443 : cluster 0 osdmap e351: 8 total, 8 up, 8 in 2024-03-04T15:59:02.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:01 smithi117 bash[21480]: audit 2024-03-04T15:59:00.646589+0000 mon.c (mon.2) 542 : audit 1 from='client.? 172.21.15.117:0/988106252' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:59:02.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:01 smithi117 bash[21480]: audit 2024-03-04T15:59:00.650519+0000 mon.c (mon.2) 543 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-04T15:59:02.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:01 smithi117 bash[21480]: audit 2024-03-04T15:59:00.650772+0000 mon.a (mon.0) 2444 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:59:02.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:01 smithi117 bash[21480]: audit 2024-03-04T15:59:00.651612+0000 mon.a (mon.0) 2445 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-04T15:59:02.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:01 smithi117 bash[21480]: cluster 2024-03-04T15:59:00.935256+0000 mgr.y (mgr.24428) 373 : cluster 0 pgmap v533: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:59:02.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:01 smithi117 bash[28843]: audit 2024-03-04T15:59:00.636591+0000 mon.a (mon.0) 2441 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-04T15:59:02.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:01 smithi117 bash[28843]: audit 2024-03-04T15:59:00.636754+0000 mon.a (mon.0) 2442 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi117-63923-62"}]': finished 2024-03-04T15:59:02.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:01 smithi117 bash[28843]: cluster 2024-03-04T15:59:00.644228+0000 mon.a (mon.0) 2443 : cluster 0 osdmap e351: 8 total, 8 up, 8 in 2024-03-04T15:59:02.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:01 smithi117 bash[28843]: audit 2024-03-04T15:59:00.646589+0000 mon.c (mon.2) 542 : audit 1 from='client.? 172.21.15.117:0/988106252' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:59:02.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:01 smithi117 bash[28843]: audit 2024-03-04T15:59:00.650519+0000 mon.c (mon.2) 543 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-04T15:59:02.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:01 smithi117 bash[28843]: audit 2024-03-04T15:59:00.650772+0000 mon.a (mon.0) 2444 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi117-63923-62"}]: dispatch 2024-03-04T15:59:02.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:01 smithi117 bash[28843]: audit 2024-03-04T15:59:00.651612+0000 mon.a (mon.0) 2445 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-04T15:59:02.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:01 smithi117 bash[28843]: cluster 2024-03-04T15:59:00.935256+0000 mgr.y (mgr.24428) 373 : cluster 0 pgmap v533: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:59:02.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:02 smithi151 bash[23756]: audit 2024-03-04T15:59:01.647280+0000 mon.a (mon.0) 2446 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi117-63923-62"}]': finished 2024-03-04T15:59:02.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:02 smithi151 bash[23756]: audit 2024-03-04T15:59:01.647511+0000 mon.a (mon.0) 2447 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-04T15:59:02.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:02 smithi151 bash[23756]: cluster 2024-03-04T15:59:01.660046+0000 mon.a (mon.0) 2448 : cluster 0 osdmap e352: 8 total, 8 up, 8 in 2024-03-04T15:59:02.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:02 smithi151 bash[23756]: audit 2024-03-04T15:59:01.660927+0000 mon.c (mon.2) 544 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-04T15:59:02.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:02 smithi151 bash[23756]: audit 2024-03-04T15:59:01.663045+0000 mon.a (mon.0) 2449 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-04T15:59:02.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:02 smithi151 bash[23756]: audit 2024-03-04T15:59:01.702836+0000 mon.c (mon.2) 545 : audit 1 from='client.? 172.21.15.117:0/4033383151' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:02.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:02 smithi151 bash[23756]: audit 2024-03-04T15:59:01.703228+0000 mon.a (mon.0) 2450 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:02.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:02 smithi151 bash[23756]: audit 2024-03-04T15:59:01.704009+0000 mon.c (mon.2) 546 : audit 1 from='client.? 172.21.15.117:0/4033383151' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:02.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:02 smithi151 bash[23756]: audit 2024-03-04T15:59:01.704413+0000 mon.a (mon.0) 2451 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:02.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:02 smithi151 bash[23756]: audit 2024-03-04T15:59:01.704978+0000 mon.c (mon.2) 547 : audit 1 from='client.? 172.21.15.117:0/4033383151' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi117-63923-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:02.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:02 smithi151 bash[23756]: audit 2024-03-04T15:59:01.705304+0000 mon.a (mon.0) 2452 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi117-63923-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:03.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[21480]: audit 2024-03-04T15:59:01.647280+0000 mon.a (mon.0) 2446 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi117-63923-62"}]': finished 2024-03-04T15:59:03.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[21480]: audit 2024-03-04T15:59:01.647511+0000 mon.a (mon.0) 2447 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-04T15:59:03.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[21480]: cluster 2024-03-04T15:59:01.660046+0000 mon.a (mon.0) 2448 : cluster 0 osdmap e352: 8 total, 8 up, 8 in 2024-03-04T15:59:03.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[21480]: audit 2024-03-04T15:59:01.660927+0000 mon.c (mon.2) 544 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-04T15:59:03.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[21480]: audit 2024-03-04T15:59:01.663045+0000 mon.a (mon.0) 2449 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-04T15:59:03.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[21480]: audit 2024-03-04T15:59:01.702836+0000 mon.c (mon.2) 545 : audit 1 from='client.? 172.21.15.117:0/4033383151' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:03.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[21480]: audit 2024-03-04T15:59:01.703228+0000 mon.a (mon.0) 2450 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:03.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[21480]: audit 2024-03-04T15:59:01.704009+0000 mon.c (mon.2) 546 : audit 1 from='client.? 172.21.15.117:0/4033383151' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:03.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[21480]: audit 2024-03-04T15:59:01.704413+0000 mon.a (mon.0) 2451 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:03.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[21480]: audit 2024-03-04T15:59:01.704978+0000 mon.c (mon.2) 547 : audit 1 from='client.? 172.21.15.117:0/4033383151' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi117-63923-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:03.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[21480]: audit 2024-03-04T15:59:01.705304+0000 mon.a (mon.0) 2452 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi117-63923-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:03.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[28843]: audit 2024-03-04T15:59:01.647280+0000 mon.a (mon.0) 2446 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi117-63923-62"}]': finished 2024-03-04T15:59:03.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[28843]: audit 2024-03-04T15:59:01.647511+0000 mon.a (mon.0) 2447 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-04T15:59:03.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[28843]: cluster 2024-03-04T15:59:01.660046+0000 mon.a (mon.0) 2448 : cluster 0 osdmap e352: 8 total, 8 up, 8 in 2024-03-04T15:59:03.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[28843]: audit 2024-03-04T15:59:01.660927+0000 mon.c (mon.2) 544 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-04T15:59:03.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[28843]: audit 2024-03-04T15:59:01.663045+0000 mon.a (mon.0) 2449 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-04T15:59:03.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[28843]: audit 2024-03-04T15:59:01.702836+0000 mon.c (mon.2) 545 : audit 1 from='client.? 172.21.15.117:0/4033383151' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:03.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[28843]: audit 2024-03-04T15:59:01.703228+0000 mon.a (mon.0) 2450 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:03.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[28843]: audit 2024-03-04T15:59:01.704009+0000 mon.c (mon.2) 546 : audit 1 from='client.? 172.21.15.117:0/4033383151' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:03.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[28843]: audit 2024-03-04T15:59:01.704413+0000 mon.a (mon.0) 2451 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:03.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[28843]: audit 2024-03-04T15:59:01.704978+0000 mon.c (mon.2) 547 : audit 1 from='client.? 172.21.15.117:0/4033383151' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi117-63923-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:03.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:02 smithi117 bash[28843]: audit 2024-03-04T15:59:01.705304+0000 mon.a (mon.0) 2452 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi117-63923-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:04 smithi151 bash[23756]: audit 2024-03-04T15:59:02.649969+0000 mon.a (mon.0) 2453 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-04T15:59:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:04 smithi151 bash[23756]: audit 2024-03-04T15:59:02.650256+0000 mon.a (mon.0) 2454 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi117-63923-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:59:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:04 smithi151 bash[23756]: cluster 2024-03-04T15:59:02.658404+0000 mon.a (mon.0) 2455 : cluster 0 osdmap e353: 8 total, 8 up, 8 in 2024-03-04T15:59:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:04 smithi151 bash[23756]: audit 2024-03-04T15:59:02.660353+0000 mon.c (mon.2) 548 : audit 1 from='client.? 172.21.15.117:0/4033383151' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi117-63923-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:04 smithi151 bash[23756]: audit 2024-03-04T15:59:02.664889+0000 mon.a (mon.0) 2456 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi117-63923-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:04 smithi151 bash[23756]: audit 2024-03-04T15:59:02.751406+0000 mon.c (mon.2) 549 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:04 smithi151 bash[23756]: audit 2024-03-04T15:59:02.752306+0000 mon.a (mon.0) 2457 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:04 smithi151 bash[23756]: cluster 2024-03-04T15:59:02.936117+0000 mgr.y (mgr.24428) 374 : cluster 0 pgmap v536: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:59:04.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:04 smithi151 bash[23756]: audit 2024-03-04T15:59:03.653638+0000 mon.a (mon.0) 2458 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:59:04.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:04 smithi117 bash[21480]: audit 2024-03-04T15:59:02.649969+0000 mon.a (mon.0) 2453 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-04T15:59:04.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:04 smithi117 bash[21480]: audit 2024-03-04T15:59:02.650256+0000 mon.a (mon.0) 2454 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi117-63923-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:59:04.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:04 smithi117 bash[21480]: cluster 2024-03-04T15:59:02.658404+0000 mon.a (mon.0) 2455 : cluster 0 osdmap e353: 8 total, 8 up, 8 in 2024-03-04T15:59:04.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:04 smithi117 bash[21480]: audit 2024-03-04T15:59:02.660353+0000 mon.c (mon.2) 548 : audit 1 from='client.? 172.21.15.117:0/4033383151' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi117-63923-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:04.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:04 smithi117 bash[21480]: audit 2024-03-04T15:59:02.664889+0000 mon.a (mon.0) 2456 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi117-63923-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:04.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:04 smithi117 bash[21480]: audit 2024-03-04T15:59:02.751406+0000 mon.c (mon.2) 549 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:04.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:04 smithi117 bash[21480]: audit 2024-03-04T15:59:02.752306+0000 mon.a (mon.0) 2457 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:04.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:04 smithi117 bash[21480]: cluster 2024-03-04T15:59:02.936117+0000 mgr.y (mgr.24428) 374 : cluster 0 pgmap v536: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:59:04.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:04 smithi117 bash[21480]: audit 2024-03-04T15:59:03.653638+0000 mon.a (mon.0) 2458 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:59:04.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:04 smithi117 bash[28843]: audit 2024-03-04T15:59:02.649969+0000 mon.a (mon.0) 2453 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-04T15:59:04.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:04 smithi117 bash[28843]: audit 2024-03-04T15:59:02.650256+0000 mon.a (mon.0) 2454 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi117-63923-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:59:04.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:04 smithi117 bash[28843]: cluster 2024-03-04T15:59:02.658404+0000 mon.a (mon.0) 2455 : cluster 0 osdmap e353: 8 total, 8 up, 8 in 2024-03-04T15:59:04.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:04 smithi117 bash[28843]: audit 2024-03-04T15:59:02.660353+0000 mon.c (mon.2) 548 : audit 1 from='client.? 172.21.15.117:0/4033383151' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi117-63923-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:04.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:04 smithi117 bash[28843]: audit 2024-03-04T15:59:02.664889+0000 mon.a (mon.0) 2456 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi117-63923-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:04.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:04 smithi117 bash[28843]: audit 2024-03-04T15:59:02.751406+0000 mon.c (mon.2) 549 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:04.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:04 smithi117 bash[28843]: audit 2024-03-04T15:59:02.752306+0000 mon.a (mon.0) 2457 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:04.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:04 smithi117 bash[28843]: cluster 2024-03-04T15:59:02.936117+0000 mgr.y (mgr.24428) 374 : cluster 0 pgmap v536: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:59:04.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:04 smithi117 bash[28843]: audit 2024-03-04T15:59:03.653638+0000 mon.a (mon.0) 2458 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:59:05.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:05 smithi151 bash[23756]: audit 2024-03-04T15:59:03.666568+0000 mon.c (mon.2) 550 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47"}]: dispatch 2024-03-04T15:59:05.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:05 smithi151 bash[23756]: cluster 2024-03-04T15:59:04.111293+0000 mon.a (mon.0) 2459 : cluster 0 osdmap e354: 8 total, 8 up, 8 in 2024-03-04T15:59:05.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:05 smithi151 bash[23756]: audit 2024-03-04T15:59:04.112861+0000 mon.a (mon.0) 2460 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47"}]: dispatch 2024-03-04T15:59:05.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:05 smithi151 bash[23756]: audit 2024-03-04T15:59:04.656955+0000 mon.a (mon.0) 2461 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi117-63923-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi117-63923-63"}]': finished 2024-03-04T15:59:05.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:05 smithi151 bash[23756]: audit 2024-03-04T15:59:04.657533+0000 mon.a (mon.0) 2462 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47"}]': finished 2024-03-04T15:59:05.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:05 smithi151 bash[23756]: cluster 2024-03-04T15:59:04.665239+0000 mon.a (mon.0) 2463 : cluster 0 osdmap e355: 8 total, 8 up, 8 in 2024-03-04T15:59:05.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:05 smithi151 bash[23756]: audit 2024-03-04T15:59:04.716011+0000 mon.c (mon.2) 551 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:05.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:05 smithi151 bash[23756]: audit 2024-03-04T15:59:04.716560+0000 mon.a (mon.0) 2464 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:05.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:05 smithi151 bash[23756]: audit 2024-03-04T15:59:04.717499+0000 mon.c (mon.2) 552 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47"}]: dispatch 2024-03-04T15:59:05.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:05 smithi151 bash[23756]: audit 2024-03-04T15:59:04.718020+0000 mon.a (mon.0) 2465 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47"}]: dispatch 2024-03-04T15:59:05.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:05 smithi117 bash[21480]: audit 2024-03-04T15:59:03.666568+0000 mon.c (mon.2) 550 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47"}]: dispatch 2024-03-04T15:59:05.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:05 smithi117 bash[21480]: cluster 2024-03-04T15:59:04.111293+0000 mon.a (mon.0) 2459 : cluster 0 osdmap e354: 8 total, 8 up, 8 in 2024-03-04T15:59:05.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:05 smithi117 bash[21480]: audit 2024-03-04T15:59:04.112861+0000 mon.a (mon.0) 2460 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47"}]: dispatch 2024-03-04T15:59:05.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:05 smithi117 bash[21480]: audit 2024-03-04T15:59:04.656955+0000 mon.a (mon.0) 2461 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi117-63923-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi117-63923-63"}]': finished 2024-03-04T15:59:05.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:05 smithi117 bash[21480]: audit 2024-03-04T15:59:04.657533+0000 mon.a (mon.0) 2462 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47"}]': finished 2024-03-04T15:59:05.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:05 smithi117 bash[21480]: cluster 2024-03-04T15:59:04.665239+0000 mon.a (mon.0) 2463 : cluster 0 osdmap e355: 8 total, 8 up, 8 in 2024-03-04T15:59:05.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:05 smithi117 bash[21480]: audit 2024-03-04T15:59:04.716011+0000 mon.c (mon.2) 551 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:05.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:05 smithi117 bash[21480]: audit 2024-03-04T15:59:04.716560+0000 mon.a (mon.0) 2464 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:05.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:05 smithi117 bash[21480]: audit 2024-03-04T15:59:04.717499+0000 mon.c (mon.2) 552 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47"}]: dispatch 2024-03-04T15:59:05.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:05 smithi117 bash[21480]: audit 2024-03-04T15:59:04.718020+0000 mon.a (mon.0) 2465 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47"}]: dispatch 2024-03-04T15:59:05.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:05 smithi117 bash[28843]: audit 2024-03-04T15:59:03.666568+0000 mon.c (mon.2) 550 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47"}]: dispatch 2024-03-04T15:59:05.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:05 smithi117 bash[28843]: cluster 2024-03-04T15:59:04.111293+0000 mon.a (mon.0) 2459 : cluster 0 osdmap e354: 8 total, 8 up, 8 in 2024-03-04T15:59:05.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:05 smithi117 bash[28843]: audit 2024-03-04T15:59:04.112861+0000 mon.a (mon.0) 2460 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47"}]: dispatch 2024-03-04T15:59:05.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:05 smithi117 bash[28843]: audit 2024-03-04T15:59:04.656955+0000 mon.a (mon.0) 2461 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi117-63923-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi117-63923-63"}]': finished 2024-03-04T15:59:05.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:05 smithi117 bash[28843]: audit 2024-03-04T15:59:04.657533+0000 mon.a (mon.0) 2462 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47"}]': finished 2024-03-04T15:59:05.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:05 smithi117 bash[28843]: cluster 2024-03-04T15:59:04.665239+0000 mon.a (mon.0) 2463 : cluster 0 osdmap e355: 8 total, 8 up, 8 in 2024-03-04T15:59:05.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:05 smithi117 bash[28843]: audit 2024-03-04T15:59:04.716011+0000 mon.c (mon.2) 551 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:05.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:05 smithi117 bash[28843]: audit 2024-03-04T15:59:04.716560+0000 mon.a (mon.0) 2464 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:05.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:05 smithi117 bash[28843]: audit 2024-03-04T15:59:04.717499+0000 mon.c (mon.2) 552 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47"}]: dispatch 2024-03-04T15:59:05.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:05 smithi117 bash[28843]: audit 2024-03-04T15:59:04.718020+0000 mon.a (mon.0) 2465 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-47"}]: dispatch 2024-03-04T15:59:06.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:06 smithi151 bash[23756]: cluster 2024-03-04T15:59:04.937012+0000 mgr.y (mgr.24428) 375 : cluster 0 pgmap v539: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:59:06.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:06 smithi151 bash[23756]: cluster 2024-03-04T15:59:05.656731+0000 mon.a (mon.0) 2466 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:06.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:06 smithi151 bash[23756]: cluster 2024-03-04T15:59:05.675572+0000 mon.a (mon.0) 2467 : cluster 0 osdmap e356: 8 total, 8 up, 8 in 2024-03-04T15:59:06.567 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:59:06 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:59:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:59:06.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:06 smithi117 bash[21480]: cluster 2024-03-04T15:59:04.937012+0000 mgr.y (mgr.24428) 375 : cluster 0 pgmap v539: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:59:06.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:06 smithi117 bash[21480]: cluster 2024-03-04T15:59:05.656731+0000 mon.a (mon.0) 2466 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:06.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:06 smithi117 bash[21480]: cluster 2024-03-04T15:59:05.675572+0000 mon.a (mon.0) 2467 : cluster 0 osdmap e356: 8 total, 8 up, 8 in 2024-03-04T15:59:06.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:06 smithi117 bash[28843]: cluster 2024-03-04T15:59:04.937012+0000 mgr.y (mgr.24428) 375 : cluster 0 pgmap v539: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:59:06.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:06 smithi117 bash[28843]: cluster 2024-03-04T15:59:05.656731+0000 mon.a (mon.0) 2466 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:06.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:06 smithi117 bash[28843]: cluster 2024-03-04T15:59:05.675572+0000 mon.a (mon.0) 2467 : cluster 0 osdmap e356: 8 total, 8 up, 8 in 2024-03-04T15:59:07.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:59:06 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:59:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:07 smithi151 bash[23756]: cluster 2024-03-04T15:59:06.679655+0000 mon.a (mon.0) 2468 : cluster 0 osdmap e357: 8 total, 8 up, 8 in 2024-03-04T15:59:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:07 smithi151 bash[23756]: audit 2024-03-04T15:59:06.682812+0000 mon.c (mon.2) 553 : audit 1 from='client.? 172.21.15.117:0/4033383151' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:07 smithi151 bash[23756]: audit 2024-03-04T15:59:06.683593+0000 mon.c (mon.2) 554 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:07 smithi151 bash[23756]: audit 2024-03-04T15:59:06.687473+0000 mon.a (mon.0) 2469 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:07 smithi151 bash[23756]: audit 2024-03-04T15:59:06.688224+0000 mon.a (mon.0) 2470 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:07 smithi151 bash[23756]: audit 2024-03-04T15:59:06.731325+0000 mgr.y (mgr.24428) 376 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:59:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:07 smithi151 bash[23756]: cluster 2024-03-04T15:59:06.937845+0000 mgr.y (mgr.24428) 377 : cluster 0 pgmap v542: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-04T15:59:08.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:07 smithi117 bash[21480]: cluster 2024-03-04T15:59:06.679655+0000 mon.a (mon.0) 2468 : cluster 0 osdmap e357: 8 total, 8 up, 8 in 2024-03-04T15:59:08.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:07 smithi117 bash[21480]: audit 2024-03-04T15:59:06.682812+0000 mon.c (mon.2) 553 : audit 1 from='client.? 172.21.15.117:0/4033383151' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:08.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:07 smithi117 bash[21480]: audit 2024-03-04T15:59:06.683593+0000 mon.c (mon.2) 554 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:08.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:07 smithi117 bash[21480]: audit 2024-03-04T15:59:06.687473+0000 mon.a (mon.0) 2469 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:08.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:07 smithi117 bash[21480]: audit 2024-03-04T15:59:06.688224+0000 mon.a (mon.0) 2470 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:08.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:07 smithi117 bash[21480]: audit 2024-03-04T15:59:06.731325+0000 mgr.y (mgr.24428) 376 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:59:08.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:07 smithi117 bash[21480]: cluster 2024-03-04T15:59:06.937845+0000 mgr.y (mgr.24428) 377 : cluster 0 pgmap v542: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-04T15:59:08.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:07 smithi117 bash[28843]: cluster 2024-03-04T15:59:06.679655+0000 mon.a (mon.0) 2468 : cluster 0 osdmap e357: 8 total, 8 up, 8 in 2024-03-04T15:59:08.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:07 smithi117 bash[28843]: audit 2024-03-04T15:59:06.682812+0000 mon.c (mon.2) 553 : audit 1 from='client.? 172.21.15.117:0/4033383151' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:08.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:07 smithi117 bash[28843]: audit 2024-03-04T15:59:06.683593+0000 mon.c (mon.2) 554 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:08.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:07 smithi117 bash[28843]: audit 2024-03-04T15:59:06.687473+0000 mon.a (mon.0) 2469 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:08.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:07 smithi117 bash[28843]: audit 2024-03-04T15:59:06.688224+0000 mon.a (mon.0) 2470 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:08.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:07 smithi117 bash[28843]: audit 2024-03-04T15:59:06.731325+0000 mgr.y (mgr.24428) 376 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:59:08.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:07 smithi117 bash[28843]: cluster 2024-03-04T15:59:06.937845+0000 mgr.y (mgr.24428) 377 : cluster 0 pgmap v542: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-04T15:59:09.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:08 smithi117 bash[21480]: audit 2024-03-04T15:59:07.682616+0000 mon.a (mon.0) 2471 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi117-63923-63"}]': finished 2024-03-04T15:59:09.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:08 smithi117 bash[21480]: audit 2024-03-04T15:59:07.682808+0000 mon.a (mon.0) 2472 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:59:09.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:08 smithi117 bash[21480]: cluster 2024-03-04T15:59:07.695065+0000 mon.a (mon.0) 2473 : cluster 0 osdmap e358: 8 total, 8 up, 8 in 2024-03-04T15:59:09.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:08 smithi117 bash[21480]: audit 2024-03-04T15:59:07.695974+0000 mon.c (mon.2) 555 : audit 1 from='client.? 172.21.15.117:0/4033383151' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:09.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:08 smithi117 bash[21480]: audit 2024-03-04T15:59:07.700044+0000 mon.a (mon.0) 2474 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:09.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:08 smithi117 bash[21480]: audit 2024-03-04T15:59:07.733236+0000 mon.c (mon.2) 556 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:59:09.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:08 smithi117 bash[21480]: audit 2024-03-04T15:59:07.735114+0000 mon.a (mon.0) 2475 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:59:09.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:08 smithi117 bash[28843]: audit 2024-03-04T15:59:07.682616+0000 mon.a (mon.0) 2471 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi117-63923-63"}]': finished 2024-03-04T15:59:09.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:08 smithi117 bash[28843]: audit 2024-03-04T15:59:07.682808+0000 mon.a (mon.0) 2472 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:59:09.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:08 smithi117 bash[28843]: cluster 2024-03-04T15:59:07.695065+0000 mon.a (mon.0) 2473 : cluster 0 osdmap e358: 8 total, 8 up, 8 in 2024-03-04T15:59:09.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:08 smithi117 bash[28843]: audit 2024-03-04T15:59:07.695974+0000 mon.c (mon.2) 555 : audit 1 from='client.? 172.21.15.117:0/4033383151' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:09.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:08 smithi117 bash[28843]: audit 2024-03-04T15:59:07.700044+0000 mon.a (mon.0) 2474 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:09.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:08 smithi117 bash[28843]: audit 2024-03-04T15:59:07.733236+0000 mon.c (mon.2) 556 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:59:09.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:08 smithi117 bash[28843]: audit 2024-03-04T15:59:07.735114+0000 mon.a (mon.0) 2475 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:59:09.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:08 smithi151 bash[23756]: audit 2024-03-04T15:59:07.682616+0000 mon.a (mon.0) 2471 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi117-63923-63"}]': finished 2024-03-04T15:59:09.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:08 smithi151 bash[23756]: audit 2024-03-04T15:59:07.682808+0000 mon.a (mon.0) 2472 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:59:09.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:08 smithi151 bash[23756]: cluster 2024-03-04T15:59:07.695065+0000 mon.a (mon.0) 2473 : cluster 0 osdmap e358: 8 total, 8 up, 8 in 2024-03-04T15:59:09.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:08 smithi151 bash[23756]: audit 2024-03-04T15:59:07.695974+0000 mon.c (mon.2) 555 : audit 1 from='client.? 172.21.15.117:0/4033383151' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:09.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:08 smithi151 bash[23756]: audit 2024-03-04T15:59:07.700044+0000 mon.a (mon.0) 2474 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi117-63923-63"}]: dispatch 2024-03-04T15:59:09.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:08 smithi151 bash[23756]: audit 2024-03-04T15:59:07.733236+0000 mon.c (mon.2) 556 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:59:09.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:08 smithi151 bash[23756]: audit 2024-03-04T15:59:07.735114+0000 mon.a (mon.0) 2475 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:59:10.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:09 smithi117 bash[28843]: audit 2024-03-04T15:59:08.694056+0000 mon.a (mon.0) 2476 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi117-63923-63"}]': finished 2024-03-04T15:59:10.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:09 smithi117 bash[28843]: audit 2024-03-04T15:59:08.694318+0000 mon.a (mon.0) 2477 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:59:10.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:09 smithi117 bash[28843]: cluster 2024-03-04T15:59:08.701732+0000 mon.a (mon.0) 2478 : cluster 0 osdmap e359: 8 total, 8 up, 8 in 2024-03-04T15:59:10.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:09 smithi117 bash[28843]: audit 2024-03-04T15:59:08.703844+0000 mon.c (mon.2) 557 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-49"}]: dispatch 2024-03-04T15:59:10.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:09 smithi117 bash[28843]: audit 2024-03-04T15:59:08.704829+0000 mon.a (mon.0) 2479 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-49"}]: dispatch 2024-03-04T15:59:10.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:09 smithi117 bash[28843]: audit 2024-03-04T15:59:08.740854+0000 mon.a (mon.0) 2480 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi117-63923-64"}]: dispatch 2024-03-04T15:59:10.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:09 smithi117 bash[28843]: audit 2024-03-04T15:59:08.741955+0000 mon.a (mon.0) 2481 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi117-63923-64"}]: dispatch 2024-03-04T15:59:10.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:09 smithi117 bash[28843]: audit 2024-03-04T15:59:08.742466+0000 mon.a (mon.0) 2482 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi117-63923-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:10.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:09 smithi117 bash[28843]: cluster 2024-03-04T15:59:08.938666+0000 mgr.y (mgr.24428) 378 : cluster 0 pgmap v545: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-04T15:59:10.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:09 smithi117 bash[21480]: audit 2024-03-04T15:59:08.694056+0000 mon.a (mon.0) 2476 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi117-63923-63"}]': finished 2024-03-04T15:59:10.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:09 smithi117 bash[21480]: audit 2024-03-04T15:59:08.694318+0000 mon.a (mon.0) 2477 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:59:10.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:09 smithi117 bash[21480]: cluster 2024-03-04T15:59:08.701732+0000 mon.a (mon.0) 2478 : cluster 0 osdmap e359: 8 total, 8 up, 8 in 2024-03-04T15:59:10.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:09 smithi117 bash[21480]: audit 2024-03-04T15:59:08.703844+0000 mon.c (mon.2) 557 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-49"}]: dispatch 2024-03-04T15:59:10.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:09 smithi117 bash[21480]: audit 2024-03-04T15:59:08.704829+0000 mon.a (mon.0) 2479 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-49"}]: dispatch 2024-03-04T15:59:10.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:09 smithi117 bash[21480]: audit 2024-03-04T15:59:08.740854+0000 mon.a (mon.0) 2480 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi117-63923-64"}]: dispatch 2024-03-04T15:59:10.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:09 smithi117 bash[21480]: audit 2024-03-04T15:59:08.741955+0000 mon.a (mon.0) 2481 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi117-63923-64"}]: dispatch 2024-03-04T15:59:10.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:09 smithi117 bash[21480]: audit 2024-03-04T15:59:08.742466+0000 mon.a (mon.0) 2482 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi117-63923-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:10.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:09 smithi117 bash[21480]: cluster 2024-03-04T15:59:08.938666+0000 mgr.y (mgr.24428) 378 : cluster 0 pgmap v545: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-04T15:59:10.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:09 smithi151 bash[23756]: audit 2024-03-04T15:59:08.694056+0000 mon.a (mon.0) 2476 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi117-63923-63"}]': finished 2024-03-04T15:59:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:09 smithi151 bash[23756]: audit 2024-03-04T15:59:08.694318+0000 mon.a (mon.0) 2477 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:59:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:09 smithi151 bash[23756]: cluster 2024-03-04T15:59:08.701732+0000 mon.a (mon.0) 2478 : cluster 0 osdmap e359: 8 total, 8 up, 8 in 2024-03-04T15:59:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:09 smithi151 bash[23756]: audit 2024-03-04T15:59:08.703844+0000 mon.c (mon.2) 557 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-49"}]: dispatch 2024-03-04T15:59:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:09 smithi151 bash[23756]: audit 2024-03-04T15:59:08.704829+0000 mon.a (mon.0) 2479 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-49"}]: dispatch 2024-03-04T15:59:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:09 smithi151 bash[23756]: audit 2024-03-04T15:59:08.740854+0000 mon.a (mon.0) 2480 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi117-63923-64"}]: dispatch 2024-03-04T15:59:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:09 smithi151 bash[23756]: audit 2024-03-04T15:59:08.741955+0000 mon.a (mon.0) 2481 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi117-63923-64"}]: dispatch 2024-03-04T15:59:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:09 smithi151 bash[23756]: audit 2024-03-04T15:59:08.742466+0000 mon.a (mon.0) 2482 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi117-63923-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:09 smithi151 bash[23756]: cluster 2024-03-04T15:59:08.938666+0000 mgr.y (mgr.24428) 378 : cluster 0 pgmap v545: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-04T15:59:11.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:10 smithi117 bash[28843]: audit 2024-03-04T15:59:09.703120+0000 mon.a (mon.0) 2483 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-49"}]': finished 2024-03-04T15:59:11.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:10 smithi117 bash[28843]: audit 2024-03-04T15:59:09.703356+0000 mon.a (mon.0) 2484 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi117-63923-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:59:11.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:10 smithi117 bash[28843]: audit 2024-03-04T15:59:09.711675+0000 mon.c (mon.2) 558 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-49", "mode": "readproxy"}]: dispatch 2024-03-04T15:59:11.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:10 smithi117 bash[28843]: cluster 2024-03-04T15:59:09.712298+0000 mon.a (mon.0) 2485 : cluster 0 osdmap e360: 8 total, 8 up, 8 in 2024-03-04T15:59:11.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:10 smithi117 bash[28843]: audit 2024-03-04T15:59:09.713878+0000 mon.a (mon.0) 2486 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi117-63923-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi117-63923-64"}]: dispatch 2024-03-04T15:59:11.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:10 smithi117 bash[28843]: audit 2024-03-04T15:59:09.714856+0000 mon.a (mon.0) 2487 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-49", "mode": "readproxy"}]: dispatch 2024-03-04T15:59:11.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:10 smithi117 bash[21480]: audit 2024-03-04T15:59:09.703120+0000 mon.a (mon.0) 2483 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-49"}]': finished 2024-03-04T15:59:11.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:10 smithi117 bash[21480]: audit 2024-03-04T15:59:09.703356+0000 mon.a (mon.0) 2484 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi117-63923-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:59:11.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:10 smithi117 bash[21480]: audit 2024-03-04T15:59:09.711675+0000 mon.c (mon.2) 558 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-49", "mode": "readproxy"}]: dispatch 2024-03-04T15:59:11.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:10 smithi117 bash[21480]: cluster 2024-03-04T15:59:09.712298+0000 mon.a (mon.0) 2485 : cluster 0 osdmap e360: 8 total, 8 up, 8 in 2024-03-04T15:59:11.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:10 smithi117 bash[21480]: audit 2024-03-04T15:59:09.713878+0000 mon.a (mon.0) 2486 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi117-63923-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi117-63923-64"}]: dispatch 2024-03-04T15:59:11.070 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:10 smithi117 bash[21480]: audit 2024-03-04T15:59:09.714856+0000 mon.a (mon.0) 2487 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-49", "mode": "readproxy"}]: dispatch 2024-03-04T15:59:11.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:10 smithi151 bash[23756]: audit 2024-03-04T15:59:09.703120+0000 mon.a (mon.0) 2483 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-49"}]': finished 2024-03-04T15:59:11.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:10 smithi151 bash[23756]: audit 2024-03-04T15:59:09.703356+0000 mon.a (mon.0) 2484 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi117-63923-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:59:11.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:10 smithi151 bash[23756]: audit 2024-03-04T15:59:09.711675+0000 mon.c (mon.2) 558 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-49", "mode": "readproxy"}]: dispatch 2024-03-04T15:59:11.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:10 smithi151 bash[23756]: cluster 2024-03-04T15:59:09.712298+0000 mon.a (mon.0) 2485 : cluster 0 osdmap e360: 8 total, 8 up, 8 in 2024-03-04T15:59:11.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:10 smithi151 bash[23756]: audit 2024-03-04T15:59:09.713878+0000 mon.a (mon.0) 2486 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi117-63923-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi117-63923-64"}]: dispatch 2024-03-04T15:59:11.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:10 smithi151 bash[23756]: audit 2024-03-04T15:59:09.714856+0000 mon.a (mon.0) 2487 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-49", "mode": "readproxy"}]: dispatch 2024-03-04T15:59:12.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:11 smithi117 bash[28843]: cluster 2024-03-04T15:59:10.704849+0000 mon.a (mon.0) 2488 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:59:12.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:11 smithi117 bash[28843]: audit 2024-03-04T15:59:10.715683+0000 mon.a (mon.0) 2489 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-49", "mode": "readproxy"}]': finished 2024-03-04T15:59:12.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:11 smithi117 bash[28843]: cluster 2024-03-04T15:59:10.939429+0000 mgr.y (mgr.24428) 379 : cluster 0 pgmap v548: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:59:12.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:11 smithi117 bash[28843]: cluster 2024-03-04T15:59:11.171467+0000 mon.a (mon.0) 2490 : cluster 0 osdmap e361: 8 total, 8 up, 8 in 2024-03-04T15:59:12.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:11 smithi117 bash[21480]: cluster 2024-03-04T15:59:10.704849+0000 mon.a (mon.0) 2488 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:59:12.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:11 smithi117 bash[21480]: audit 2024-03-04T15:59:10.715683+0000 mon.a (mon.0) 2489 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-49", "mode": "readproxy"}]': finished 2024-03-04T15:59:12.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:11 smithi117 bash[21480]: cluster 2024-03-04T15:59:10.939429+0000 mgr.y (mgr.24428) 379 : cluster 0 pgmap v548: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:59:12.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:11 smithi117 bash[21480]: cluster 2024-03-04T15:59:11.171467+0000 mon.a (mon.0) 2490 : cluster 0 osdmap e361: 8 total, 8 up, 8 in 2024-03-04T15:59:12.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:11 smithi151 bash[23756]: cluster 2024-03-04T15:59:10.704849+0000 mon.a (mon.0) 2488 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:59:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:11 smithi151 bash[23756]: audit 2024-03-04T15:59:10.715683+0000 mon.a (mon.0) 2489 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-49", "mode": "readproxy"}]': finished 2024-03-04T15:59:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:11 smithi151 bash[23756]: cluster 2024-03-04T15:59:10.939429+0000 mgr.y (mgr.24428) 379 : cluster 0 pgmap v548: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:59:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:11 smithi151 bash[23756]: cluster 2024-03-04T15:59:11.171467+0000 mon.a (mon.0) 2490 : cluster 0 osdmap e361: 8 total, 8 up, 8 in 2024-03-04T15:59:13.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:12 smithi117 bash[21480]: audit 2024-03-04T15:59:11.721173+0000 mon.a (mon.0) 2491 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi117-63923-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi117-63923-64"}]': finished 2024-03-04T15:59:13.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:12 smithi117 bash[21480]: cluster 2024-03-04T15:59:11.728995+0000 mon.a (mon.0) 2492 : cluster 0 osdmap e362: 8 total, 8 up, 8 in 2024-03-04T15:59:13.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:12 smithi117 bash[28843]: audit 2024-03-04T15:59:11.721173+0000 mon.a (mon.0) 2491 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi117-63923-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi117-63923-64"}]': finished 2024-03-04T15:59:13.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:12 smithi117 bash[28843]: cluster 2024-03-04T15:59:11.728995+0000 mon.a (mon.0) 2492 : cluster 0 osdmap e362: 8 total, 8 up, 8 in 2024-03-04T15:59:13.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:12 smithi151 bash[23756]: audit 2024-03-04T15:59:11.721173+0000 mon.a (mon.0) 2491 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi117-63923-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi117-63923-64"}]': finished 2024-03-04T15:59:13.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:12 smithi151 bash[23756]: cluster 2024-03-04T15:59:11.728995+0000 mon.a (mon.0) 2492 : cluster 0 osdmap e362: 8 total, 8 up, 8 in 2024-03-04T15:59:14.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:13 smithi117 bash[28843]: cluster 2024-03-04T15:59:12.739557+0000 mon.a (mon.0) 2493 : cluster 0 osdmap e363: 8 total, 8 up, 8 in 2024-03-04T15:59:14.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:13 smithi117 bash[28843]: cluster 2024-03-04T15:59:12.940263+0000 mgr.y (mgr.24428) 380 : cluster 0 pgmap v551: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:59:14.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:13 smithi117 bash[28843]: cluster 2024-03-04T15:59:13.268612+0000 mon.a (mon.0) 2494 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:14.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:13 smithi117 bash[21480]: cluster 2024-03-04T15:59:12.739557+0000 mon.a (mon.0) 2493 : cluster 0 osdmap e363: 8 total, 8 up, 8 in 2024-03-04T15:59:14.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:13 smithi117 bash[21480]: cluster 2024-03-04T15:59:12.940263+0000 mgr.y (mgr.24428) 380 : cluster 0 pgmap v551: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:59:14.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:13 smithi117 bash[21480]: cluster 2024-03-04T15:59:13.268612+0000 mon.a (mon.0) 2494 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:14.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:13 smithi151 bash[23756]: cluster 2024-03-04T15:59:12.739557+0000 mon.a (mon.0) 2493 : cluster 0 osdmap e363: 8 total, 8 up, 8 in 2024-03-04T15:59:14.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:13 smithi151 bash[23756]: cluster 2024-03-04T15:59:12.940263+0000 mgr.y (mgr.24428) 380 : cluster 0 pgmap v551: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:59:14.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:13 smithi151 bash[23756]: cluster 2024-03-04T15:59:13.268612+0000 mon.a (mon.0) 2494 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:15.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:14 smithi117 bash[21480]: cluster 2024-03-04T15:59:13.752721+0000 mon.a (mon.0) 2495 : cluster 0 osdmap e364: 8 total, 8 up, 8 in 2024-03-04T15:59:15.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:14 smithi117 bash[21480]: audit 2024-03-04T15:59:13.756533+0000 mon.a (mon.0) 2496 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi117-63923-64"}]: dispatch 2024-03-04T15:59:15.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:14 smithi117 bash[21480]: audit 2024-03-04T15:59:13.909367+0000 mon.c (mon.2) 559 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:59:15.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:14 smithi117 bash[28843]: cluster 2024-03-04T15:59:13.752721+0000 mon.a (mon.0) 2495 : cluster 0 osdmap e364: 8 total, 8 up, 8 in 2024-03-04T15:59:15.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:14 smithi117 bash[28843]: audit 2024-03-04T15:59:13.756533+0000 mon.a (mon.0) 2496 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi117-63923-64"}]: dispatch 2024-03-04T15:59:15.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:14 smithi117 bash[28843]: audit 2024-03-04T15:59:13.909367+0000 mon.c (mon.2) 559 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:59:15.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:14 smithi151 bash[23756]: cluster 2024-03-04T15:59:13.752721+0000 mon.a (mon.0) 2495 : cluster 0 osdmap e364: 8 total, 8 up, 8 in 2024-03-04T15:59:15.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:14 smithi151 bash[23756]: audit 2024-03-04T15:59:13.756533+0000 mon.a (mon.0) 2496 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi117-63923-64"}]: dispatch 2024-03-04T15:59:15.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:14 smithi151 bash[23756]: audit 2024-03-04T15:59:13.909367+0000 mon.c (mon.2) 559 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:59:16.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:15 smithi117 bash[21480]: audit 2024-03-04T15:59:14.745228+0000 mon.a (mon.0) 2497 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi117-63923-64"}]': finished 2024-03-04T15:59:16.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:15 smithi117 bash[21480]: cluster 2024-03-04T15:59:14.754764+0000 mon.a (mon.0) 2498 : cluster 0 osdmap e365: 8 total, 8 up, 8 in 2024-03-04T15:59:16.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:15 smithi117 bash[21480]: audit 2024-03-04T15:59:14.757787+0000 mon.a (mon.0) 2499 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi117-63923-64"}]: dispatch 2024-03-04T15:59:16.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:15 smithi117 bash[21480]: cluster 2024-03-04T15:59:14.941186+0000 mgr.y (mgr.24428) 381 : cluster 0 pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:59:16.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:15 smithi117 bash[21480]: audit 2024-03-04T15:59:15.748580+0000 mon.a (mon.0) 2500 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi117-63923-64"}]': finished 2024-03-04T15:59:16.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:15 smithi117 bash[21480]: cluster 2024-03-04T15:59:15.756213+0000 mon.a (mon.0) 2501 : cluster 0 osdmap e366: 8 total, 8 up, 8 in 2024-03-04T15:59:16.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:15 smithi117 bash[28843]: audit 2024-03-04T15:59:14.745228+0000 mon.a (mon.0) 2497 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi117-63923-64"}]': finished 2024-03-04T15:59:16.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:15 smithi117 bash[28843]: cluster 2024-03-04T15:59:14.754764+0000 mon.a (mon.0) 2498 : cluster 0 osdmap e365: 8 total, 8 up, 8 in 2024-03-04T15:59:16.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:15 smithi117 bash[28843]: audit 2024-03-04T15:59:14.757787+0000 mon.a (mon.0) 2499 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi117-63923-64"}]: dispatch 2024-03-04T15:59:16.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:15 smithi117 bash[28843]: cluster 2024-03-04T15:59:14.941186+0000 mgr.y (mgr.24428) 381 : cluster 0 pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:59:16.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:15 smithi117 bash[28843]: audit 2024-03-04T15:59:15.748580+0000 mon.a (mon.0) 2500 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi117-63923-64"}]': finished 2024-03-04T15:59:16.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:15 smithi117 bash[28843]: cluster 2024-03-04T15:59:15.756213+0000 mon.a (mon.0) 2501 : cluster 0 osdmap e366: 8 total, 8 up, 8 in 2024-03-04T15:59:16.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:15 smithi151 bash[23756]: audit 2024-03-04T15:59:14.745228+0000 mon.a (mon.0) 2497 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi117-63923-64"}]': finished 2024-03-04T15:59:16.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:15 smithi151 bash[23756]: cluster 2024-03-04T15:59:14.754764+0000 mon.a (mon.0) 2498 : cluster 0 osdmap e365: 8 total, 8 up, 8 in 2024-03-04T15:59:16.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:15 smithi151 bash[23756]: audit 2024-03-04T15:59:14.757787+0000 mon.a (mon.0) 2499 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi117-63923-64"}]: dispatch 2024-03-04T15:59:16.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:15 smithi151 bash[23756]: cluster 2024-03-04T15:59:14.941186+0000 mgr.y (mgr.24428) 381 : cluster 0 pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T15:59:16.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:15 smithi151 bash[23756]: audit 2024-03-04T15:59:15.748580+0000 mon.a (mon.0) 2500 : audit 1 from='client.? 172.21.15.117:0/2919879545' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi117-63923-64"}]': finished 2024-03-04T15:59:16.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:15 smithi151 bash[23756]: cluster 2024-03-04T15:59:15.756213+0000 mon.a (mon.0) 2501 : cluster 0 osdmap e366: 8 total, 8 up, 8 in 2024-03-04T15:59:16.567 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:59:16 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:59:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:59:17.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:16 smithi117 bash[21480]: audit 2024-03-04T15:59:15.810476+0000 mon.c (mon.2) 560 : audit 1 from='client.? 172.21.15.117:0/1753464141' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:17.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:16 smithi117 bash[21480]: audit 2024-03-04T15:59:15.810857+0000 mon.a (mon.0) 2502 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:17.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:16 smithi117 bash[21480]: audit 2024-03-04T15:59:15.811698+0000 mon.c (mon.2) 561 : audit 1 from='client.? 172.21.15.117:0/1753464141' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:17.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:16 smithi117 bash[21480]: audit 2024-03-04T15:59:15.812016+0000 mon.a (mon.0) 2503 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:17.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:16 smithi117 bash[21480]: audit 2024-03-04T15:59:15.812533+0000 mon.c (mon.2) 562 : audit 1 from='client.? 172.21.15.117:0/1753464141' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi117-63923-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:17.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:16 smithi117 bash[21480]: audit 2024-03-04T15:59:15.812822+0000 mon.a (mon.0) 2504 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi117-63923-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:17.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:16 smithi117 bash[28843]: audit 2024-03-04T15:59:15.810476+0000 mon.c (mon.2) 560 : audit 1 from='client.? 172.21.15.117:0/1753464141' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:17.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:16 smithi117 bash[28843]: audit 2024-03-04T15:59:15.810857+0000 mon.a (mon.0) 2502 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:17.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:16 smithi117 bash[28843]: audit 2024-03-04T15:59:15.811698+0000 mon.c (mon.2) 561 : audit 1 from='client.? 172.21.15.117:0/1753464141' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:17.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:16 smithi117 bash[28843]: audit 2024-03-04T15:59:15.812016+0000 mon.a (mon.0) 2503 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:17.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:16 smithi117 bash[28843]: audit 2024-03-04T15:59:15.812533+0000 mon.c (mon.2) 562 : audit 1 from='client.? 172.21.15.117:0/1753464141' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi117-63923-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:17.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:16 smithi117 bash[28843]: audit 2024-03-04T15:59:15.812822+0000 mon.a (mon.0) 2504 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi117-63923-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:17.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:59:16 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:59:17.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:16 smithi151 bash[23756]: audit 2024-03-04T15:59:15.810476+0000 mon.c (mon.2) 560 : audit 1 from='client.? 172.21.15.117:0/1753464141' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:17.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:16 smithi151 bash[23756]: audit 2024-03-04T15:59:15.810857+0000 mon.a (mon.0) 2502 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:17.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:16 smithi151 bash[23756]: audit 2024-03-04T15:59:15.811698+0000 mon.c (mon.2) 561 : audit 1 from='client.? 172.21.15.117:0/1753464141' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:17.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:16 smithi151 bash[23756]: audit 2024-03-04T15:59:15.812016+0000 mon.a (mon.0) 2503 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:17.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:16 smithi151 bash[23756]: audit 2024-03-04T15:59:15.812533+0000 mon.c (mon.2) 562 : audit 1 from='client.? 172.21.15.117:0/1753464141' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi117-63923-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:17.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:16 smithi151 bash[23756]: audit 2024-03-04T15:59:15.812822+0000 mon.a (mon.0) 2504 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi117-63923-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:18.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:17 smithi117 bash[21480]: audit 2024-03-04T15:59:16.742782+0000 mgr.y (mgr.24428) 382 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:59:18.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:17 smithi117 bash[21480]: audit 2024-03-04T15:59:16.776254+0000 mon.a (mon.0) 2505 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi117-63923-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:59:18.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:17 smithi117 bash[21480]: cluster 2024-03-04T15:59:16.786128+0000 mon.a (mon.0) 2506 : cluster 0 osdmap e367: 8 total, 8 up, 8 in 2024-03-04T15:59:18.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:17 smithi117 bash[21480]: audit 2024-03-04T15:59:16.791008+0000 mon.c (mon.2) 563 : audit 1 from='client.? 172.21.15.117:0/1753464141' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi117-63923-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:18.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:17 smithi117 bash[21480]: audit 2024-03-04T15:59:16.801951+0000 mon.a (mon.0) 2507 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi117-63923-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:18.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:17 smithi117 bash[21480]: cluster 2024-03-04T15:59:16.942073+0000 mgr.y (mgr.24428) 383 : cluster 0 pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:59:18.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:17 smithi117 bash[28843]: audit 2024-03-04T15:59:16.742782+0000 mgr.y (mgr.24428) 382 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:59:18.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:17 smithi117 bash[28843]: audit 2024-03-04T15:59:16.776254+0000 mon.a (mon.0) 2505 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi117-63923-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:59:18.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:17 smithi117 bash[28843]: cluster 2024-03-04T15:59:16.786128+0000 mon.a (mon.0) 2506 : cluster 0 osdmap e367: 8 total, 8 up, 8 in 2024-03-04T15:59:18.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:17 smithi117 bash[28843]: audit 2024-03-04T15:59:16.791008+0000 mon.c (mon.2) 563 : audit 1 from='client.? 172.21.15.117:0/1753464141' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi117-63923-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:18.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:17 smithi117 bash[28843]: audit 2024-03-04T15:59:16.801951+0000 mon.a (mon.0) 2507 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi117-63923-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:18.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:17 smithi117 bash[28843]: cluster 2024-03-04T15:59:16.942073+0000 mgr.y (mgr.24428) 383 : cluster 0 pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:59:18.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:17 smithi151 bash[23756]: audit 2024-03-04T15:59:16.742782+0000 mgr.y (mgr.24428) 382 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:59:18.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:17 smithi151 bash[23756]: audit 2024-03-04T15:59:16.776254+0000 mon.a (mon.0) 2505 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi117-63923-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:59:18.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:17 smithi151 bash[23756]: cluster 2024-03-04T15:59:16.786128+0000 mon.a (mon.0) 2506 : cluster 0 osdmap e367: 8 total, 8 up, 8 in 2024-03-04T15:59:18.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:17 smithi151 bash[23756]: audit 2024-03-04T15:59:16.791008+0000 mon.c (mon.2) 563 : audit 1 from='client.? 172.21.15.117:0/1753464141' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi117-63923-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:18.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:17 smithi151 bash[23756]: audit 2024-03-04T15:59:16.801951+0000 mon.a (mon.0) 2507 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi117-63923-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:18.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:17 smithi151 bash[23756]: cluster 2024-03-04T15:59:16.942073+0000 mgr.y (mgr.24428) 383 : cluster 0 pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:59:19.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:18 smithi117 bash[21480]: cluster 2024-03-04T15:59:18.264506+0000 mon.a (mon.0) 2508 : cluster 0 osdmap e368: 8 total, 8 up, 8 in 2024-03-04T15:59:19.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:18 smithi117 bash[28843]: cluster 2024-03-04T15:59:18.264506+0000 mon.a (mon.0) 2508 : cluster 0 osdmap e368: 8 total, 8 up, 8 in 2024-03-04T15:59:19.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:18 smithi151 bash[23756]: cluster 2024-03-04T15:59:18.264506+0000 mon.a (mon.0) 2508 : cluster 0 osdmap e368: 8 total, 8 up, 8 in 2024-03-04T15:59:20.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:19 smithi151 bash[23756]: audit 2024-03-04T15:59:18.815683+0000 mon.a (mon.0) 2509 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi117-63923-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi117-63923-65"}]': finished 2024-03-04T15:59:20.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:19 smithi151 bash[23756]: cluster 2024-03-04T15:59:18.824396+0000 mon.a (mon.0) 2510 : cluster 0 osdmap e369: 8 total, 8 up, 8 in 2024-03-04T15:59:20.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:19 smithi151 bash[23756]: cluster 2024-03-04T15:59:18.942944+0000 mgr.y (mgr.24428) 384 : cluster 0 pgmap v560: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:59:20.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:19 smithi117 bash[21480]: audit 2024-03-04T15:59:18.815683+0000 mon.a (mon.0) 2509 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi117-63923-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi117-63923-65"}]': finished 2024-03-04T15:59:20.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:19 smithi117 bash[21480]: cluster 2024-03-04T15:59:18.824396+0000 mon.a (mon.0) 2510 : cluster 0 osdmap e369: 8 total, 8 up, 8 in 2024-03-04T15:59:20.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:19 smithi117 bash[21480]: cluster 2024-03-04T15:59:18.942944+0000 mgr.y (mgr.24428) 384 : cluster 0 pgmap v560: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:59:20.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:19 smithi117 bash[28843]: audit 2024-03-04T15:59:18.815683+0000 mon.a (mon.0) 2509 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi117-63923-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi117-63923-65"}]': finished 2024-03-04T15:59:20.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:19 smithi117 bash[28843]: cluster 2024-03-04T15:59:18.824396+0000 mon.a (mon.0) 2510 : cluster 0 osdmap e369: 8 total, 8 up, 8 in 2024-03-04T15:59:20.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:19 smithi117 bash[28843]: cluster 2024-03-04T15:59:18.942944+0000 mgr.y (mgr.24428) 384 : cluster 0 pgmap v560: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:59:21.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:20 smithi151 bash[23756]: cluster 2024-03-04T15:59:19.816828+0000 mon.a (mon.0) 2511 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:21.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:20 smithi151 bash[23756]: cluster 2024-03-04T15:59:19.827433+0000 mon.a (mon.0) 2512 : cluster 0 osdmap e370: 8 total, 8 up, 8 in 2024-03-04T15:59:21.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:20 smithi117 bash[21480]: cluster 2024-03-04T15:59:19.816828+0000 mon.a (mon.0) 2511 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:21.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:20 smithi117 bash[21480]: cluster 2024-03-04T15:59:19.827433+0000 mon.a (mon.0) 2512 : cluster 0 osdmap e370: 8 total, 8 up, 8 in 2024-03-04T15:59:21.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:20 smithi117 bash[28843]: cluster 2024-03-04T15:59:19.816828+0000 mon.a (mon.0) 2511 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:21.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:20 smithi117 bash[28843]: cluster 2024-03-04T15:59:19.827433+0000 mon.a (mon.0) 2512 : cluster 0 osdmap e370: 8 total, 8 up, 8 in 2024-03-04T15:59:22.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:21 smithi151 bash[23756]: cluster 2024-03-04T15:59:20.863267+0000 mon.a (mon.0) 2513 : cluster 0 osdmap e371: 8 total, 8 up, 8 in 2024-03-04T15:59:22.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:21 smithi151 bash[23756]: audit 2024-03-04T15:59:20.868960+0000 mon.c (mon.2) 564 : audit 1 from='client.? 172.21.15.117:0/1753464141' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:22.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:21 smithi151 bash[23756]: audit 2024-03-04T15:59:20.869930+0000 mon.a (mon.0) 2514 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:22.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:21 smithi151 bash[23756]: cluster 2024-03-04T15:59:20.943729+0000 mgr.y (mgr.24428) 385 : cluster 0 pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:59:22.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:21 smithi151 bash[23756]: audit 2024-03-04T15:59:21.180713+0000 mon.c (mon.2) 565 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:22.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:21 smithi151 bash[23756]: audit 2024-03-04T15:59:21.181516+0000 mon.a (mon.0) 2515 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:22.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:21 smithi117 bash[21480]: cluster 2024-03-04T15:59:20.863267+0000 mon.a (mon.0) 2513 : cluster 0 osdmap e371: 8 total, 8 up, 8 in 2024-03-04T15:59:22.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:21 smithi117 bash[21480]: audit 2024-03-04T15:59:20.868960+0000 mon.c (mon.2) 564 : audit 1 from='client.? 172.21.15.117:0/1753464141' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:22.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:21 smithi117 bash[21480]: audit 2024-03-04T15:59:20.869930+0000 mon.a (mon.0) 2514 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:22.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:21 smithi117 bash[21480]: cluster 2024-03-04T15:59:20.943729+0000 mgr.y (mgr.24428) 385 : cluster 0 pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:59:22.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:21 smithi117 bash[21480]: audit 2024-03-04T15:59:21.180713+0000 mon.c (mon.2) 565 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:22.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:21 smithi117 bash[21480]: audit 2024-03-04T15:59:21.181516+0000 mon.a (mon.0) 2515 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:22.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:21 smithi117 bash[28843]: cluster 2024-03-04T15:59:20.863267+0000 mon.a (mon.0) 2513 : cluster 0 osdmap e371: 8 total, 8 up, 8 in 2024-03-04T15:59:22.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:21 smithi117 bash[28843]: audit 2024-03-04T15:59:20.868960+0000 mon.c (mon.2) 564 : audit 1 from='client.? 172.21.15.117:0/1753464141' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:22.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:21 smithi117 bash[28843]: audit 2024-03-04T15:59:20.869930+0000 mon.a (mon.0) 2514 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:22.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:21 smithi117 bash[28843]: cluster 2024-03-04T15:59:20.943729+0000 mgr.y (mgr.24428) 385 : cluster 0 pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:59:22.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:21 smithi117 bash[28843]: audit 2024-03-04T15:59:21.180713+0000 mon.c (mon.2) 565 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:22.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:21 smithi117 bash[28843]: audit 2024-03-04T15:59:21.181516+0000 mon.a (mon.0) 2515 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:23.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:22 smithi151 bash[23756]: audit 2024-03-04T15:59:21.859298+0000 mon.a (mon.0) 2516 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi117-63923-65"}]': finished 2024-03-04T15:59:23.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:22 smithi151 bash[23756]: audit 2024-03-04T15:59:21.859542+0000 mon.a (mon.0) 2517 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:59:23.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:22 smithi151 bash[23756]: cluster 2024-03-04T15:59:21.868304+0000 mon.a (mon.0) 2518 : cluster 0 osdmap e372: 8 total, 8 up, 8 in 2024-03-04T15:59:23.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:22 smithi151 bash[23756]: audit 2024-03-04T15:59:21.872060+0000 mon.c (mon.2) 566 : audit 1 from='client.? 172.21.15.117:0/1753464141' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:23.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:22 smithi151 bash[23756]: audit 2024-03-04T15:59:21.872658+0000 mon.c (mon.2) 567 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49"}]: dispatch 2024-03-04T15:59:23.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:22 smithi151 bash[23756]: audit 2024-03-04T15:59:21.885122+0000 mon.a (mon.0) 2519 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:23.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:22 smithi151 bash[23756]: audit 2024-03-04T15:59:21.885846+0000 mon.a (mon.0) 2520 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49"}]: dispatch 2024-03-04T15:59:23.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:22 smithi117 bash[21480]: audit 2024-03-04T15:59:21.859298+0000 mon.a (mon.0) 2516 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi117-63923-65"}]': finished 2024-03-04T15:59:23.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:22 smithi117 bash[21480]: audit 2024-03-04T15:59:21.859542+0000 mon.a (mon.0) 2517 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:59:23.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:22 smithi117 bash[21480]: cluster 2024-03-04T15:59:21.868304+0000 mon.a (mon.0) 2518 : cluster 0 osdmap e372: 8 total, 8 up, 8 in 2024-03-04T15:59:23.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:22 smithi117 bash[21480]: audit 2024-03-04T15:59:21.872060+0000 mon.c (mon.2) 566 : audit 1 from='client.? 172.21.15.117:0/1753464141' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:23.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:22 smithi117 bash[21480]: audit 2024-03-04T15:59:21.872658+0000 mon.c (mon.2) 567 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49"}]: dispatch 2024-03-04T15:59:23.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:22 smithi117 bash[21480]: audit 2024-03-04T15:59:21.885122+0000 mon.a (mon.0) 2519 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:23.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:22 smithi117 bash[21480]: audit 2024-03-04T15:59:21.885846+0000 mon.a (mon.0) 2520 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49"}]: dispatch 2024-03-04T15:59:23.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:22 smithi117 bash[28843]: audit 2024-03-04T15:59:21.859298+0000 mon.a (mon.0) 2516 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi117-63923-65"}]': finished 2024-03-04T15:59:23.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:22 smithi117 bash[28843]: audit 2024-03-04T15:59:21.859542+0000 mon.a (mon.0) 2517 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:59:23.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:22 smithi117 bash[28843]: cluster 2024-03-04T15:59:21.868304+0000 mon.a (mon.0) 2518 : cluster 0 osdmap e372: 8 total, 8 up, 8 in 2024-03-04T15:59:23.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:22 smithi117 bash[28843]: audit 2024-03-04T15:59:21.872060+0000 mon.c (mon.2) 566 : audit 1 from='client.? 172.21.15.117:0/1753464141' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:23.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:22 smithi117 bash[28843]: audit 2024-03-04T15:59:21.872658+0000 mon.c (mon.2) 567 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49"}]: dispatch 2024-03-04T15:59:23.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:22 smithi117 bash[28843]: audit 2024-03-04T15:59:21.885122+0000 mon.a (mon.0) 2519 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi117-63923-65"}]: dispatch 2024-03-04T15:59:23.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:22 smithi117 bash[28843]: audit 2024-03-04T15:59:21.885846+0000 mon.a (mon.0) 2520 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49"}]: dispatch 2024-03-04T15:59:23.915 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP2163:head 2024-03-04T15:59:23.915 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:5d165639:::164:head 2024-03-04T15:59:23.915 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:f43765fc:::165:head 2024-03-04T15:59:23.915 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:b4c720e9:::166:head 2024-03-04T15:59:23.915 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:e694b040:::167:head 2024-03-04T15:59:23.916 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:afa38db2:::168:head 2024-03-04T15:59:23.916 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:77ba9f53:::169:head 2024-03-04T15:59:23.916 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:87495034:::170:head 2024-03-04T15:59:23.916 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:7c96bf0e:::171:head 2024-03-04T15:59:23.916 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:dbe346cc:::172:head 2024-03-04T15:59:23.916 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:e943ec24:::173:head 2024-03-04T15:59:23.916 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:f97a9c0c:::174:head 2024-03-04T15:59:23.916 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:6f26e74d:::175:head 2024-03-04T15:59:23.916 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:4f95e106:::176:head 2024-03-04T15:59:23.916 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:0e6f2f8f:::177:head 2024-03-04T15:59:23.916 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:05db05f1:::178:head 2024-03-04T15:59:23.916 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:38a78d66:::179:head 2024-03-04T15:59:23.916 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:d095610b:::180:head 2024-03-04T15:59:23.917 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:a1a9d709:::181:head 2024-03-04T15:59:23.917 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:1e5d39db:::182:head 2024-03-04T15:59:23.917 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:f7df4fb9:::183:head 2024-03-04T15:59:23.917 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:03a7f161:::184:head 2024-03-04T15:59:23.917 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:ba70721e:::185:head 2024-03-04T15:59:23.917 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:28e5662d:::186:head 2024-03-04T15:59:23.917 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:973d52de:::187:head 2024-03-04T15:59:23.917 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:4303eb1c:::188:head 2024-03-04T15:59:23.917 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:b990b48e:::189:head 2024-03-04T15:59:23.917 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:29b8165b:::190:head 2024-03-04T15:59:23.918 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:3547f197:::191:head 2024-03-04T15:59:23.918 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:7e260936:::192:head 2024-03-04T15:59:23.918 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:1abec7b1:::193:head 2024-03-04T15:59:23.918 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:10fdda93:::194:head 2024-03-04T15:59:23.918 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:15817eea:::195:head 2024-03-04T15:59:23.918 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:770bab57:::196:head 2024-03-04T15:59:23.918 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:ed9e13e7:::197:head 2024-03-04T15:59:23.918 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:71471a8f:::198:head 2024-03-04T15:59:23.918 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: checking for 277:10fb1d02:::199:head 2024-03-04T15:59:23.918 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8024 ms) 2024-03-04T15:59:23.918 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-03-04T15:59:23.918 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709567918,0 2024-03-04T15:59:23.918 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: first is 1709567918 2024-03-04T15:59:23.918 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709567918,0 2024-03-04T15:59:23.918 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709567918,0 2024-03-04T15:59:23.919 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709567918,1709567920,1709567921,0 2024-03-04T15:59:23.919 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709567918,1709567920,1709567921,0 2024-03-04T15:59:23.919 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709567918,1709567920,1709567921,0 2024-03-04T15:59:23.919 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709567918,1709567920,1709567921,1709567923,1709567924,0 2024-03-04T15:59:23.919 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709567918,1709567920,1709567921,1709567923,1709567924,0 2024-03-04T15:59:23.919 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709567918,1709567920,1709567921,1709567923,1709567924,0 2024-03-04T15:59:23.919 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709567918,1709567920,1709567921,1709567923,1709567924,1709567926,1709567927,0 2024-03-04T15:59:23.919 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709567918,1709567920,1709567921,1709567923,1709567924,1709567926,1709567927,0 2024-03-04T15:59:23.919 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709567918,1709567920,1709567921,1709567923,1709567924,1709567926,1709567927,0 2024-03-04T15:59:23.919 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709567921,1709567923,1709567924,1709567926,1709567927,1709567929,1709567930,0 2024-03-04T15:59:23.919 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: first now 1709567921, trimmed 2024-03-04T15:59:23.919 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (21210 ms) 2024-03-04T15:59:23.919 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-03-04T15:59:23.920 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: foo0 2024-03-04T15:59:23.920 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-03-04T15:59:23.920 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14055 ms) 2024-03-04T15:59:23.920 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-03-04T15:59:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:23 smithi151 bash[23756]: cluster 2024-03-04T15:59:22.876088+0000 mon.a (mon.0) 2521 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:59:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:23 smithi151 bash[23756]: audit 2024-03-04T15:59:22.878520+0000 mon.a (mon.0) 2522 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi117-63923-65"}]': finished 2024-03-04T15:59:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:23 smithi151 bash[23756]: audit 2024-03-04T15:59:22.878722+0000 mon.a (mon.0) 2523 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49"}]': finished 2024-03-04T15:59:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:23 smithi151 bash[23756]: cluster 2024-03-04T15:59:22.888117+0000 mon.a (mon.0) 2524 : cluster 0 osdmap e373: 8 total, 8 up, 8 in 2024-03-04T15:59:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:23 smithi151 bash[23756]: cluster 2024-03-04T15:59:22.944300+0000 mgr.y (mgr.24428) 386 : cluster 0 pgmap v566: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:59:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:23 smithi151 bash[23756]: audit 2024-03-04T15:59:22.948063+0000 mon.c (mon.2) 568 : audit 1 from='client.? 172.21.15.117:0/3228559736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:23 smithi151 bash[23756]: audit 2024-03-04T15:59:22.948415+0000 mon.a (mon.0) 2525 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:23 smithi151 bash[23756]: audit 2024-03-04T15:59:22.949363+0000 mon.c (mon.2) 569 : audit 1 from='client.? 172.21.15.117:0/3228559736' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:23 smithi151 bash[23756]: audit 2024-03-04T15:59:22.949735+0000 mon.a (mon.0) 2526 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:23 smithi151 bash[23756]: audit 2024-03-04T15:59:22.950279+0000 mon.c (mon.2) 570 : audit 1 from='client.? 172.21.15.117:0/3228559736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi117-63923-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:23 smithi151 bash[23756]: audit 2024-03-04T15:59:22.950582+0000 mon.a (mon.0) 2527 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi117-63923-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:23 smithi151 bash[23756]: audit 2024-03-04T15:59:22.959003+0000 mon.c (mon.2) 571 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:23 smithi151 bash[23756]: audit 2024-03-04T15:59:22.959292+0000 mon.a (mon.0) 2528 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:23 smithi151 bash[23756]: audit 2024-03-04T15:59:22.959823+0000 mon.c (mon.2) 572 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49"}]: dispatch 2024-03-04T15:59:24.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:23 smithi151 bash[23756]: audit 2024-03-04T15:59:22.960081+0000 mon.a (mon.0) 2529 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49"}]: dispatch 2024-03-04T15:59:24.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[28843]: cluster 2024-03-04T15:59:22.876088+0000 mon.a (mon.0) 2521 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:59:24.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[28843]: audit 2024-03-04T15:59:22.878520+0000 mon.a (mon.0) 2522 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi117-63923-65"}]': finished 2024-03-04T15:59:24.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[28843]: audit 2024-03-04T15:59:22.878722+0000 mon.a (mon.0) 2523 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49"}]': finished 2024-03-04T15:59:24.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[28843]: cluster 2024-03-04T15:59:22.888117+0000 mon.a (mon.0) 2524 : cluster 0 osdmap e373: 8 total, 8 up, 8 in 2024-03-04T15:59:24.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[28843]: cluster 2024-03-04T15:59:22.944300+0000 mgr.y (mgr.24428) 386 : cluster 0 pgmap v566: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:59:24.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[28843]: audit 2024-03-04T15:59:22.948063+0000 mon.c (mon.2) 568 : audit 1 from='client.? 172.21.15.117:0/3228559736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:24.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[28843]: audit 2024-03-04T15:59:22.948415+0000 mon.a (mon.0) 2525 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:24.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[28843]: audit 2024-03-04T15:59:22.949363+0000 mon.c (mon.2) 569 : audit 1 from='client.? 172.21.15.117:0/3228559736' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:24.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[28843]: audit 2024-03-04T15:59:22.949735+0000 mon.a (mon.0) 2526 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:24.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[28843]: audit 2024-03-04T15:59:22.950279+0000 mon.c (mon.2) 570 : audit 1 from='client.? 172.21.15.117:0/3228559736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi117-63923-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:24.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[28843]: audit 2024-03-04T15:59:22.950582+0000 mon.a (mon.0) 2527 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi117-63923-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:24.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[28843]: audit 2024-03-04T15:59:22.959003+0000 mon.c (mon.2) 571 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:24.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[28843]: audit 2024-03-04T15:59:22.959292+0000 mon.a (mon.0) 2528 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:24.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[28843]: audit 2024-03-04T15:59:22.959823+0000 mon.c (mon.2) 572 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49"}]: dispatch 2024-03-04T15:59:24.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[28843]: audit 2024-03-04T15:59:22.960081+0000 mon.a (mon.0) 2529 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49"}]: dispatch 2024-03-04T15:59:24.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[21480]: cluster 2024-03-04T15:59:22.876088+0000 mon.a (mon.0) 2521 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:59:24.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[21480]: audit 2024-03-04T15:59:22.878520+0000 mon.a (mon.0) 2522 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi117-63923-65"}]': finished 2024-03-04T15:59:24.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[21480]: audit 2024-03-04T15:59:22.878722+0000 mon.a (mon.0) 2523 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49"}]': finished 2024-03-04T15:59:24.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[21480]: cluster 2024-03-04T15:59:22.888117+0000 mon.a (mon.0) 2524 : cluster 0 osdmap e373: 8 total, 8 up, 8 in 2024-03-04T15:59:24.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[21480]: cluster 2024-03-04T15:59:22.944300+0000 mgr.y (mgr.24428) 386 : cluster 0 pgmap v566: 228 pgs: 228 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:59:24.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[21480]: audit 2024-03-04T15:59:22.948063+0000 mon.c (mon.2) 568 : audit 1 from='client.? 172.21.15.117:0/3228559736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:24.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[21480]: audit 2024-03-04T15:59:22.948415+0000 mon.a (mon.0) 2525 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:24.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[21480]: audit 2024-03-04T15:59:22.949363+0000 mon.c (mon.2) 569 : audit 1 from='client.? 172.21.15.117:0/3228559736' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:24.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[21480]: audit 2024-03-04T15:59:22.949735+0000 mon.a (mon.0) 2526 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:24.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[21480]: audit 2024-03-04T15:59:22.950279+0000 mon.c (mon.2) 570 : audit 1 from='client.? 172.21.15.117:0/3228559736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi117-63923-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:24.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[21480]: audit 2024-03-04T15:59:22.950582+0000 mon.a (mon.0) 2527 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi117-63923-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T15:59:24.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[21480]: audit 2024-03-04T15:59:22.959003+0000 mon.c (mon.2) 571 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:24.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[21480]: audit 2024-03-04T15:59:22.959292+0000 mon.a (mon.0) 2528 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:24.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[21480]: audit 2024-03-04T15:59:22.959823+0000 mon.c (mon.2) 572 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49"}]: dispatch 2024-03-04T15:59:24.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:23 smithi117 bash[21480]: audit 2024-03-04T15:59:22.960081+0000 mon.a (mon.0) 2529 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-49"}]: dispatch 2024-03-04T15:59:25.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:24 smithi151 bash[23756]: audit 2024-03-04T15:59:23.903240+0000 mon.a (mon.0) 2530 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi117-63923-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:59:25.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:24 smithi151 bash[23756]: cluster 2024-03-04T15:59:23.915478+0000 mon.a (mon.0) 2531 : cluster 0 osdmap e374: 8 total, 8 up, 8 in 2024-03-04T15:59:25.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:24 smithi151 bash[23756]: audit 2024-03-04T15:59:23.917086+0000 mon.c (mon.2) 573 : audit 1 from='client.? 172.21.15.117:0/3228559736' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi117-63923-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:25.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:24 smithi151 bash[23756]: audit 2024-03-04T15:59:23.918191+0000 mon.a (mon.0) 2532 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi117-63923-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:25.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:24 smithi117 bash[21480]: audit 2024-03-04T15:59:23.903240+0000 mon.a (mon.0) 2530 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi117-63923-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:59:25.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:24 smithi117 bash[21480]: cluster 2024-03-04T15:59:23.915478+0000 mon.a (mon.0) 2531 : cluster 0 osdmap e374: 8 total, 8 up, 8 in 2024-03-04T15:59:25.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:24 smithi117 bash[21480]: audit 2024-03-04T15:59:23.917086+0000 mon.c (mon.2) 573 : audit 1 from='client.? 172.21.15.117:0/3228559736' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi117-63923-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:25.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:24 smithi117 bash[21480]: audit 2024-03-04T15:59:23.918191+0000 mon.a (mon.0) 2532 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi117-63923-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:25.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:24 smithi117 bash[28843]: audit 2024-03-04T15:59:23.903240+0000 mon.a (mon.0) 2530 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi117-63923-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T15:59:25.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:24 smithi117 bash[28843]: cluster 2024-03-04T15:59:23.915478+0000 mon.a (mon.0) 2531 : cluster 0 osdmap e374: 8 total, 8 up, 8 in 2024-03-04T15:59:25.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:24 smithi117 bash[28843]: audit 2024-03-04T15:59:23.917086+0000 mon.c (mon.2) 573 : audit 1 from='client.? 172.21.15.117:0/3228559736' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi117-63923-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:25.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:24 smithi117 bash[28843]: audit 2024-03-04T15:59:23.918191+0000 mon.a (mon.0) 2532 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi117-63923-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:26.317 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:59:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:59:26.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:25 smithi117 bash[28843]: cluster 2024-03-04T15:59:24.945408+0000 mgr.y (mgr.24428) 387 : cluster 0 pgmap v569: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:59:26.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:25 smithi117 bash[28843]: cluster 2024-03-04T15:59:25.371803+0000 mon.a (mon.0) 2533 : cluster 0 osdmap e375: 8 total, 8 up, 8 in 2024-03-04T15:59:26.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:25 smithi117 bash[28843]: audit 2024-03-04T15:59:25.376476+0000 mon.c (mon.2) 574 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:26.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:25 smithi117 bash[28843]: audit 2024-03-04T15:59:25.378726+0000 mon.a (mon.0) 2534 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:26.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:25 smithi117 bash[21480]: cluster 2024-03-04T15:59:24.945408+0000 mgr.y (mgr.24428) 387 : cluster 0 pgmap v569: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:59:26.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:25 smithi117 bash[21480]: cluster 2024-03-04T15:59:25.371803+0000 mon.a (mon.0) 2533 : cluster 0 osdmap e375: 8 total, 8 up, 8 in 2024-03-04T15:59:26.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:25 smithi117 bash[21480]: audit 2024-03-04T15:59:25.376476+0000 mon.c (mon.2) 574 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:26.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:25 smithi117 bash[21480]: audit 2024-03-04T15:59:25.378726+0000 mon.a (mon.0) 2534 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:26.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:25 smithi151 bash[23756]: cluster 2024-03-04T15:59:24.945408+0000 mgr.y (mgr.24428) 387 : cluster 0 pgmap v569: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 624 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T15:59:26.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:25 smithi151 bash[23756]: cluster 2024-03-04T15:59:25.371803+0000 mon.a (mon.0) 2533 : cluster 0 osdmap e375: 8 total, 8 up, 8 in 2024-03-04T15:59:26.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:25 smithi151 bash[23756]: audit 2024-03-04T15:59:25.376476+0000 mon.c (mon.2) 574 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:26.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:25 smithi151 bash[23756]: audit 2024-03-04T15:59:25.378726+0000 mon.a (mon.0) 2534 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:27.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:59:26 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:59:27.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:26 smithi151 bash[23756]: cluster 2024-03-04T15:59:25.920786+0000 mon.a (mon.0) 2535 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:26 smithi151 bash[23756]: audit 2024-03-04T15:59:25.922857+0000 mon.a (mon.0) 2536 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi117-63923-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi117-63923-66"}]': finished 2024-03-04T15:59:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:26 smithi151 bash[23756]: audit 2024-03-04T15:59:25.923103+0000 mon.a (mon.0) 2537 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:59:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:26 smithi151 bash[23756]: cluster 2024-03-04T15:59:25.932951+0000 mon.a (mon.0) 2538 : cluster 0 osdmap e376: 8 total, 8 up, 8 in 2024-03-04T15:59:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:26 smithi151 bash[23756]: audit 2024-03-04T15:59:25.989447+0000 mon.c (mon.2) 575 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:59:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:26 smithi151 bash[23756]: audit 2024-03-04T15:59:25.989988+0000 mon.a (mon.0) 2539 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:59:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:26 smithi151 bash[23756]: audit 2024-03-04T15:59:26.925980+0000 mon.a (mon.0) 2540 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:59:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:26 smithi151 bash[23756]: cluster 2024-03-04T15:59:26.933568+0000 mon.a (mon.0) 2541 : cluster 0 osdmap e377: 8 total, 8 up, 8 in 2024-03-04T15:59:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:26 smithi151 bash[23756]: audit 2024-03-04T15:59:26.936366+0000 mon.c (mon.2) 576 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-51"}]: dispatch 2024-03-04T15:59:27.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:26 smithi151 bash[23756]: audit 2024-03-04T15:59:26.937322+0000 mon.a (mon.0) 2542 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-51"}]: dispatch 2024-03-04T15:59:27.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[21480]: cluster 2024-03-04T15:59:25.920786+0000 mon.a (mon.0) 2535 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:27.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[21480]: audit 2024-03-04T15:59:25.922857+0000 mon.a (mon.0) 2536 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi117-63923-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi117-63923-66"}]': finished 2024-03-04T15:59:27.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[21480]: audit 2024-03-04T15:59:25.923103+0000 mon.a (mon.0) 2537 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:59:27.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[21480]: cluster 2024-03-04T15:59:25.932951+0000 mon.a (mon.0) 2538 : cluster 0 osdmap e376: 8 total, 8 up, 8 in 2024-03-04T15:59:27.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[21480]: audit 2024-03-04T15:59:25.989447+0000 mon.c (mon.2) 575 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:59:27.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[21480]: audit 2024-03-04T15:59:25.989988+0000 mon.a (mon.0) 2539 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:59:27.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[21480]: audit 2024-03-04T15:59:26.925980+0000 mon.a (mon.0) 2540 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:59:27.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[21480]: cluster 2024-03-04T15:59:26.933568+0000 mon.a (mon.0) 2541 : cluster 0 osdmap e377: 8 total, 8 up, 8 in 2024-03-04T15:59:27.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[21480]: audit 2024-03-04T15:59:26.936366+0000 mon.c (mon.2) 576 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-51"}]: dispatch 2024-03-04T15:59:27.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[21480]: audit 2024-03-04T15:59:26.937322+0000 mon.a (mon.0) 2542 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-51"}]: dispatch 2024-03-04T15:59:27.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[28843]: cluster 2024-03-04T15:59:25.920786+0000 mon.a (mon.0) 2535 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:27.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[28843]: audit 2024-03-04T15:59:25.922857+0000 mon.a (mon.0) 2536 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi117-63923-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi117-63923-66"}]': finished 2024-03-04T15:59:27.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[28843]: audit 2024-03-04T15:59:25.923103+0000 mon.a (mon.0) 2537 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:59:27.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[28843]: cluster 2024-03-04T15:59:25.932951+0000 mon.a (mon.0) 2538 : cluster 0 osdmap e376: 8 total, 8 up, 8 in 2024-03-04T15:59:27.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[28843]: audit 2024-03-04T15:59:25.989447+0000 mon.c (mon.2) 575 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:59:27.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[28843]: audit 2024-03-04T15:59:25.989988+0000 mon.a (mon.0) 2539 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T15:59:27.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[28843]: audit 2024-03-04T15:59:26.925980+0000 mon.a (mon.0) 2540 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T15:59:27.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[28843]: cluster 2024-03-04T15:59:26.933568+0000 mon.a (mon.0) 2541 : cluster 0 osdmap e377: 8 total, 8 up, 8 in 2024-03-04T15:59:27.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[28843]: audit 2024-03-04T15:59:26.936366+0000 mon.c (mon.2) 576 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-51"}]: dispatch 2024-03-04T15:59:27.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:26 smithi117 bash[28843]: audit 2024-03-04T15:59:26.937322+0000 mon.a (mon.0) 2542 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-51"}]: dispatch 2024-03-04T15:59:28.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:27 smithi117 bash[21480]: audit 2024-03-04T15:59:26.753949+0000 mgr.y (mgr.24428) 388 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:59:28.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:27 smithi117 bash[21480]: cluster 2024-03-04T15:59:26.946678+0000 mgr.y (mgr.24428) 389 : cluster 0 pgmap v572: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-04T15:59:28.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:27 smithi117 bash[21480]: audit 2024-03-04T15:59:27.928889+0000 mon.a (mon.0) 2543 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-51"}]': finished 2024-03-04T15:59:28.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:27 smithi117 bash[21480]: cluster 2024-03-04T15:59:27.937198+0000 mon.a (mon.0) 2544 : cluster 0 osdmap e378: 8 total, 8 up, 8 in 2024-03-04T15:59:28.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:27 smithi117 bash[21480]: audit 2024-03-04T15:59:27.946361+0000 mon.c (mon.2) 577 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-51", "mode": "writeback"}]: dispatch 2024-03-04T15:59:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:27 smithi117 bash[28843]: audit 2024-03-04T15:59:26.753949+0000 mgr.y (mgr.24428) 388 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:59:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:27 smithi117 bash[28843]: cluster 2024-03-04T15:59:26.946678+0000 mgr.y (mgr.24428) 389 : cluster 0 pgmap v572: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-04T15:59:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:27 smithi117 bash[28843]: audit 2024-03-04T15:59:27.928889+0000 mon.a (mon.0) 2543 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-51"}]': finished 2024-03-04T15:59:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:27 smithi117 bash[28843]: cluster 2024-03-04T15:59:27.937198+0000 mon.a (mon.0) 2544 : cluster 0 osdmap e378: 8 total, 8 up, 8 in 2024-03-04T15:59:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:27 smithi117 bash[28843]: audit 2024-03-04T15:59:27.946361+0000 mon.c (mon.2) 577 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-51", "mode": "writeback"}]: dispatch 2024-03-04T15:59:28.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:27 smithi151 bash[23756]: audit 2024-03-04T15:59:26.753949+0000 mgr.y (mgr.24428) 388 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:59:28.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:27 smithi151 bash[23756]: cluster 2024-03-04T15:59:26.946678+0000 mgr.y (mgr.24428) 389 : cluster 0 pgmap v572: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-04T15:59:28.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:27 smithi151 bash[23756]: audit 2024-03-04T15:59:27.928889+0000 mon.a (mon.0) 2543 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-51"}]': finished 2024-03-04T15:59:28.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:27 smithi151 bash[23756]: cluster 2024-03-04T15:59:27.937198+0000 mon.a (mon.0) 2544 : cluster 0 osdmap e378: 8 total, 8 up, 8 in 2024-03-04T15:59:28.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:27 smithi151 bash[23756]: audit 2024-03-04T15:59:27.946361+0000 mon.c (mon.2) 577 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-51", "mode": "writeback"}]: dispatch 2024-03-04T15:59:29.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:28 smithi117 bash[21480]: audit 2024-03-04T15:59:27.947206+0000 mon.c (mon.2) 578 : audit 1 from='client.? 172.21.15.117:0/3228559736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:29.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:28 smithi117 bash[21480]: audit 2024-03-04T15:59:27.947866+0000 mon.a (mon.0) 2545 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-51", "mode": "writeback"}]: dispatch 2024-03-04T15:59:29.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:28 smithi117 bash[21480]: audit 2024-03-04T15:59:27.949543+0000 mon.a (mon.0) 2546 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:29.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:28 smithi117 bash[21480]: cluster 2024-03-04T15:59:28.929205+0000 mon.a (mon.0) 2547 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:59:29.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:28 smithi117 bash[21480]: audit 2024-03-04T15:59:28.933359+0000 mon.a (mon.0) 2548 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-51", "mode": "writeback"}]': finished 2024-03-04T15:59:29.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:28 smithi117 bash[21480]: audit 2024-03-04T15:59:28.933598+0000 mon.a (mon.0) 2549 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi117-63923-66"}]': finished 2024-03-04T15:59:29.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:28 smithi117 bash[21480]: cluster 2024-03-04T15:59:28.941457+0000 mon.a (mon.0) 2550 : cluster 0 osdmap e379: 8 total, 8 up, 8 in 2024-03-04T15:59:29.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:28 smithi117 bash[21480]: audit 2024-03-04T15:59:28.943987+0000 mon.c (mon.2) 579 : audit 1 from='client.? 172.21.15.117:0/3228559736' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:29.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:28 smithi117 bash[21480]: audit 2024-03-04T15:59:28.945132+0000 mon.c (mon.2) 580 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:59:29.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:28 smithi117 bash[21480]: audit 2024-03-04T15:59:28.946643+0000 mon.a (mon.0) 2551 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:29.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:28 smithi117 bash[28843]: audit 2024-03-04T15:59:27.947206+0000 mon.c (mon.2) 578 : audit 1 from='client.? 172.21.15.117:0/3228559736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:29.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:28 smithi117 bash[28843]: audit 2024-03-04T15:59:27.947866+0000 mon.a (mon.0) 2545 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-51", "mode": "writeback"}]: dispatch 2024-03-04T15:59:29.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:28 smithi117 bash[28843]: audit 2024-03-04T15:59:27.949543+0000 mon.a (mon.0) 2546 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:29.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:28 smithi117 bash[28843]: cluster 2024-03-04T15:59:28.929205+0000 mon.a (mon.0) 2547 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:59:29.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:28 smithi117 bash[28843]: audit 2024-03-04T15:59:28.933359+0000 mon.a (mon.0) 2548 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-51", "mode": "writeback"}]': finished 2024-03-04T15:59:29.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:28 smithi117 bash[28843]: audit 2024-03-04T15:59:28.933598+0000 mon.a (mon.0) 2549 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi117-63923-66"}]': finished 2024-03-04T15:59:29.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:28 smithi117 bash[28843]: cluster 2024-03-04T15:59:28.941457+0000 mon.a (mon.0) 2550 : cluster 0 osdmap e379: 8 total, 8 up, 8 in 2024-03-04T15:59:29.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:28 smithi117 bash[28843]: audit 2024-03-04T15:59:28.943987+0000 mon.c (mon.2) 579 : audit 1 from='client.? 172.21.15.117:0/3228559736' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:29.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:28 smithi117 bash[28843]: audit 2024-03-04T15:59:28.945132+0000 mon.c (mon.2) 580 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:59:29.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:28 smithi117 bash[28843]: audit 2024-03-04T15:59:28.946643+0000 mon.a (mon.0) 2551 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:29.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:28 smithi151 bash[23756]: audit 2024-03-04T15:59:27.947206+0000 mon.c (mon.2) 578 : audit 1 from='client.? 172.21.15.117:0/3228559736' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:29.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:28 smithi151 bash[23756]: audit 2024-03-04T15:59:27.947866+0000 mon.a (mon.0) 2545 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-51", "mode": "writeback"}]: dispatch 2024-03-04T15:59:29.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:28 smithi151 bash[23756]: audit 2024-03-04T15:59:27.949543+0000 mon.a (mon.0) 2546 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:29.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:28 smithi151 bash[23756]: cluster 2024-03-04T15:59:28.929205+0000 mon.a (mon.0) 2547 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T15:59:29.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:28 smithi151 bash[23756]: audit 2024-03-04T15:59:28.933359+0000 mon.a (mon.0) 2548 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-51", "mode": "writeback"}]': finished 2024-03-04T15:59:29.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:28 smithi151 bash[23756]: audit 2024-03-04T15:59:28.933598+0000 mon.a (mon.0) 2549 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi117-63923-66"}]': finished 2024-03-04T15:59:29.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:28 smithi151 bash[23756]: cluster 2024-03-04T15:59:28.941457+0000 mon.a (mon.0) 2550 : cluster 0 osdmap e379: 8 total, 8 up, 8 in 2024-03-04T15:59:29.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:28 smithi151 bash[23756]: audit 2024-03-04T15:59:28.943987+0000 mon.c (mon.2) 579 : audit 1 from='client.? 172.21.15.117:0/3228559736' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:29.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:28 smithi151 bash[23756]: audit 2024-03-04T15:59:28.945132+0000 mon.c (mon.2) 580 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:59:29.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:28 smithi151 bash[23756]: audit 2024-03-04T15:59:28.946643+0000 mon.a (mon.0) 2551 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi117-63923-66"}]: dispatch 2024-03-04T15:59:29.958 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ 2024-03-04T15:59:29.958 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3032 ms) 2024-03-04T15:59:29.958 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-03-04T15:59:29.958 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7073 ms) 2024-03-04T15:59:29.958 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-03-04T15:59:29.959 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7036 ms) 2024-03-04T15:59:29.959 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-03-04T15:59:29.959 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7042 ms) 2024-03-04T15:59:29.959 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-03-04T15:59:29.959 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7066 ms) 2024-03-04T15:59:29.959 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-03-04T15:59:29.959 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7132 ms) 2024-03-04T15:59:29.959 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-03-04T15:59:29.959 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7050 ms) 2024-03-04T15:59:29.959 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139775 ms total) 2024-03-04T15:59:29.959 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: 2024-03-04T15:59:29.959 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-03-04T15:59:29.959 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (330070 ms total) 2024-03-04T15:59:29.959 INFO:tasks.workunit.client.0.smithi117.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-03-04T15:59:30.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:29 smithi117 bash[28843]: cluster 2024-03-04T15:59:28.947619+0000 mgr.y (mgr.24428) 390 : cluster 0 pgmap v575: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-04T15:59:30.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:29 smithi117 bash[28843]: audit 2024-03-04T15:59:29.030123+0000 mon.c (mon.2) 581 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T15:59:30.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:29 smithi117 bash[28843]: audit 2024-03-04T15:59:29.030801+0000 mon.a (mon.0) 2552 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T15:59:30.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:29 smithi117 bash[28843]: audit 2024-03-04T15:59:29.942804+0000 mon.a (mon.0) 2553 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi117-63923-66"}]': finished 2024-03-04T15:59:30.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:29 smithi117 bash[28843]: audit 2024-03-04T15:59:29.943048+0000 mon.a (mon.0) 2554 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_count","val": "2"}]': finished 2024-03-04T15:59:30.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:29 smithi117 bash[28843]: cluster 2024-03-04T15:59:29.950008+0000 mon.a (mon.0) 2555 : cluster 0 osdmap e380: 8 total, 8 up, 8 in 2024-03-04T15:59:30.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:29 smithi117 bash[28843]: audit 2024-03-04T15:59:29.952136+0000 mon.c (mon.2) 582 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:59:30.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:29 smithi117 bash[28843]: audit 2024-03-04T15:59:29.954059+0000 mon.a (mon.0) 2556 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:59:30.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:29 smithi117 bash[21480]: cluster 2024-03-04T15:59:28.947619+0000 mgr.y (mgr.24428) 390 : cluster 0 pgmap v575: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-04T15:59:30.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:29 smithi117 bash[21480]: audit 2024-03-04T15:59:29.030123+0000 mon.c (mon.2) 581 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T15:59:30.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:29 smithi117 bash[21480]: audit 2024-03-04T15:59:29.030801+0000 mon.a (mon.0) 2552 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T15:59:30.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:29 smithi117 bash[21480]: audit 2024-03-04T15:59:29.942804+0000 mon.a (mon.0) 2553 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi117-63923-66"}]': finished 2024-03-04T15:59:30.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:29 smithi117 bash[21480]: audit 2024-03-04T15:59:29.943048+0000 mon.a (mon.0) 2554 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_count","val": "2"}]': finished 2024-03-04T15:59:30.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:29 smithi117 bash[21480]: cluster 2024-03-04T15:59:29.950008+0000 mon.a (mon.0) 2555 : cluster 0 osdmap e380: 8 total, 8 up, 8 in 2024-03-04T15:59:30.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:29 smithi117 bash[21480]: audit 2024-03-04T15:59:29.952136+0000 mon.c (mon.2) 582 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:59:30.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:29 smithi117 bash[21480]: audit 2024-03-04T15:59:29.954059+0000 mon.a (mon.0) 2556 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:59:30.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:29 smithi151 bash[23756]: cluster 2024-03-04T15:59:28.947619+0000 mgr.y (mgr.24428) 390 : cluster 0 pgmap v575: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-04T15:59:30.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:29 smithi151 bash[23756]: audit 2024-03-04T15:59:29.030123+0000 mon.c (mon.2) 581 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T15:59:30.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:29 smithi151 bash[23756]: audit 2024-03-04T15:59:29.030801+0000 mon.a (mon.0) 2552 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T15:59:30.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:29 smithi151 bash[23756]: audit 2024-03-04T15:59:29.942804+0000 mon.a (mon.0) 2553 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi117-63923-66"}]': finished 2024-03-04T15:59:30.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:29 smithi151 bash[23756]: audit 2024-03-04T15:59:29.943048+0000 mon.a (mon.0) 2554 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_count","val": "2"}]': finished 2024-03-04T15:59:30.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:29 smithi151 bash[23756]: cluster 2024-03-04T15:59:29.950008+0000 mon.a (mon.0) 2555 : cluster 0 osdmap e380: 8 total, 8 up, 8 in 2024-03-04T15:59:30.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:29 smithi151 bash[23756]: audit 2024-03-04T15:59:29.952136+0000 mon.c (mon.2) 582 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:59:30.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:29 smithi151 bash[23756]: audit 2024-03-04T15:59:29.954059+0000 mon.a (mon.0) 2556 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T15:59:32.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:31 smithi117 bash[21480]: audit 2024-03-04T15:59:30.945593+0000 mon.a (mon.0) 2557 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_period","val": "600"}]': finished 2024-03-04T15:59:32.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:31 smithi117 bash[21480]: cluster 2024-03-04T15:59:30.948729+0000 mgr.y (mgr.24428) 391 : cluster 0 pgmap v578: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:59:32.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:31 smithi117 bash[21480]: cluster 2024-03-04T15:59:30.953382+0000 mon.a (mon.0) 2558 : cluster 0 osdmap e381: 8 total, 8 up, 8 in 2024-03-04T15:59:32.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:31 smithi117 bash[21480]: audit 2024-03-04T15:59:30.954875+0000 mon.c (mon.2) 583 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T15:59:32.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:31 smithi117 bash[21480]: audit 2024-03-04T15:59:30.959693+0000 mon.a (mon.0) 2559 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T15:59:32.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:31 smithi117 bash[28843]: audit 2024-03-04T15:59:30.945593+0000 mon.a (mon.0) 2557 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_period","val": "600"}]': finished 2024-03-04T15:59:32.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:31 smithi117 bash[28843]: cluster 2024-03-04T15:59:30.948729+0000 mgr.y (mgr.24428) 391 : cluster 0 pgmap v578: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:59:32.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:31 smithi117 bash[28843]: cluster 2024-03-04T15:59:30.953382+0000 mon.a (mon.0) 2558 : cluster 0 osdmap e381: 8 total, 8 up, 8 in 2024-03-04T15:59:32.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:31 smithi117 bash[28843]: audit 2024-03-04T15:59:30.954875+0000 mon.c (mon.2) 583 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T15:59:32.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:31 smithi117 bash[28843]: audit 2024-03-04T15:59:30.959693+0000 mon.a (mon.0) 2559 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T15:59:32.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:31 smithi151 bash[23756]: audit 2024-03-04T15:59:30.945593+0000 mon.a (mon.0) 2557 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_period","val": "600"}]': finished 2024-03-04T15:59:32.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:31 smithi151 bash[23756]: cluster 2024-03-04T15:59:30.948729+0000 mgr.y (mgr.24428) 391 : cluster 0 pgmap v578: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:59:32.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:31 smithi151 bash[23756]: cluster 2024-03-04T15:59:30.953382+0000 mon.a (mon.0) 2558 : cluster 0 osdmap e381: 8 total, 8 up, 8 in 2024-03-04T15:59:32.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:31 smithi151 bash[23756]: audit 2024-03-04T15:59:30.954875+0000 mon.c (mon.2) 583 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T15:59:32.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:31 smithi151 bash[23756]: audit 2024-03-04T15:59:30.959693+0000 mon.a (mon.0) 2559 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T15:59:33.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:32 smithi117 bash[21480]: cluster 2024-03-04T15:59:31.945970+0000 mon.a (mon.0) 2560 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:59:33.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:32 smithi117 bash[21480]: audit 2024-03-04T15:59:31.948318+0000 mon.a (mon.0) 2561 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T15:59:33.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:32 smithi117 bash[21480]: cluster 2024-03-04T15:59:31.956661+0000 mon.a (mon.0) 2562 : cluster 0 osdmap e382: 8 total, 8 up, 8 in 2024-03-04T15:59:33.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:32 smithi117 bash[21480]: audit 2024-03-04T15:59:31.962802+0000 mon.c (mon.2) 584 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-04T15:59:33.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:32 smithi117 bash[21480]: audit 2024-03-04T15:59:31.972457+0000 mon.a (mon.0) 2563 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-04T15:59:33.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:32 smithi117 bash[28843]: cluster 2024-03-04T15:59:31.945970+0000 mon.a (mon.0) 2560 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:59:33.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:32 smithi117 bash[28843]: audit 2024-03-04T15:59:31.948318+0000 mon.a (mon.0) 2561 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T15:59:33.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:32 smithi117 bash[28843]: cluster 2024-03-04T15:59:31.956661+0000 mon.a (mon.0) 2562 : cluster 0 osdmap e382: 8 total, 8 up, 8 in 2024-03-04T15:59:33.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:32 smithi117 bash[28843]: audit 2024-03-04T15:59:31.962802+0000 mon.c (mon.2) 584 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-04T15:59:33.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:32 smithi117 bash[28843]: audit 2024-03-04T15:59:31.972457+0000 mon.a (mon.0) 2563 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-04T15:59:33.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:32 smithi151 bash[23756]: cluster 2024-03-04T15:59:31.945970+0000 mon.a (mon.0) 2560 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T15:59:33.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:32 smithi151 bash[23756]: audit 2024-03-04T15:59:31.948318+0000 mon.a (mon.0) 2561 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T15:59:33.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:32 smithi151 bash[23756]: cluster 2024-03-04T15:59:31.956661+0000 mon.a (mon.0) 2562 : cluster 0 osdmap e382: 8 total, 8 up, 8 in 2024-03-04T15:59:33.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:32 smithi151 bash[23756]: audit 2024-03-04T15:59:31.962802+0000 mon.c (mon.2) 584 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-04T15:59:33.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:32 smithi151 bash[23756]: audit 2024-03-04T15:59:31.972457+0000 mon.a (mon.0) 2563 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-04T15:59:34.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:33 smithi117 bash[21480]: cluster 2024-03-04T15:59:32.949757+0000 mgr.y (mgr.24428) 392 : cluster 0 pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-04T15:59:34.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:33 smithi117 bash[21480]: audit 2024-03-04T15:59:32.975273+0000 mon.a (mon.0) 2564 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-04T15:59:34.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:33 smithi117 bash[21480]: cluster 2024-03-04T15:59:32.988243+0000 mon.a (mon.0) 2565 : cluster 0 osdmap e383: 8 total, 8 up, 8 in 2024-03-04T15:59:34.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:33 smithi117 bash[21480]: audit 2024-03-04T15:59:32.991888+0000 mon.c (mon.2) 585 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-04T15:59:34.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:33 smithi117 bash[21480]: audit 2024-03-04T15:59:32.994474+0000 mon.a (mon.0) 2566 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-04T15:59:34.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:33 smithi117 bash[21480]: cluster 2024-03-04T15:59:33.275914+0000 mon.a (mon.0) 2567 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:34.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:33 smithi117 bash[28843]: cluster 2024-03-04T15:59:32.949757+0000 mgr.y (mgr.24428) 392 : cluster 0 pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-04T15:59:34.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:33 smithi117 bash[28843]: audit 2024-03-04T15:59:32.975273+0000 mon.a (mon.0) 2564 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-04T15:59:34.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:33 smithi117 bash[28843]: cluster 2024-03-04T15:59:32.988243+0000 mon.a (mon.0) 2565 : cluster 0 osdmap e383: 8 total, 8 up, 8 in 2024-03-04T15:59:34.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:33 smithi117 bash[28843]: audit 2024-03-04T15:59:32.991888+0000 mon.c (mon.2) 585 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-04T15:59:34.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:33 smithi117 bash[28843]: audit 2024-03-04T15:59:32.994474+0000 mon.a (mon.0) 2566 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-04T15:59:34.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:33 smithi117 bash[28843]: cluster 2024-03-04T15:59:33.275914+0000 mon.a (mon.0) 2567 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:34.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:33 smithi151 bash[23756]: cluster 2024-03-04T15:59:32.949757+0000 mgr.y (mgr.24428) 392 : cluster 0 pgmap v580: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-04T15:59:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:33 smithi151 bash[23756]: audit 2024-03-04T15:59:32.975273+0000 mon.a (mon.0) 2564 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-04T15:59:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:33 smithi151 bash[23756]: cluster 2024-03-04T15:59:32.988243+0000 mon.a (mon.0) 2565 : cluster 0 osdmap e383: 8 total, 8 up, 8 in 2024-03-04T15:59:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:33 smithi151 bash[23756]: audit 2024-03-04T15:59:32.991888+0000 mon.c (mon.2) 585 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-04T15:59:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:33 smithi151 bash[23756]: audit 2024-03-04T15:59:32.994474+0000 mon.a (mon.0) 2566 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "target_max_objects","val": "1"}]: dispatch 2024-03-04T15:59:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:33 smithi151 bash[23756]: cluster 2024-03-04T15:59:33.275914+0000 mon.a (mon.0) 2567 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:35.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:34 smithi117 bash[21480]: audit 2024-03-04T15:59:33.977980+0000 mon.a (mon.0) 2568 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "target_max_objects","val": "1"}]': finished 2024-03-04T15:59:35.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:34 smithi117 bash[21480]: cluster 2024-03-04T15:59:33.987714+0000 mon.a (mon.0) 2569 : cluster 0 osdmap e384: 8 total, 8 up, 8 in 2024-03-04T15:59:35.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:34 smithi117 bash[28843]: audit 2024-03-04T15:59:33.977980+0000 mon.a (mon.0) 2568 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "target_max_objects","val": "1"}]': finished 2024-03-04T15:59:35.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:34 smithi117 bash[28843]: cluster 2024-03-04T15:59:33.987714+0000 mon.a (mon.0) 2569 : cluster 0 osdmap e384: 8 total, 8 up, 8 in 2024-03-04T15:59:35.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:34 smithi151 bash[23756]: audit 2024-03-04T15:59:33.977980+0000 mon.a (mon.0) 2568 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-51","var": "target_max_objects","val": "1"}]': finished 2024-03-04T15:59:35.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:34 smithi151 bash[23756]: cluster 2024-03-04T15:59:33.987714+0000 mon.a (mon.0) 2569 : cluster 0 osdmap e384: 8 total, 8 up, 8 in 2024-03-04T15:59:36.317 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:59:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:59:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:59:36.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:35 smithi117 bash[21480]: cluster 2024-03-04T15:59:34.950690+0000 mgr.y (mgr.24428) 393 : cluster 0 pgmap v583: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-04T15:59:36.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:35 smithi117 bash[21480]: cluster 2024-03-04T15:59:34.993101+0000 mon.a (mon.0) 2570 : cluster 3 Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-04T15:59:36.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:35 smithi117 bash[28843]: cluster 2024-03-04T15:59:34.950690+0000 mgr.y (mgr.24428) 393 : cluster 0 pgmap v583: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-04T15:59:36.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:35 smithi117 bash[28843]: cluster 2024-03-04T15:59:34.993101+0000 mon.a (mon.0) 2570 : cluster 3 Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-04T15:59:36.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:35 smithi151 bash[23756]: cluster 2024-03-04T15:59:34.950690+0000 mgr.y (mgr.24428) 393 : cluster 0 pgmap v583: 228 pgs: 228 active+clean; 8.4 MiB data, 609 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-04T15:59:36.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:35 smithi151 bash[23756]: cluster 2024-03-04T15:59:34.993101+0000 mon.a (mon.0) 2570 : cluster 3 Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-04T15:59:37.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:59:36 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:59:38.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:38 smithi117 bash[21480]: audit 2024-03-04T15:59:36.761620+0000 mgr.y (mgr.24428) 394 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:59:38.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:38 smithi117 bash[21480]: cluster 2024-03-04T15:59:36.952934+0000 mgr.y (mgr.24428) 395 : cluster 0 pgmap v584: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-04T15:59:38.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:38 smithi117 bash[28843]: audit 2024-03-04T15:59:36.761620+0000 mgr.y (mgr.24428) 394 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:59:38.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:38 smithi117 bash[28843]: cluster 2024-03-04T15:59:36.952934+0000 mgr.y (mgr.24428) 395 : cluster 0 pgmap v584: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-04T15:59:38.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:37 smithi151 bash[23756]: audit 2024-03-04T15:59:36.761620+0000 mgr.y (mgr.24428) 394 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:59:38.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:37 smithi151 bash[23756]: cluster 2024-03-04T15:59:36.952934+0000 mgr.y (mgr.24428) 395 : cluster 0 pgmap v584: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-04T15:59:40.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:40 smithi117 bash[21480]: cluster 2024-03-04T15:59:38.953783+0000 mgr.y (mgr.24428) 396 : cluster 0 pgmap v585: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 731 B/s rd, 0 B/s wr, 0 op/s 2024-03-04T15:59:40.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:40 smithi117 bash[28843]: cluster 2024-03-04T15:59:38.953783+0000 mgr.y (mgr.24428) 396 : cluster 0 pgmap v585: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 731 B/s rd, 0 B/s wr, 0 op/s 2024-03-04T15:59:40.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:40 smithi151 bash[23756]: cluster 2024-03-04T15:59:38.953783+0000 mgr.y (mgr.24428) 396 : cluster 0 pgmap v585: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 731 B/s rd, 0 B/s wr, 0 op/s 2024-03-04T15:59:41.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:41 smithi117 bash[21480]: audit 2024-03-04T15:59:40.420517+0000 mon.c (mon.2) 586 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:59:41.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:41 smithi117 bash[28843]: audit 2024-03-04T15:59:40.420517+0000 mon.c (mon.2) 586 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:59:41.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:41 smithi151 bash[23756]: audit 2024-03-04T15:59:40.420517+0000 mon.c (mon.2) 586 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T15:59:42.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:42 smithi117 bash[21480]: cluster 2024-03-04T15:59:40.954576+0000 mgr.y (mgr.24428) 397 : cluster 0 pgmap v586: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-03-04T15:59:42.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:42 smithi117 bash[21480]: audit 2024-03-04T15:59:41.109204+0000 mon.c (mon.2) 587 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:59:42.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:42 smithi117 bash[21480]: audit 2024-03-04T15:59:41.111151+0000 mon.c (mon.2) 588 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:59:42.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:42 smithi117 bash[21480]: audit 2024-03-04T15:59:41.121855+0000 mon.a (mon.0) 2571 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:59:42.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:42 smithi117 bash[28843]: cluster 2024-03-04T15:59:40.954576+0000 mgr.y (mgr.24428) 397 : cluster 0 pgmap v586: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-03-04T15:59:42.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:42 smithi117 bash[28843]: audit 2024-03-04T15:59:41.109204+0000 mon.c (mon.2) 587 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:59:42.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:42 smithi117 bash[28843]: audit 2024-03-04T15:59:41.111151+0000 mon.c (mon.2) 588 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:59:42.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:42 smithi117 bash[28843]: audit 2024-03-04T15:59:41.121855+0000 mon.a (mon.0) 2571 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:59:42.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:42 smithi151 bash[23756]: cluster 2024-03-04T15:59:40.954576+0000 mgr.y (mgr.24428) 397 : cluster 0 pgmap v586: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 B/s wr, 0 op/s 2024-03-04T15:59:42.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:42 smithi151 bash[23756]: audit 2024-03-04T15:59:41.109204+0000 mon.c (mon.2) 587 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T15:59:42.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:42 smithi151 bash[23756]: audit 2024-03-04T15:59:41.111151+0000 mon.c (mon.2) 588 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T15:59:42.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:42 smithi151 bash[23756]: audit 2024-03-04T15:59:41.121855+0000 mon.a (mon.0) 2571 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:59:44.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:44 smithi117 bash[28843]: cluster 2024-03-04T15:59:42.956761+0000 mgr.y (mgr.24428) 398 : cluster 0 pgmap v587: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T15:59:44.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:44 smithi117 bash[28843]: audit 2024-03-04T15:59:43.999927+0000 mon.a (mon.0) 2572 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:59:44.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:44 smithi117 bash[28843]: audit 2024-03-04T15:59:44.003509+0000 mon.c (mon.2) 589 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:59:44.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:44 smithi117 bash[21480]: cluster 2024-03-04T15:59:42.956761+0000 mgr.y (mgr.24428) 398 : cluster 0 pgmap v587: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T15:59:44.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:44 smithi117 bash[21480]: audit 2024-03-04T15:59:43.999927+0000 mon.a (mon.0) 2572 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:59:44.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:44 smithi117 bash[21480]: audit 2024-03-04T15:59:44.003509+0000 mon.c (mon.2) 589 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:59:44.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:44 smithi151 bash[23756]: cluster 2024-03-04T15:59:42.956761+0000 mgr.y (mgr.24428) 398 : cluster 0 pgmap v587: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T15:59:44.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:44 smithi151 bash[23756]: audit 2024-03-04T15:59:43.999927+0000 mon.a (mon.0) 2572 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:59:44.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:44 smithi151 bash[23756]: audit 2024-03-04T15:59:44.003509+0000 mon.c (mon.2) 589 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:59:45.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:45 smithi117 bash[21480]: audit 2024-03-04T15:59:44.020522+0000 mon.c (mon.2) 590 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:45.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:45 smithi117 bash[21480]: audit 2024-03-04T15:59:44.028916+0000 mon.a (mon.0) 2573 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:45.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:45 smithi117 bash[28843]: audit 2024-03-04T15:59:44.020522+0000 mon.c (mon.2) 590 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:45.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:45 smithi117 bash[28843]: audit 2024-03-04T15:59:44.028916+0000 mon.a (mon.0) 2573 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:45.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:45 smithi151 bash[23756]: audit 2024-03-04T15:59:44.020522+0000 mon.c (mon.2) 590 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:45.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:45 smithi151 bash[23756]: audit 2024-03-04T15:59:44.028916+0000 mon.a (mon.0) 2573 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:46.317 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:59:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:59:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:59:46.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:46 smithi117 bash[28843]: cluster 2024-03-04T15:59:44.957639+0000 mgr.y (mgr.24428) 399 : cluster 0 pgmap v588: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 933 B/s rd, 0 B/s wr, 1 op/s 2024-03-04T15:59:46.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:46 smithi117 bash[28843]: audit 2024-03-04T15:59:45.033126+0000 mon.a (mon.0) 2574 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:59:46.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:46 smithi117 bash[28843]: cluster 2024-03-04T15:59:45.040872+0000 mon.a (mon.0) 2575 : cluster 0 osdmap e385: 8 total, 8 up, 8 in 2024-03-04T15:59:46.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:46 smithi117 bash[28843]: audit 2024-03-04T15:59:45.042973+0000 mon.c (mon.2) 591 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51"}]: dispatch 2024-03-04T15:59:46.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:46 smithi117 bash[28843]: audit 2024-03-04T15:59:45.046306+0000 mon.a (mon.0) 2576 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51"}]: dispatch 2024-03-04T15:59:46.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:46 smithi117 bash[21480]: cluster 2024-03-04T15:59:44.957639+0000 mgr.y (mgr.24428) 399 : cluster 0 pgmap v588: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 933 B/s rd, 0 B/s wr, 1 op/s 2024-03-04T15:59:46.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:46 smithi117 bash[21480]: audit 2024-03-04T15:59:45.033126+0000 mon.a (mon.0) 2574 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:59:46.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:46 smithi117 bash[21480]: cluster 2024-03-04T15:59:45.040872+0000 mon.a (mon.0) 2575 : cluster 0 osdmap e385: 8 total, 8 up, 8 in 2024-03-04T15:59:46.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:46 smithi117 bash[21480]: audit 2024-03-04T15:59:45.042973+0000 mon.c (mon.2) 591 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51"}]: dispatch 2024-03-04T15:59:46.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:46 smithi117 bash[21480]: audit 2024-03-04T15:59:45.046306+0000 mon.a (mon.0) 2576 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51"}]: dispatch 2024-03-04T15:59:46.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:46 smithi151 bash[23756]: cluster 2024-03-04T15:59:44.957639+0000 mgr.y (mgr.24428) 399 : cluster 0 pgmap v588: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 933 B/s rd, 0 B/s wr, 1 op/s 2024-03-04T15:59:46.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:46 smithi151 bash[23756]: audit 2024-03-04T15:59:45.033126+0000 mon.a (mon.0) 2574 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T15:59:46.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:46 smithi151 bash[23756]: cluster 2024-03-04T15:59:45.040872+0000 mon.a (mon.0) 2575 : cluster 0 osdmap e385: 8 total, 8 up, 8 in 2024-03-04T15:59:46.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:46 smithi151 bash[23756]: audit 2024-03-04T15:59:45.042973+0000 mon.c (mon.2) 591 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51"}]: dispatch 2024-03-04T15:59:46.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:46 smithi151 bash[23756]: audit 2024-03-04T15:59:45.046306+0000 mon.a (mon.0) 2576 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51"}]: dispatch 2024-03-04T15:59:47.057 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:59:46 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:59:47.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:47 smithi117 bash[28843]: audit 2024-03-04T15:59:46.035967+0000 mon.a (mon.0) 2577 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51"}]': finished 2024-03-04T15:59:47.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:47 smithi117 bash[28843]: cluster 2024-03-04T15:59:46.044290+0000 mon.a (mon.0) 2578 : cluster 0 osdmap e386: 8 total, 8 up, 8 in 2024-03-04T15:59:47.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:47 smithi117 bash[28843]: audit 2024-03-04T15:59:46.128105+0000 mon.c (mon.2) 592 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:47.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:47 smithi117 bash[28843]: audit 2024-03-04T15:59:46.128860+0000 mon.a (mon.0) 2579 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:47.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:47 smithi117 bash[28843]: audit 2024-03-04T15:59:46.130126+0000 mon.c (mon.2) 593 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51"}]: dispatch 2024-03-04T15:59:47.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:47 smithi117 bash[28843]: audit 2024-03-04T15:59:46.130760+0000 mon.a (mon.0) 2580 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51"}]: dispatch 2024-03-04T15:59:47.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:47 smithi117 bash[21480]: audit 2024-03-04T15:59:46.035967+0000 mon.a (mon.0) 2577 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51"}]': finished 2024-03-04T15:59:47.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:47 smithi117 bash[21480]: cluster 2024-03-04T15:59:46.044290+0000 mon.a (mon.0) 2578 : cluster 0 osdmap e386: 8 total, 8 up, 8 in 2024-03-04T15:59:47.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:47 smithi117 bash[21480]: audit 2024-03-04T15:59:46.128105+0000 mon.c (mon.2) 592 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:47.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:47 smithi117 bash[21480]: audit 2024-03-04T15:59:46.128860+0000 mon.a (mon.0) 2579 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:47.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:47 smithi117 bash[21480]: audit 2024-03-04T15:59:46.130126+0000 mon.c (mon.2) 593 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51"}]: dispatch 2024-03-04T15:59:47.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:47 smithi117 bash[21480]: audit 2024-03-04T15:59:46.130760+0000 mon.a (mon.0) 2580 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51"}]: dispatch 2024-03-04T15:59:47.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:47 smithi151 bash[23756]: audit 2024-03-04T15:59:46.035967+0000 mon.a (mon.0) 2577 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51"}]': finished 2024-03-04T15:59:47.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:47 smithi151 bash[23756]: cluster 2024-03-04T15:59:46.044290+0000 mon.a (mon.0) 2578 : cluster 0 osdmap e386: 8 total, 8 up, 8 in 2024-03-04T15:59:47.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:47 smithi151 bash[23756]: audit 2024-03-04T15:59:46.128105+0000 mon.c (mon.2) 592 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:47.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:47 smithi151 bash[23756]: audit 2024-03-04T15:59:46.128860+0000 mon.a (mon.0) 2579 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:47.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:47 smithi151 bash[23756]: audit 2024-03-04T15:59:46.130126+0000 mon.c (mon.2) 593 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51"}]: dispatch 2024-03-04T15:59:47.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:47 smithi151 bash[23756]: audit 2024-03-04T15:59:46.130760+0000 mon.a (mon.0) 2580 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-51"}]: dispatch 2024-03-04T15:59:48.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:48 smithi117 bash[21480]: audit 2024-03-04T15:59:46.771542+0000 mgr.y (mgr.24428) 400 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:59:48.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:48 smithi117 bash[21480]: cluster 2024-03-04T15:59:46.959765+0000 mgr.y (mgr.24428) 401 : cluster 0 pgmap v591: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 90 op/s 2024-03-04T15:59:48.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:48 smithi117 bash[21480]: cluster 2024-03-04T15:59:47.050109+0000 mon.a (mon.0) 2581 : cluster 1 Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-04T15:59:48.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:48 smithi117 bash[21480]: cluster 2024-03-04T15:59:47.070180+0000 mon.a (mon.0) 2582 : cluster 0 osdmap e387: 8 total, 8 up, 8 in 2024-03-04T15:59:48.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:48 smithi117 bash[28843]: audit 2024-03-04T15:59:46.771542+0000 mgr.y (mgr.24428) 400 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:59:48.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:48 smithi117 bash[28843]: cluster 2024-03-04T15:59:46.959765+0000 mgr.y (mgr.24428) 401 : cluster 0 pgmap v591: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 90 op/s 2024-03-04T15:59:48.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:48 smithi117 bash[28843]: cluster 2024-03-04T15:59:47.050109+0000 mon.a (mon.0) 2581 : cluster 1 Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-04T15:59:48.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:48 smithi117 bash[28843]: cluster 2024-03-04T15:59:47.070180+0000 mon.a (mon.0) 2582 : cluster 0 osdmap e387: 8 total, 8 up, 8 in 2024-03-04T15:59:48.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:48 smithi151 bash[23756]: audit 2024-03-04T15:59:46.771542+0000 mgr.y (mgr.24428) 400 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:59:48.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:48 smithi151 bash[23756]: cluster 2024-03-04T15:59:46.959765+0000 mgr.y (mgr.24428) 401 : cluster 0 pgmap v591: 228 pgs: 228 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 55 KiB/s rd, 0 B/s wr, 90 op/s 2024-03-04T15:59:48.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:48 smithi151 bash[23756]: cluster 2024-03-04T15:59:47.050109+0000 mon.a (mon.0) 2581 : cluster 1 Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-04T15:59:48.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:48 smithi151 bash[23756]: cluster 2024-03-04T15:59:47.070180+0000 mon.a (mon.0) 2582 : cluster 0 osdmap e387: 8 total, 8 up, 8 in 2024-03-04T15:59:49.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:49 smithi151 bash[23756]: cluster 2024-03-04T15:59:48.082526+0000 mon.a (mon.0) 2583 : cluster 0 osdmap e388: 8 total, 8 up, 8 in 2024-03-04T15:59:49.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:49 smithi151 bash[23756]: audit 2024-03-04T15:59:48.086467+0000 mon.c (mon.2) 594 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:49.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:49 smithi151 bash[23756]: audit 2024-03-04T15:59:48.089871+0000 mon.a (mon.0) 2584 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:49.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:49 smithi117 bash[21480]: cluster 2024-03-04T15:59:48.082526+0000 mon.a (mon.0) 2583 : cluster 0 osdmap e388: 8 total, 8 up, 8 in 2024-03-04T15:59:49.619 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:49 smithi117 bash[21480]: audit 2024-03-04T15:59:48.086467+0000 mon.c (mon.2) 594 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:49.619 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:49 smithi117 bash[21480]: audit 2024-03-04T15:59:48.089871+0000 mon.a (mon.0) 2584 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:49.619 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:49 smithi117 bash[28843]: cluster 2024-03-04T15:59:48.082526+0000 mon.a (mon.0) 2583 : cluster 0 osdmap e388: 8 total, 8 up, 8 in 2024-03-04T15:59:49.619 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:49 smithi117 bash[28843]: audit 2024-03-04T15:59:48.086467+0000 mon.c (mon.2) 594 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:49.619 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:49 smithi117 bash[28843]: audit 2024-03-04T15:59:48.089871+0000 mon.a (mon.0) 2584 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:50.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:50 smithi151 bash[23756]: cluster 2024-03-04T15:59:48.960711+0000 mgr.y (mgr.24428) 402 : cluster 0 pgmap v594: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 107 KiB/s rd, 0 B/s wr, 177 op/s 2024-03-04T15:59:50.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:50 smithi151 bash[23756]: cluster 2024-03-04T15:59:49.075264+0000 mon.a (mon.0) 2585 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:50.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:50 smithi151 bash[23756]: audit 2024-03-04T15:59:49.088338+0000 mon.a (mon.0) 2586 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:59:50.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:50 smithi151 bash[23756]: cluster 2024-03-04T15:59:49.097819+0000 mon.a (mon.0) 2587 : cluster 0 osdmap e389: 8 total, 8 up, 8 in 2024-03-04T15:59:50.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:50 smithi151 bash[23756]: audit 2024-03-04T15:59:49.179167+0000 mon.c (mon.2) 595 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:50.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:50 smithi151 bash[23756]: audit 2024-03-04T15:59:49.179687+0000 mon.a (mon.0) 2588 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:50.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:50 smithi151 bash[23756]: audit 2024-03-04T15:59:49.180477+0000 mon.c (mon.2) 596 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-53"}]: dispatch 2024-03-04T15:59:50.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:50 smithi151 bash[23756]: audit 2024-03-04T15:59:49.180964+0000 mon.a (mon.0) 2589 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-53"}]: dispatch 2024-03-04T15:59:50.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:50 smithi117 bash[21480]: cluster 2024-03-04T15:59:48.960711+0000 mgr.y (mgr.24428) 402 : cluster 0 pgmap v594: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 107 KiB/s rd, 0 B/s wr, 177 op/s 2024-03-04T15:59:50.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:50 smithi117 bash[21480]: cluster 2024-03-04T15:59:49.075264+0000 mon.a (mon.0) 2585 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:50.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:50 smithi117 bash[21480]: audit 2024-03-04T15:59:49.088338+0000 mon.a (mon.0) 2586 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:59:50.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:50 smithi117 bash[21480]: cluster 2024-03-04T15:59:49.097819+0000 mon.a (mon.0) 2587 : cluster 0 osdmap e389: 8 total, 8 up, 8 in 2024-03-04T15:59:50.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:50 smithi117 bash[21480]: audit 2024-03-04T15:59:49.179167+0000 mon.c (mon.2) 595 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:50.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:50 smithi117 bash[21480]: audit 2024-03-04T15:59:49.179687+0000 mon.a (mon.0) 2588 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:50.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:50 smithi117 bash[21480]: audit 2024-03-04T15:59:49.180477+0000 mon.c (mon.2) 596 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-53"}]: dispatch 2024-03-04T15:59:50.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:50 smithi117 bash[21480]: audit 2024-03-04T15:59:49.180964+0000 mon.a (mon.0) 2589 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-53"}]: dispatch 2024-03-04T15:59:50.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:50 smithi117 bash[28843]: cluster 2024-03-04T15:59:48.960711+0000 mgr.y (mgr.24428) 402 : cluster 0 pgmap v594: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail; 107 KiB/s rd, 0 B/s wr, 177 op/s 2024-03-04T15:59:50.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:50 smithi117 bash[28843]: cluster 2024-03-04T15:59:49.075264+0000 mon.a (mon.0) 2585 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:50.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:50 smithi117 bash[28843]: audit 2024-03-04T15:59:49.088338+0000 mon.a (mon.0) 2586 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:59:50.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:50 smithi117 bash[28843]: cluster 2024-03-04T15:59:49.097819+0000 mon.a (mon.0) 2587 : cluster 0 osdmap e389: 8 total, 8 up, 8 in 2024-03-04T15:59:50.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:50 smithi117 bash[28843]: audit 2024-03-04T15:59:49.179167+0000 mon.c (mon.2) 595 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:50.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:50 smithi117 bash[28843]: audit 2024-03-04T15:59:49.179687+0000 mon.a (mon.0) 2588 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:50.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:50 smithi117 bash[28843]: audit 2024-03-04T15:59:49.180477+0000 mon.c (mon.2) 596 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-53"}]: dispatch 2024-03-04T15:59:50.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:50 smithi117 bash[28843]: audit 2024-03-04T15:59:49.180964+0000 mon.a (mon.0) 2589 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-53"}]: dispatch 2024-03-04T15:59:51.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:51 smithi151 bash[23756]: cluster 2024-03-04T15:59:50.098589+0000 mon.a (mon.0) 2590 : cluster 0 osdmap e390: 8 total, 8 up, 8 in 2024-03-04T15:59:51.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:51 smithi117 bash[28843]: cluster 2024-03-04T15:59:50.098589+0000 mon.a (mon.0) 2590 : cluster 0 osdmap e390: 8 total, 8 up, 8 in 2024-03-04T15:59:51.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:51 smithi117 bash[21480]: cluster 2024-03-04T15:59:50.098589+0000 mon.a (mon.0) 2590 : cluster 0 osdmap e390: 8 total, 8 up, 8 in 2024-03-04T15:59:52.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:52 smithi151 bash[23756]: cluster 2024-03-04T15:59:50.961602+0000 mgr.y (mgr.24428) 403 : cluster 0 pgmap v597: 196 pgs: 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:59:52.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:52 smithi151 bash[23756]: cluster 2024-03-04T15:59:51.129890+0000 mon.a (mon.0) 2591 : cluster 0 osdmap e391: 8 total, 8 up, 8 in 2024-03-04T15:59:52.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:52 smithi151 bash[23756]: audit 2024-03-04T15:59:51.133907+0000 mon.c (mon.2) 597 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:52.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:52 smithi151 bash[23756]: audit 2024-03-04T15:59:51.136273+0000 mon.a (mon.0) 2592 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:52 smithi117 bash[28843]: cluster 2024-03-04T15:59:50.961602+0000 mgr.y (mgr.24428) 403 : cluster 0 pgmap v597: 196 pgs: 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:59:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:52 smithi117 bash[28843]: cluster 2024-03-04T15:59:51.129890+0000 mon.a (mon.0) 2591 : cluster 0 osdmap e391: 8 total, 8 up, 8 in 2024-03-04T15:59:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:52 smithi117 bash[28843]: audit 2024-03-04T15:59:51.133907+0000 mon.c (mon.2) 597 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:52 smithi117 bash[28843]: audit 2024-03-04T15:59:51.136273+0000 mon.a (mon.0) 2592 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:52.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:52 smithi117 bash[21480]: cluster 2024-03-04T15:59:50.961602+0000 mgr.y (mgr.24428) 403 : cluster 0 pgmap v597: 196 pgs: 196 active+clean; 8.4 MiB data, 610 MiB used, 715 GiB / 715 GiB avail 2024-03-04T15:59:52.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:52 smithi117 bash[21480]: cluster 2024-03-04T15:59:51.129890+0000 mon.a (mon.0) 2591 : cluster 0 osdmap e391: 8 total, 8 up, 8 in 2024-03-04T15:59:52.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:52 smithi117 bash[21480]: audit 2024-03-04T15:59:51.133907+0000 mon.c (mon.2) 597 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:52.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:52 smithi117 bash[21480]: audit 2024-03-04T15:59:51.136273+0000 mon.a (mon.0) 2592 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:53.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:53 smithi151 bash[23756]: audit 2024-03-04T15:59:52.120814+0000 mon.a (mon.0) 2593 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:59:53.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:53 smithi151 bash[23756]: cluster 2024-03-04T15:59:52.129184+0000 mon.a (mon.0) 2594 : cluster 0 osdmap e392: 8 total, 8 up, 8 in 2024-03-04T15:59:53.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:53 smithi151 bash[23756]: audit 2024-03-04T15:59:52.132854+0000 mon.c (mon.2) 598 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T15:59:53.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:53 smithi151 bash[23756]: audit 2024-03-04T15:59:52.230147+0000 mon.c (mon.2) 599 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:53.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:53 smithi151 bash[23756]: audit 2024-03-04T15:59:52.230759+0000 mon.a (mon.0) 2595 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:53.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:53 smithi151 bash[23756]: audit 2024-03-04T15:59:52.231713+0000 mon.c (mon.2) 600 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-55"}]: dispatch 2024-03-04T15:59:53.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:53 smithi151 bash[23756]: audit 2024-03-04T15:59:52.232166+0000 mon.a (mon.0) 2596 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-55"}]: dispatch 2024-03-04T15:59:53.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:53 smithi117 bash[21480]: audit 2024-03-04T15:59:52.120814+0000 mon.a (mon.0) 2593 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:59:53.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:53 smithi117 bash[21480]: cluster 2024-03-04T15:59:52.129184+0000 mon.a (mon.0) 2594 : cluster 0 osdmap e392: 8 total, 8 up, 8 in 2024-03-04T15:59:53.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:53 smithi117 bash[21480]: audit 2024-03-04T15:59:52.132854+0000 mon.c (mon.2) 598 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T15:59:53.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:53 smithi117 bash[21480]: audit 2024-03-04T15:59:52.230147+0000 mon.c (mon.2) 599 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:53.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:53 smithi117 bash[21480]: audit 2024-03-04T15:59:52.230759+0000 mon.a (mon.0) 2595 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:53.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:53 smithi117 bash[21480]: audit 2024-03-04T15:59:52.231713+0000 mon.c (mon.2) 600 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-55"}]: dispatch 2024-03-04T15:59:53.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:53 smithi117 bash[21480]: audit 2024-03-04T15:59:52.232166+0000 mon.a (mon.0) 2596 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-55"}]: dispatch 2024-03-04T15:59:53.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:53 smithi117 bash[28843]: audit 2024-03-04T15:59:52.120814+0000 mon.a (mon.0) 2593 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:59:53.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:53 smithi117 bash[28843]: cluster 2024-03-04T15:59:52.129184+0000 mon.a (mon.0) 2594 : cluster 0 osdmap e392: 8 total, 8 up, 8 in 2024-03-04T15:59:53.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:53 smithi117 bash[28843]: audit 2024-03-04T15:59:52.132854+0000 mon.c (mon.2) 598 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T15:59:53.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:53 smithi117 bash[28843]: audit 2024-03-04T15:59:52.230147+0000 mon.c (mon.2) 599 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:53.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:53 smithi117 bash[28843]: audit 2024-03-04T15:59:52.230759+0000 mon.a (mon.0) 2595 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:53.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:53 smithi117 bash[28843]: audit 2024-03-04T15:59:52.231713+0000 mon.c (mon.2) 600 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-55"}]: dispatch 2024-03-04T15:59:53.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:53 smithi117 bash[28843]: audit 2024-03-04T15:59:52.232166+0000 mon.a (mon.0) 2596 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-55"}]: dispatch 2024-03-04T15:59:54.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:54 smithi151 bash[23756]: cluster 2024-03-04T15:59:52.962618+0000 mgr.y (mgr.24428) 404 : cluster 0 pgmap v600: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-04T15:59:54.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:54 smithi151 bash[23756]: cluster 2024-03-04T15:59:53.159700+0000 mon.a (mon.0) 2597 : cluster 0 osdmap e393: 8 total, 8 up, 8 in 2024-03-04T15:59:54.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:54 smithi117 bash[21480]: cluster 2024-03-04T15:59:52.962618+0000 mgr.y (mgr.24428) 404 : cluster 0 pgmap v600: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-04T15:59:54.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:54 smithi117 bash[21480]: cluster 2024-03-04T15:59:53.159700+0000 mon.a (mon.0) 2597 : cluster 0 osdmap e393: 8 total, 8 up, 8 in 2024-03-04T15:59:54.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:54 smithi117 bash[28843]: cluster 2024-03-04T15:59:52.962618+0000 mgr.y (mgr.24428) 404 : cluster 0 pgmap v600: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-04T15:59:54.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:54 smithi117 bash[28843]: cluster 2024-03-04T15:59:53.159700+0000 mon.a (mon.0) 2597 : cluster 0 osdmap e393: 8 total, 8 up, 8 in 2024-03-04T15:59:55.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:55 smithi151 bash[23756]: cluster 2024-03-04T15:59:54.174171+0000 mon.a (mon.0) 2598 : cluster 0 osdmap e394: 8 total, 8 up, 8 in 2024-03-04T15:59:55.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:55 smithi151 bash[23756]: audit 2024-03-04T15:59:54.182978+0000 mon.c (mon.2) 601 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:55.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:55 smithi151 bash[23756]: audit 2024-03-04T15:59:54.185820+0000 mon.a (mon.0) 2599 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:55.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:55 smithi117 bash[21480]: cluster 2024-03-04T15:59:54.174171+0000 mon.a (mon.0) 2598 : cluster 0 osdmap e394: 8 total, 8 up, 8 in 2024-03-04T15:59:55.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:55 smithi117 bash[21480]: audit 2024-03-04T15:59:54.182978+0000 mon.c (mon.2) 601 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:55.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:55 smithi117 bash[21480]: audit 2024-03-04T15:59:54.185820+0000 mon.a (mon.0) 2599 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:55.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:55 smithi117 bash[28843]: cluster 2024-03-04T15:59:54.174171+0000 mon.a (mon.0) 2598 : cluster 0 osdmap e394: 8 total, 8 up, 8 in 2024-03-04T15:59:55.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:55 smithi117 bash[28843]: audit 2024-03-04T15:59:54.182978+0000 mon.c (mon.2) 601 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:55.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:55 smithi117 bash[28843]: audit 2024-03-04T15:59:54.185820+0000 mon.a (mon.0) 2599 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:56.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:56 smithi151 bash[23756]: cluster 2024-03-04T15:59:54.963468+0000 mgr.y (mgr.24428) 405 : cluster 0 pgmap v603: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-04T15:59:56.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:56 smithi151 bash[23756]: cluster 2024-03-04T15:59:55.165242+0000 mon.a (mon.0) 2600 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:56.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:56 smithi151 bash[23756]: audit 2024-03-04T15:59:55.175547+0000 mon.a (mon.0) 2601 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:59:56.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:56 smithi151 bash[23756]: cluster 2024-03-04T15:59:55.187214+0000 mon.a (mon.0) 2602 : cluster 0 osdmap e395: 8 total, 8 up, 8 in 2024-03-04T15:59:56.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:56 smithi151 bash[23756]: audit 2024-03-04T15:59:55.283468+0000 mon.c (mon.2) 602 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:56.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:56 smithi151 bash[23756]: audit 2024-03-04T15:59:55.284132+0000 mon.a (mon.0) 2603 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:56.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:56 smithi151 bash[23756]: audit 2024-03-04T15:59:55.285094+0000 mon.c (mon.2) 603 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-57"}]: dispatch 2024-03-04T15:59:56.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:56 smithi151 bash[23756]: audit 2024-03-04T15:59:55.285632+0000 mon.a (mon.0) 2604 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-57"}]: dispatch 2024-03-04T15:59:56.567 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 15:59:56 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:15:59:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T15:59:56.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:56 smithi117 bash[28843]: cluster 2024-03-04T15:59:54.963468+0000 mgr.y (mgr.24428) 405 : cluster 0 pgmap v603: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-04T15:59:56.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:56 smithi117 bash[28843]: cluster 2024-03-04T15:59:55.165242+0000 mon.a (mon.0) 2600 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:56.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:56 smithi117 bash[28843]: audit 2024-03-04T15:59:55.175547+0000 mon.a (mon.0) 2601 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:59:56.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:56 smithi117 bash[28843]: cluster 2024-03-04T15:59:55.187214+0000 mon.a (mon.0) 2602 : cluster 0 osdmap e395: 8 total, 8 up, 8 in 2024-03-04T15:59:56.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:56 smithi117 bash[28843]: audit 2024-03-04T15:59:55.283468+0000 mon.c (mon.2) 602 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:56.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:56 smithi117 bash[28843]: audit 2024-03-04T15:59:55.284132+0000 mon.a (mon.0) 2603 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:56.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:56 smithi117 bash[28843]: audit 2024-03-04T15:59:55.285094+0000 mon.c (mon.2) 603 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-57"}]: dispatch 2024-03-04T15:59:56.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:56 smithi117 bash[28843]: audit 2024-03-04T15:59:55.285632+0000 mon.a (mon.0) 2604 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-57"}]: dispatch 2024-03-04T15:59:56.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:56 smithi117 bash[21480]: cluster 2024-03-04T15:59:54.963468+0000 mgr.y (mgr.24428) 405 : cluster 0 pgmap v603: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 614 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-04T15:59:56.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:56 smithi117 bash[21480]: cluster 2024-03-04T15:59:55.165242+0000 mon.a (mon.0) 2600 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T15:59:56.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:56 smithi117 bash[21480]: audit 2024-03-04T15:59:55.175547+0000 mon.a (mon.0) 2601 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:59:56.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:56 smithi117 bash[21480]: cluster 2024-03-04T15:59:55.187214+0000 mon.a (mon.0) 2602 : cluster 0 osdmap e395: 8 total, 8 up, 8 in 2024-03-04T15:59:56.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:56 smithi117 bash[21480]: audit 2024-03-04T15:59:55.283468+0000 mon.c (mon.2) 602 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:56.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:56 smithi117 bash[21480]: audit 2024-03-04T15:59:55.284132+0000 mon.a (mon.0) 2603 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:56.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:56 smithi117 bash[21480]: audit 2024-03-04T15:59:55.285094+0000 mon.c (mon.2) 603 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-57"}]: dispatch 2024-03-04T15:59:56.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:56 smithi117 bash[21480]: audit 2024-03-04T15:59:55.285632+0000 mon.a (mon.0) 2604 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-57"}]: dispatch 2024-03-04T15:59:57.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 15:59:56 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T15:59:57.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:57 smithi117 bash[21480]: cluster 2024-03-04T15:59:56.193689+0000 mon.a (mon.0) 2605 : cluster 0 osdmap e396: 8 total, 8 up, 8 in 2024-03-04T15:59:57.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:57 smithi117 bash[28843]: cluster 2024-03-04T15:59:56.193689+0000 mon.a (mon.0) 2605 : cluster 0 osdmap e396: 8 total, 8 up, 8 in 2024-03-04T15:59:57.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:57 smithi151 bash[23756]: cluster 2024-03-04T15:59:56.193689+0000 mon.a (mon.0) 2605 : cluster 0 osdmap e396: 8 total, 8 up, 8 in 2024-03-04T15:59:58.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:58 smithi117 bash[21480]: audit 2024-03-04T15:59:56.783149+0000 mgr.y (mgr.24428) 406 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:59:58.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:58 smithi117 bash[21480]: cluster 2024-03-04T15:59:56.965453+0000 mgr.y (mgr.24428) 407 : cluster 0 pgmap v606: 196 pgs: 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-04T15:59:58.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:58 smithi117 bash[21480]: cluster 2024-03-04T15:59:57.201496+0000 mon.a (mon.0) 2606 : cluster 0 osdmap e397: 8 total, 8 up, 8 in 2024-03-04T15:59:58.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:58 smithi117 bash[21480]: audit 2024-03-04T15:59:57.205635+0000 mon.c (mon.2) 604 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:58.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:58 smithi117 bash[21480]: audit 2024-03-04T15:59:57.230357+0000 mon.a (mon.0) 2607 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:58.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:58 smithi117 bash[21480]: audit 2024-03-04T15:59:58.194553+0000 mon.a (mon.0) 2608 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:59:58.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:58 smithi117 bash[21480]: cluster 2024-03-04T15:59:58.201469+0000 mon.a (mon.0) 2609 : cluster 0 osdmap e398: 8 total, 8 up, 8 in 2024-03-04T15:59:58.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:58 smithi117 bash[28843]: audit 2024-03-04T15:59:56.783149+0000 mgr.y (mgr.24428) 406 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:59:58.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:58 smithi117 bash[28843]: cluster 2024-03-04T15:59:56.965453+0000 mgr.y (mgr.24428) 407 : cluster 0 pgmap v606: 196 pgs: 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-04T15:59:58.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:58 smithi117 bash[28843]: cluster 2024-03-04T15:59:57.201496+0000 mon.a (mon.0) 2606 : cluster 0 osdmap e397: 8 total, 8 up, 8 in 2024-03-04T15:59:58.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:58 smithi117 bash[28843]: audit 2024-03-04T15:59:57.205635+0000 mon.c (mon.2) 604 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:58.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:58 smithi117 bash[28843]: audit 2024-03-04T15:59:57.230357+0000 mon.a (mon.0) 2607 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:58.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:58 smithi117 bash[28843]: audit 2024-03-04T15:59:58.194553+0000 mon.a (mon.0) 2608 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:59:58.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:58 smithi117 bash[28843]: cluster 2024-03-04T15:59:58.201469+0000 mon.a (mon.0) 2609 : cluster 0 osdmap e398: 8 total, 8 up, 8 in 2024-03-04T15:59:58.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:58 smithi151 bash[23756]: audit 2024-03-04T15:59:56.783149+0000 mgr.y (mgr.24428) 406 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T15:59:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:58 smithi151 bash[23756]: cluster 2024-03-04T15:59:56.965453+0000 mgr.y (mgr.24428) 407 : cluster 0 pgmap v606: 196 pgs: 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-04T15:59:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:58 smithi151 bash[23756]: cluster 2024-03-04T15:59:57.201496+0000 mon.a (mon.0) 2606 : cluster 0 osdmap e397: 8 total, 8 up, 8 in 2024-03-04T15:59:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:58 smithi151 bash[23756]: audit 2024-03-04T15:59:57.205635+0000 mon.c (mon.2) 604 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:58 smithi151 bash[23756]: audit 2024-03-04T15:59:57.230357+0000 mon.a (mon.0) 2607 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T15:59:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:58 smithi151 bash[23756]: audit 2024-03-04T15:59:58.194553+0000 mon.a (mon.0) 2608 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T15:59:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:58 smithi151 bash[23756]: cluster 2024-03-04T15:59:58.201469+0000 mon.a (mon.0) 2609 : cluster 0 osdmap e398: 8 total, 8 up, 8 in 2024-03-04T15:59:59.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:59 smithi117 bash[28843]: audit 2024-03-04T15:59:58.216168+0000 mon.c (mon.2) 605 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T15:59:59.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:59 smithi117 bash[28843]: audit 2024-03-04T15:59:58.314445+0000 mon.c (mon.2) 606 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:59.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:59 smithi117 bash[28843]: audit 2024-03-04T15:59:58.315103+0000 mon.a (mon.0) 2610 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:59.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:59 smithi117 bash[28843]: audit 2024-03-04T15:59:58.316202+0000 mon.c (mon.2) 607 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-59"}]: dispatch 2024-03-04T15:59:59.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:59 smithi117 bash[28843]: audit 2024-03-04T15:59:58.316658+0000 mon.a (mon.0) 2611 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-59"}]: dispatch 2024-03-04T15:59:59.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:59 smithi117 bash[28843]: audit 2024-03-04T15:59:59.047023+0000 mon.a (mon.0) 2612 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:59:59.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 15:59:59 smithi117 bash[28843]: audit 2024-03-04T15:59:59.050287+0000 mon.c (mon.2) 608 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:59:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:59 smithi117 bash[21480]: audit 2024-03-04T15:59:58.216168+0000 mon.c (mon.2) 605 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T15:59:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:59 smithi117 bash[21480]: audit 2024-03-04T15:59:58.314445+0000 mon.c (mon.2) 606 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:59 smithi117 bash[21480]: audit 2024-03-04T15:59:58.315103+0000 mon.a (mon.0) 2610 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:59 smithi117 bash[21480]: audit 2024-03-04T15:59:58.316202+0000 mon.c (mon.2) 607 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-59"}]: dispatch 2024-03-04T15:59:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:59 smithi117 bash[21480]: audit 2024-03-04T15:59:58.316658+0000 mon.a (mon.0) 2611 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-59"}]: dispatch 2024-03-04T15:59:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:59 smithi117 bash[21480]: audit 2024-03-04T15:59:59.047023+0000 mon.a (mon.0) 2612 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:59:59.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 15:59:59 smithi117 bash[21480]: audit 2024-03-04T15:59:59.050287+0000 mon.c (mon.2) 608 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T15:59:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:59 smithi151 bash[23756]: audit 2024-03-04T15:59:58.216168+0000 mon.c (mon.2) 605 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T15:59:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:59 smithi151 bash[23756]: audit 2024-03-04T15:59:58.314445+0000 mon.c (mon.2) 606 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:59 smithi151 bash[23756]: audit 2024-03-04T15:59:58.315103+0000 mon.a (mon.0) 2610 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T15:59:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:59 smithi151 bash[23756]: audit 2024-03-04T15:59:58.316202+0000 mon.c (mon.2) 607 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-59"}]: dispatch 2024-03-04T15:59:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:59 smithi151 bash[23756]: audit 2024-03-04T15:59:58.316658+0000 mon.a (mon.0) 2611 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-59"}]: dispatch 2024-03-04T15:59:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:59 smithi151 bash[23756]: audit 2024-03-04T15:59:59.047023+0000 mon.a (mon.0) 2612 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T15:59:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 15:59:59 smithi151 bash[23756]: audit 2024-03-04T15:59:59.050287+0000 mon.c (mon.2) 608 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:00:00.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:00 smithi117 bash[21480]: cluster 2024-03-04T15:59:58.966267+0000 mgr.y (mgr.24428) 408 : cluster 0 pgmap v609: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-04T16:00:00.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:00 smithi117 bash[21480]: cluster 2024-03-04T15:59:59.234953+0000 mon.a (mon.0) 2613 : cluster 0 osdmap e399: 8 total, 8 up, 8 in 2024-03-04T16:00:00.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:00 smithi117 bash[21480]: cluster 2024-03-04T16:00:00.000202+0000 mon.a (mon.0) 2614 : cluster 3 Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-03-04T16:00:00.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:00 smithi117 bash[21480]: cluster 2024-03-04T16:00:00.000294+0000 mon.a (mon.0) 2615 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-03-04T16:00:00.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:00 smithi117 bash[21480]: cluster 2024-03-04T16:00:00.000351+0000 mon.a (mon.0) 2616 : cluster 3 application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-04T16:00:00.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:00 smithi117 bash[21480]: cluster 2024-03-04T16:00:00.000421+0000 mon.a (mon.0) 2617 : cluster 3 use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-04T16:00:00.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:00 smithi117 bash[28843]: cluster 2024-03-04T15:59:58.966267+0000 mgr.y (mgr.24428) 408 : cluster 0 pgmap v609: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-04T16:00:00.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:00 smithi117 bash[28843]: cluster 2024-03-04T15:59:59.234953+0000 mon.a (mon.0) 2613 : cluster 0 osdmap e399: 8 total, 8 up, 8 in 2024-03-04T16:00:00.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:00 smithi117 bash[28843]: cluster 2024-03-04T16:00:00.000202+0000 mon.a (mon.0) 2614 : cluster 3 Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-03-04T16:00:00.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:00 smithi117 bash[28843]: cluster 2024-03-04T16:00:00.000294+0000 mon.a (mon.0) 2615 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-03-04T16:00:00.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:00 smithi117 bash[28843]: cluster 2024-03-04T16:00:00.000351+0000 mon.a (mon.0) 2616 : cluster 3 application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-04T16:00:00.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:00 smithi117 bash[28843]: cluster 2024-03-04T16:00:00.000421+0000 mon.a (mon.0) 2617 : cluster 3 use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-04T16:00:00.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:00 smithi151 bash[23756]: cluster 2024-03-04T15:59:58.966267+0000 mgr.y (mgr.24428) 408 : cluster 0 pgmap v609: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-04T16:00:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:00 smithi151 bash[23756]: cluster 2024-03-04T15:59:59.234953+0000 mon.a (mon.0) 2613 : cluster 0 osdmap e399: 8 total, 8 up, 8 in 2024-03-04T16:00:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:00 smithi151 bash[23756]: cluster 2024-03-04T16:00:00.000202+0000 mon.a (mon.0) 2614 : cluster 3 Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled 2024-03-04T16:00:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:00 smithi151 bash[23756]: cluster 2024-03-04T16:00:00.000294+0000 mon.a (mon.0) 2615 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-03-04T16:00:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:00 smithi151 bash[23756]: cluster 2024-03-04T16:00:00.000351+0000 mon.a (mon.0) 2616 : cluster 3 application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-04T16:00:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:00 smithi151 bash[23756]: cluster 2024-03-04T16:00:00.000421+0000 mon.a (mon.0) 2617 : cluster 3 use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-04T16:00:01.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:01 smithi117 bash[21480]: cluster 2024-03-04T16:00:00.240086+0000 mon.a (mon.0) 2618 : cluster 0 osdmap e400: 8 total, 8 up, 8 in 2024-03-04T16:00:01.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:01 smithi117 bash[21480]: audit 2024-03-04T16:00:00.246329+0000 mon.c (mon.2) 609 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:00:01.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:01 smithi117 bash[21480]: audit 2024-03-04T16:00:00.267598+0000 mon.a (mon.0) 2619 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:00:01.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:01 smithi117 bash[21480]: audit 2024-03-04T16:00:01.233751+0000 mon.a (mon.0) 2620 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:00:01.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:01 smithi117 bash[28843]: cluster 2024-03-04T16:00:00.240086+0000 mon.a (mon.0) 2618 : cluster 0 osdmap e400: 8 total, 8 up, 8 in 2024-03-04T16:00:01.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:01 smithi117 bash[28843]: audit 2024-03-04T16:00:00.246329+0000 mon.c (mon.2) 609 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:00:01.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:01 smithi117 bash[28843]: audit 2024-03-04T16:00:00.267598+0000 mon.a (mon.0) 2619 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:00:01.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:01 smithi117 bash[28843]: audit 2024-03-04T16:00:01.233751+0000 mon.a (mon.0) 2620 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:00:01.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:01 smithi151 bash[23756]: cluster 2024-03-04T16:00:00.240086+0000 mon.a (mon.0) 2618 : cluster 0 osdmap e400: 8 total, 8 up, 8 in 2024-03-04T16:00:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:01 smithi151 bash[23756]: audit 2024-03-04T16:00:00.246329+0000 mon.c (mon.2) 609 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:00:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:01 smithi151 bash[23756]: audit 2024-03-04T16:00:00.267598+0000 mon.a (mon.0) 2619 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:00:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:01 smithi151 bash[23756]: audit 2024-03-04T16:00:01.233751+0000 mon.a (mon.0) 2620 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:00:02.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:02 smithi117 bash[21480]: cluster 2024-03-04T16:00:00.967136+0000 mgr.y (mgr.24428) 409 : cluster 0 pgmap v612: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:00:02.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:02 smithi117 bash[21480]: cluster 2024-03-04T16:00:01.243470+0000 mon.a (mon.0) 2621 : cluster 0 osdmap e401: 8 total, 8 up, 8 in 2024-03-04T16:00:02.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:02 smithi117 bash[21480]: audit 2024-03-04T16:00:01.247332+0000 mon.c (mon.2) 610 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:00:02.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:02 smithi117 bash[21480]: cluster 2024-03-04T16:00:01.251194+0000 mon.a (mon.0) 2622 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:00:02.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:02 smithi117 bash[21480]: audit 2024-03-04T16:00:01.266336+0000 mon.c (mon.2) 611 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:00:02.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:02 smithi117 bash[21480]: audit 2024-03-04T16:00:01.291932+0000 mon.a (mon.0) 2623 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:00:02.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:02 smithi117 bash[21480]: audit 2024-03-04T16:00:02.236418+0000 mon.a (mon.0) 2624 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:00:02.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:02 smithi117 bash[21480]: cluster 2024-03-04T16:00:02.246594+0000 mon.a (mon.0) 2625 : cluster 0 osdmap e402: 8 total, 8 up, 8 in 2024-03-04T16:00:02.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:02 smithi117 bash[28843]: cluster 2024-03-04T16:00:00.967136+0000 mgr.y (mgr.24428) 409 : cluster 0 pgmap v612: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:00:02.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:02 smithi117 bash[28843]: cluster 2024-03-04T16:00:01.243470+0000 mon.a (mon.0) 2621 : cluster 0 osdmap e401: 8 total, 8 up, 8 in 2024-03-04T16:00:02.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:02 smithi117 bash[28843]: audit 2024-03-04T16:00:01.247332+0000 mon.c (mon.2) 610 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:00:02.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:02 smithi117 bash[28843]: cluster 2024-03-04T16:00:01.251194+0000 mon.a (mon.0) 2622 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:00:02.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:02 smithi117 bash[28843]: audit 2024-03-04T16:00:01.266336+0000 mon.c (mon.2) 611 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:00:02.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:02 smithi117 bash[28843]: audit 2024-03-04T16:00:01.291932+0000 mon.a (mon.0) 2623 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:00:02.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:02 smithi117 bash[28843]: audit 2024-03-04T16:00:02.236418+0000 mon.a (mon.0) 2624 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:00:02.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:02 smithi117 bash[28843]: cluster 2024-03-04T16:00:02.246594+0000 mon.a (mon.0) 2625 : cluster 0 osdmap e402: 8 total, 8 up, 8 in 2024-03-04T16:00:02.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:02 smithi151 bash[23756]: cluster 2024-03-04T16:00:00.967136+0000 mgr.y (mgr.24428) 409 : cluster 0 pgmap v612: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:00:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:02 smithi151 bash[23756]: cluster 2024-03-04T16:00:01.243470+0000 mon.a (mon.0) 2621 : cluster 0 osdmap e401: 8 total, 8 up, 8 in 2024-03-04T16:00:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:02 smithi151 bash[23756]: audit 2024-03-04T16:00:01.247332+0000 mon.c (mon.2) 610 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:00:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:02 smithi151 bash[23756]: cluster 2024-03-04T16:00:01.251194+0000 mon.a (mon.0) 2622 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:00:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:02 smithi151 bash[23756]: audit 2024-03-04T16:00:01.266336+0000 mon.c (mon.2) 611 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:00:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:02 smithi151 bash[23756]: audit 2024-03-04T16:00:01.291932+0000 mon.a (mon.0) 2623 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:00:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:02 smithi151 bash[23756]: audit 2024-03-04T16:00:02.236418+0000 mon.a (mon.0) 2624 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:00:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:02 smithi151 bash[23756]: cluster 2024-03-04T16:00:02.246594+0000 mon.a (mon.0) 2625 : cluster 0 osdmap e402: 8 total, 8 up, 8 in 2024-03-04T16:00:03.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:03 smithi117 bash[21480]: audit 2024-03-04T16:00:02.334831+0000 mon.c (mon.2) 612 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:00:03.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:03 smithi117 bash[21480]: audit 2024-03-04T16:00:02.335771+0000 mon.a (mon.0) 2626 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:00:03.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:03 smithi117 bash[21480]: audit 2024-03-04T16:00:02.336917+0000 mon.c (mon.2) 613 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-61"}]: dispatch 2024-03-04T16:00:03.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:03 smithi117 bash[21480]: audit 2024-03-04T16:00:02.337731+0000 mon.a (mon.0) 2627 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-61"}]: dispatch 2024-03-04T16:00:03.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:03 smithi117 bash[28843]: audit 2024-03-04T16:00:02.334831+0000 mon.c (mon.2) 612 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:00:03.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:03 smithi117 bash[28843]: audit 2024-03-04T16:00:02.335771+0000 mon.a (mon.0) 2626 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:00:03.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:03 smithi117 bash[28843]: audit 2024-03-04T16:00:02.336917+0000 mon.c (mon.2) 613 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-61"}]: dispatch 2024-03-04T16:00:03.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:03 smithi117 bash[28843]: audit 2024-03-04T16:00:02.337731+0000 mon.a (mon.0) 2627 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-61"}]: dispatch 2024-03-04T16:00:03.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:03 smithi151 bash[23756]: audit 2024-03-04T16:00:02.334831+0000 mon.c (mon.2) 612 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:00:03.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:03 smithi151 bash[23756]: audit 2024-03-04T16:00:02.335771+0000 mon.a (mon.0) 2626 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:00:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:03 smithi151 bash[23756]: audit 2024-03-04T16:00:02.336917+0000 mon.c (mon.2) 613 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-61"}]: dispatch 2024-03-04T16:00:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:03 smithi151 bash[23756]: audit 2024-03-04T16:00:02.337731+0000 mon.a (mon.0) 2627 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-61"}]: dispatch 2024-03-04T16:00:04.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:04 smithi151 bash[23756]: cluster 2024-03-04T16:00:02.967981+0000 mgr.y (mgr.24428) 410 : cluster 0 pgmap v615: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T16:00:04.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:04 smithi151 bash[23756]: cluster 2024-03-04T16:00:03.319954+0000 mon.a (mon.0) 2628 : cluster 0 osdmap e403: 8 total, 8 up, 8 in 2024-03-04T16:00:04.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:04 smithi117 bash[21480]: cluster 2024-03-04T16:00:02.967981+0000 mgr.y (mgr.24428) 410 : cluster 0 pgmap v615: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T16:00:04.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:04 smithi117 bash[21480]: cluster 2024-03-04T16:00:03.319954+0000 mon.a (mon.0) 2628 : cluster 0 osdmap e403: 8 total, 8 up, 8 in 2024-03-04T16:00:04.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:04 smithi117 bash[28843]: cluster 2024-03-04T16:00:02.967981+0000 mgr.y (mgr.24428) 410 : cluster 0 pgmap v615: 228 pgs: 228 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T16:00:04.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:04 smithi117 bash[28843]: cluster 2024-03-04T16:00:03.319954+0000 mon.a (mon.0) 2628 : cluster 0 osdmap e403: 8 total, 8 up, 8 in 2024-03-04T16:00:05.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:05 smithi151 bash[23756]: cluster 2024-03-04T16:00:04.324788+0000 mon.a (mon.0) 2629 : cluster 0 osdmap e404: 8 total, 8 up, 8 in 2024-03-04T16:00:05.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:05 smithi151 bash[23756]: audit 2024-03-04T16:00:04.330112+0000 mon.c (mon.2) 614 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:00:05.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:05 smithi151 bash[23756]: audit 2024-03-04T16:00:04.352120+0000 mon.a (mon.0) 2630 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:00:05.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:05 smithi151 bash[23756]: cluster 2024-03-04T16:00:04.968820+0000 mgr.y (mgr.24428) 411 : cluster 0 pgmap v618: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T16:00:05.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:05 smithi151 bash[23756]: audit 2024-03-04T16:00:05.316669+0000 mon.a (mon.0) 2631 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:00:05.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:05 smithi151 bash[23756]: cluster 2024-03-04T16:00:05.324866+0000 mon.a (mon.0) 2632 : cluster 0 osdmap e405: 8 total, 8 up, 8 in 2024-03-04T16:00:05.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:05 smithi117 bash[21480]: cluster 2024-03-04T16:00:04.324788+0000 mon.a (mon.0) 2629 : cluster 0 osdmap e404: 8 total, 8 up, 8 in 2024-03-04T16:00:05.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:05 smithi117 bash[21480]: audit 2024-03-04T16:00:04.330112+0000 mon.c (mon.2) 614 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:00:05.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:05 smithi117 bash[21480]: audit 2024-03-04T16:00:04.352120+0000 mon.a (mon.0) 2630 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:00:05.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:05 smithi117 bash[21480]: cluster 2024-03-04T16:00:04.968820+0000 mgr.y (mgr.24428) 411 : cluster 0 pgmap v618: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T16:00:05.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:05 smithi117 bash[21480]: audit 2024-03-04T16:00:05.316669+0000 mon.a (mon.0) 2631 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:00:05.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:05 smithi117 bash[21480]: cluster 2024-03-04T16:00:05.324866+0000 mon.a (mon.0) 2632 : cluster 0 osdmap e405: 8 total, 8 up, 8 in 2024-03-04T16:00:05.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:05 smithi117 bash[28843]: cluster 2024-03-04T16:00:04.324788+0000 mon.a (mon.0) 2629 : cluster 0 osdmap e404: 8 total, 8 up, 8 in 2024-03-04T16:00:05.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:05 smithi117 bash[28843]: audit 2024-03-04T16:00:04.330112+0000 mon.c (mon.2) 614 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:00:05.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:05 smithi117 bash[28843]: audit 2024-03-04T16:00:04.352120+0000 mon.a (mon.0) 2630 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:00:05.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:05 smithi117 bash[28843]: cluster 2024-03-04T16:00:04.968820+0000 mgr.y (mgr.24428) 411 : cluster 0 pgmap v618: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 628 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T16:00:05.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:05 smithi117 bash[28843]: audit 2024-03-04T16:00:05.316669+0000 mon.a (mon.0) 2631 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:00:05.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:05 smithi117 bash[28843]: cluster 2024-03-04T16:00:05.324866+0000 mon.a (mon.0) 2632 : cluster 0 osdmap e405: 8 total, 8 up, 8 in 2024-03-04T16:00:06.367 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:00:06 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:00:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:00:07.250 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:06 smithi151 bash[23756]: audit 2024-03-04T16:00:05.327769+0000 mon.c (mon.2) 615 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:00:07.250 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:06 smithi151 bash[23756]: audit 2024-03-04T16:00:05.344950+0000 mon.c (mon.2) 616 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:00:07.251 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:06 smithi151 bash[23756]: audit 2024-03-04T16:00:05.360888+0000 mon.a (mon.0) 2633 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:00:07.251 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:06 smithi151 bash[23756]: audit 2024-03-04T16:00:06.350957+0000 mon.a (mon.0) 2634 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:00:07.251 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:06 smithi151 bash[23756]: cluster 2024-03-04T16:00:06.358222+0000 mon.a (mon.0) 2635 : cluster 0 osdmap e406: 8 total, 8 up, 8 in 2024-03-04T16:00:07.251 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:06 smithi117 bash[21480]: audit 2024-03-04T16:00:05.327769+0000 mon.c (mon.2) 615 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:00:07.251 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:06 smithi117 bash[21480]: audit 2024-03-04T16:00:05.344950+0000 mon.c (mon.2) 616 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:00:07.251 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:06 smithi117 bash[21480]: audit 2024-03-04T16:00:05.360888+0000 mon.a (mon.0) 2633 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:00:07.251 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:06 smithi117 bash[21480]: audit 2024-03-04T16:00:06.350957+0000 mon.a (mon.0) 2634 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:00:07.252 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:06 smithi117 bash[21480]: cluster 2024-03-04T16:00:06.358222+0000 mon.a (mon.0) 2635 : cluster 0 osdmap e406: 8 total, 8 up, 8 in 2024-03-04T16:00:07.252 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:00:06 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:00:07.252 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:06 smithi117 bash[28843]: audit 2024-03-04T16:00:05.327769+0000 mon.c (mon.2) 615 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:00:07.252 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:06 smithi117 bash[28843]: audit 2024-03-04T16:00:05.344950+0000 mon.c (mon.2) 616 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:00:07.252 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:06 smithi117 bash[28843]: audit 2024-03-04T16:00:05.360888+0000 mon.a (mon.0) 2633 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:00:07.253 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:06 smithi117 bash[28843]: audit 2024-03-04T16:00:06.350957+0000 mon.a (mon.0) 2634 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:00:07.253 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:06 smithi117 bash[28843]: cluster 2024-03-04T16:00:06.358222+0000 mon.a (mon.0) 2635 : cluster 0 osdmap e406: 8 total, 8 up, 8 in 2024-03-04T16:00:07.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:07 smithi151 bash[23756]: audit 2024-03-04T16:00:06.794708+0000 mgr.y (mgr.24428) 412 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:00:07.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:07 smithi151 bash[23756]: cluster 2024-03-04T16:00:06.970922+0000 mgr.y (mgr.24428) 413 : cluster 0 pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T16:00:07.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:07 smithi117 bash[28843]: audit 2024-03-04T16:00:06.794708+0000 mgr.y (mgr.24428) 412 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:00:07.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:07 smithi117 bash[28843]: cluster 2024-03-04T16:00:06.970922+0000 mgr.y (mgr.24428) 413 : cluster 0 pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T16:00:07.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:07 smithi117 bash[21480]: audit 2024-03-04T16:00:06.794708+0000 mgr.y (mgr.24428) 412 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:00:07.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:07 smithi117 bash[21480]: cluster 2024-03-04T16:00:06.970922+0000 mgr.y (mgr.24428) 413 : cluster 0 pgmap v621: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T16:00:08.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:08 smithi151 bash[23756]: cluster 2024-03-04T16:00:07.370184+0000 mon.a (mon.0) 2636 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:00:08.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:08 smithi151 bash[23756]: cluster 2024-03-04T16:00:07.388959+0000 mon.a (mon.0) 2637 : cluster 0 osdmap e407: 8 total, 8 up, 8 in 2024-03-04T16:00:08.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:08 smithi117 bash[21480]: cluster 2024-03-04T16:00:07.370184+0000 mon.a (mon.0) 2636 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:00:08.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:08 smithi117 bash[21480]: cluster 2024-03-04T16:00:07.388959+0000 mon.a (mon.0) 2637 : cluster 0 osdmap e407: 8 total, 8 up, 8 in 2024-03-04T16:00:08.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:08 smithi117 bash[28843]: cluster 2024-03-04T16:00:07.370184+0000 mon.a (mon.0) 2636 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:00:08.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:08 smithi117 bash[28843]: cluster 2024-03-04T16:00:07.388959+0000 mon.a (mon.0) 2637 : cluster 0 osdmap e407: 8 total, 8 up, 8 in 2024-03-04T16:00:09.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:09 smithi151 bash[23756]: cluster 2024-03-04T16:00:08.389074+0000 mon.a (mon.0) 2638 : cluster 0 osdmap e408: 8 total, 8 up, 8 in 2024-03-04T16:00:09.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:09 smithi151 bash[23756]: cluster 2024-03-04T16:00:08.971769+0000 mgr.y (mgr.24428) 414 : cluster 0 pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T16:00:09.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:09 smithi117 bash[21480]: cluster 2024-03-04T16:00:08.389074+0000 mon.a (mon.0) 2638 : cluster 0 osdmap e408: 8 total, 8 up, 8 in 2024-03-04T16:00:09.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:09 smithi117 bash[21480]: cluster 2024-03-04T16:00:08.971769+0000 mgr.y (mgr.24428) 414 : cluster 0 pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T16:00:09.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:09 smithi117 bash[28843]: cluster 2024-03-04T16:00:08.389074+0000 mon.a (mon.0) 2638 : cluster 0 osdmap e408: 8 total, 8 up, 8 in 2024-03-04T16:00:09.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:09 smithi117 bash[28843]: cluster 2024-03-04T16:00:08.971769+0000 mgr.y (mgr.24428) 414 : cluster 0 pgmap v624: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T16:00:10.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:10 smithi151 bash[23756]: cluster 2024-03-04T16:00:09.422377+0000 mon.a (mon.0) 2639 : cluster 0 osdmap e409: 8 total, 8 up, 8 in 2024-03-04T16:00:10.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:10 smithi117 bash[28843]: cluster 2024-03-04T16:00:09.422377+0000 mon.a (mon.0) 2639 : cluster 0 osdmap e409: 8 total, 8 up, 8 in 2024-03-04T16:00:10.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:10 smithi117 bash[21480]: cluster 2024-03-04T16:00:09.422377+0000 mon.a (mon.0) 2639 : cluster 0 osdmap e409: 8 total, 8 up, 8 in 2024-03-04T16:00:11.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:11 smithi151 bash[23756]: cluster 2024-03-04T16:00:10.432909+0000 mon.a (mon.0) 2640 : cluster 0 osdmap e410: 8 total, 8 up, 8 in 2024-03-04T16:00:11.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:11 smithi151 bash[23756]: cluster 2024-03-04T16:00:10.972618+0000 mgr.y (mgr.24428) 415 : cluster 0 pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:00:11.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:11 smithi117 bash[28843]: cluster 2024-03-04T16:00:10.432909+0000 mon.a (mon.0) 2640 : cluster 0 osdmap e410: 8 total, 8 up, 8 in 2024-03-04T16:00:11.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:11 smithi117 bash[28843]: cluster 2024-03-04T16:00:10.972618+0000 mgr.y (mgr.24428) 415 : cluster 0 pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:00:11.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:11 smithi117 bash[21480]: cluster 2024-03-04T16:00:10.432909+0000 mon.a (mon.0) 2640 : cluster 0 osdmap e410: 8 total, 8 up, 8 in 2024-03-04T16:00:11.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:11 smithi117 bash[21480]: cluster 2024-03-04T16:00:10.972618+0000 mgr.y (mgr.24428) 415 : cluster 0 pgmap v627: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:00:14.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:14 smithi117 bash[21480]: cluster 2024-03-04T16:00:12.974891+0000 mgr.y (mgr.24428) 416 : cluster 0 pgmap v628: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-03-04T16:00:14.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:14 smithi117 bash[21480]: cluster 2024-03-04T16:00:13.780219+0000 mon.a (mon.0) 2641 : cluster 0 osdmap e411: 8 total, 8 up, 8 in 2024-03-04T16:00:14.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:14 smithi117 bash[28843]: cluster 2024-03-04T16:00:12.974891+0000 mgr.y (mgr.24428) 416 : cluster 0 pgmap v628: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-03-04T16:00:14.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:14 smithi117 bash[28843]: cluster 2024-03-04T16:00:13.780219+0000 mon.a (mon.0) 2641 : cluster 0 osdmap e411: 8 total, 8 up, 8 in 2024-03-04T16:00:14.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:14 smithi151 bash[23756]: cluster 2024-03-04T16:00:12.974891+0000 mgr.y (mgr.24428) 416 : cluster 0 pgmap v628: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2024-03-04T16:00:14.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:14 smithi151 bash[23756]: cluster 2024-03-04T16:00:13.780219+0000 mon.a (mon.0) 2641 : cluster 0 osdmap e411: 8 total, 8 up, 8 in 2024-03-04T16:00:15.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:15 smithi117 bash[21480]: audit 2024-03-04T16:00:14.083921+0000 mon.c (mon.2) 617 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:00:15.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:15 smithi117 bash[28843]: audit 2024-03-04T16:00:14.083921+0000 mon.c (mon.2) 617 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:00:15.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:15 smithi151 bash[23756]: audit 2024-03-04T16:00:14.083921+0000 mon.c (mon.2) 617 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:00:16.316 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:00:16 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:00:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:00:16.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:16 smithi117 bash[21480]: cluster 2024-03-04T16:00:14.975725+0000 mgr.y (mgr.24428) 417 : cluster 0 pgmap v630: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-04T16:00:16.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:16 smithi117 bash[28843]: cluster 2024-03-04T16:00:14.975725+0000 mgr.y (mgr.24428) 417 : cluster 0 pgmap v630: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-04T16:00:16.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:16 smithi151 bash[23756]: cluster 2024-03-04T16:00:14.975725+0000 mgr.y (mgr.24428) 417 : cluster 0 pgmap v630: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.3 KiB/s wr, 4 op/s 2024-03-04T16:00:17.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:00:16 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:00:18.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:18 smithi117 bash[28843]: audit 2024-03-04T16:00:16.806191+0000 mgr.y (mgr.24428) 418 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:00:18.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:18 smithi117 bash[28843]: cluster 2024-03-04T16:00:16.977961+0000 mgr.y (mgr.24428) 419 : cluster 0 pgmap v631: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-03-04T16:00:18.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:18 smithi117 bash[21480]: audit 2024-03-04T16:00:16.806191+0000 mgr.y (mgr.24428) 418 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:00:18.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:18 smithi117 bash[21480]: cluster 2024-03-04T16:00:16.977961+0000 mgr.y (mgr.24428) 419 : cluster 0 pgmap v631: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-03-04T16:00:18.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:18 smithi151 bash[23756]: audit 2024-03-04T16:00:16.806191+0000 mgr.y (mgr.24428) 418 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:00:18.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:18 smithi151 bash[23756]: cluster 2024-03-04T16:00:16.977961+0000 mgr.y (mgr.24428) 419 : cluster 0 pgmap v631: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-03-04T16:00:20.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:20 smithi117 bash[21480]: cluster 2024-03-04T16:00:18.978826+0000 mgr.y (mgr.24428) 420 : cluster 0 pgmap v632: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 957 B/s wr, 3 op/s 2024-03-04T16:00:20.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:20 smithi117 bash[28843]: cluster 2024-03-04T16:00:18.978826+0000 mgr.y (mgr.24428) 420 : cluster 0 pgmap v632: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 957 B/s wr, 3 op/s 2024-03-04T16:00:20.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:20 smithi151 bash[23756]: cluster 2024-03-04T16:00:18.978826+0000 mgr.y (mgr.24428) 420 : cluster 0 pgmap v632: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 957 B/s wr, 3 op/s 2024-03-04T16:00:22.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:22 smithi117 bash[21480]: cluster 2024-03-04T16:00:20.979682+0000 mgr.y (mgr.24428) 421 : cluster 0 pgmap v633: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 818 B/s wr, 3 op/s 2024-03-04T16:00:22.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:22 smithi117 bash[21480]: cluster 2024-03-04T16:00:21.078330+0000 mon.a (mon.0) 2642 : cluster 0 osdmap e412: 8 total, 8 up, 8 in 2024-03-04T16:00:22.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:22 smithi117 bash[28843]: cluster 2024-03-04T16:00:20.979682+0000 mgr.y (mgr.24428) 421 : cluster 0 pgmap v633: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 818 B/s wr, 3 op/s 2024-03-04T16:00:22.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:22 smithi117 bash[28843]: cluster 2024-03-04T16:00:21.078330+0000 mon.a (mon.0) 2642 : cluster 0 osdmap e412: 8 total, 8 up, 8 in 2024-03-04T16:00:22.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:22 smithi151 bash[23756]: cluster 2024-03-04T16:00:20.979682+0000 mgr.y (mgr.24428) 421 : cluster 0 pgmap v633: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 818 B/s wr, 3 op/s 2024-03-04T16:00:22.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:22 smithi151 bash[23756]: cluster 2024-03-04T16:00:21.078330+0000 mon.a (mon.0) 2642 : cluster 0 osdmap e412: 8 total, 8 up, 8 in 2024-03-04T16:00:24.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:24 smithi151 bash[23756]: cluster 2024-03-04T16:00:22.981972+0000 mgr.y (mgr.24428) 422 : cluster 0 pgmap v635: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:00:24.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:24 smithi151 bash[23756]: cluster 2024-03-04T16:00:23.785030+0000 mon.a (mon.0) 2643 : cluster 0 osdmap e413: 8 total, 8 up, 8 in 2024-03-04T16:00:24.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:24 smithi117 bash[21480]: cluster 2024-03-04T16:00:22.981972+0000 mgr.y (mgr.24428) 422 : cluster 0 pgmap v635: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:00:24.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:24 smithi117 bash[21480]: cluster 2024-03-04T16:00:23.785030+0000 mon.a (mon.0) 2643 : cluster 0 osdmap e413: 8 total, 8 up, 8 in 2024-03-04T16:00:24.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:24 smithi117 bash[28843]: cluster 2024-03-04T16:00:22.981972+0000 mgr.y (mgr.24428) 422 : cluster 0 pgmap v635: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:00:24.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:24 smithi117 bash[28843]: cluster 2024-03-04T16:00:23.785030+0000 mon.a (mon.0) 2643 : cluster 0 osdmap e413: 8 total, 8 up, 8 in 2024-03-04T16:00:26.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:26 smithi151 bash[23756]: cluster 2024-03-04T16:00:24.982809+0000 mgr.y (mgr.24428) 423 : cluster 0 pgmap v637: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:00:26.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:26 smithi117 bash[21480]: cluster 2024-03-04T16:00:24.982809+0000 mgr.y (mgr.24428) 423 : cluster 0 pgmap v637: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:00:26.567 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:00:26 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:00:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:00:26.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:26 smithi117 bash[28843]: cluster 2024-03-04T16:00:24.982809+0000 mgr.y (mgr.24428) 423 : cluster 0 pgmap v637: 228 pgs: 228 active+clean; 8.4 MiB data, 621 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:00:27.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:00:26 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:00:28.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:28 smithi151 bash[23756]: audit 2024-03-04T16:00:26.813988+0000 mgr.y (mgr.24428) 424 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:00:28.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:28 smithi151 bash[23756]: cluster 2024-03-04T16:00:26.984976+0000 mgr.y (mgr.24428) 425 : cluster 0 pgmap v638: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:00:28.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:28 smithi117 bash[21480]: audit 2024-03-04T16:00:26.813988+0000 mgr.y (mgr.24428) 424 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:00:28.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:28 smithi117 bash[21480]: cluster 2024-03-04T16:00:26.984976+0000 mgr.y (mgr.24428) 425 : cluster 0 pgmap v638: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:00:28.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:28 smithi117 bash[28843]: audit 2024-03-04T16:00:26.813988+0000 mgr.y (mgr.24428) 424 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:00:28.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:28 smithi117 bash[28843]: cluster 2024-03-04T16:00:26.984976+0000 mgr.y (mgr.24428) 425 : cluster 0 pgmap v638: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:00:30.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:30 smithi151 bash[23756]: cluster 2024-03-04T16:00:28.985795+0000 mgr.y (mgr.24428) 426 : cluster 0 pgmap v639: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:00:30.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:30 smithi151 bash[23756]: audit 2024-03-04T16:00:29.117208+0000 mon.c (mon.2) 618 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:00:30.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:30 smithi117 bash[21480]: cluster 2024-03-04T16:00:28.985795+0000 mgr.y (mgr.24428) 426 : cluster 0 pgmap v639: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:00:30.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:30 smithi117 bash[21480]: audit 2024-03-04T16:00:29.117208+0000 mon.c (mon.2) 618 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:00:30.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:30 smithi117 bash[28843]: cluster 2024-03-04T16:00:28.985795+0000 mgr.y (mgr.24428) 426 : cluster 0 pgmap v639: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:00:30.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:30 smithi117 bash[28843]: audit 2024-03-04T16:00:29.117208+0000 mon.c (mon.2) 618 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:00:32.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:32 smithi151 bash[23756]: cluster 2024-03-04T16:00:30.986614+0000 mgr.y (mgr.24428) 427 : cluster 0 pgmap v640: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:00:32.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:32 smithi117 bash[21480]: cluster 2024-03-04T16:00:30.986614+0000 mgr.y (mgr.24428) 427 : cluster 0 pgmap v640: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:00:32.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:32 smithi117 bash[28843]: cluster 2024-03-04T16:00:30.986614+0000 mgr.y (mgr.24428) 427 : cluster 0 pgmap v640: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:00:33.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:33 smithi151 bash[23756]: cluster 2024-03-04T16:00:32.106481+0000 mon.a (mon.0) 2644 : cluster 0 osdmap e414: 8 total, 8 up, 8 in 2024-03-04T16:00:33.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:33 smithi117 bash[21480]: cluster 2024-03-04T16:00:32.106481+0000 mon.a (mon.0) 2644 : cluster 0 osdmap e414: 8 total, 8 up, 8 in 2024-03-04T16:00:33.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:33 smithi117 bash[28843]: cluster 2024-03-04T16:00:32.106481+0000 mon.a (mon.0) 2644 : cluster 0 osdmap e414: 8 total, 8 up, 8 in 2024-03-04T16:00:34.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:34 smithi151 bash[23756]: cluster 2024-03-04T16:00:32.987429+0000 mgr.y (mgr.24428) 428 : cluster 0 pgmap v642: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:00:34.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:34 smithi117 bash[21480]: cluster 2024-03-04T16:00:32.987429+0000 mgr.y (mgr.24428) 428 : cluster 0 pgmap v642: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:00:34.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:34 smithi117 bash[28843]: cluster 2024-03-04T16:00:32.987429+0000 mgr.y (mgr.24428) 428 : cluster 0 pgmap v642: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:00:36.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:36 smithi151 bash[23756]: cluster 2024-03-04T16:00:34.988292+0000 mgr.y (mgr.24428) 429 : cluster 0 pgmap v643: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-04T16:00:36.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:36 smithi117 bash[21480]: cluster 2024-03-04T16:00:34.988292+0000 mgr.y (mgr.24428) 429 : cluster 0 pgmap v643: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-04T16:00:36.567 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:00:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:00:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:00:36.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:36 smithi117 bash[28843]: cluster 2024-03-04T16:00:34.988292+0000 mgr.y (mgr.24428) 429 : cluster 0 pgmap v643: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-04T16:00:37.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:00:36 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:00:38.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:38 smithi151 bash[23756]: audit 2024-03-04T16:00:36.816231+0000 mgr.y (mgr.24428) 430 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:00:38.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:38 smithi151 bash[23756]: cluster 2024-03-04T16:00:36.990550+0000 mgr.y (mgr.24428) 431 : cluster 0 pgmap v644: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-04T16:00:38.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:38 smithi117 bash[21480]: audit 2024-03-04T16:00:36.816231+0000 mgr.y (mgr.24428) 430 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:00:38.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:38 smithi117 bash[21480]: cluster 2024-03-04T16:00:36.990550+0000 mgr.y (mgr.24428) 431 : cluster 0 pgmap v644: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-04T16:00:38.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:38 smithi117 bash[28843]: audit 2024-03-04T16:00:36.816231+0000 mgr.y (mgr.24428) 430 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:00:38.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:38 smithi117 bash[28843]: cluster 2024-03-04T16:00:36.990550+0000 mgr.y (mgr.24428) 431 : cluster 0 pgmap v644: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-04T16:00:40.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:39 smithi117 bash[28843]: cluster 2024-03-04T16:00:38.787378+0000 mon.a (mon.0) 2645 : cluster 0 osdmap e415: 8 total, 8 up, 8 in 2024-03-04T16:00:40.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:39 smithi117 bash[28843]: cluster 2024-03-04T16:00:38.991388+0000 mgr.y (mgr.24428) 432 : cluster 0 pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-04T16:00:40.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:39 smithi117 bash[21480]: cluster 2024-03-04T16:00:38.787378+0000 mon.a (mon.0) 2645 : cluster 0 osdmap e415: 8 total, 8 up, 8 in 2024-03-04T16:00:40.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:39 smithi117 bash[21480]: cluster 2024-03-04T16:00:38.991388+0000 mgr.y (mgr.24428) 432 : cluster 0 pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-04T16:00:40.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:39 smithi151 bash[23756]: cluster 2024-03-04T16:00:38.787378+0000 mon.a (mon.0) 2645 : cluster 0 osdmap e415: 8 total, 8 up, 8 in 2024-03-04T16:00:40.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:39 smithi151 bash[23756]: cluster 2024-03-04T16:00:38.991388+0000 mgr.y (mgr.24428) 432 : cluster 0 pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-03-04T16:00:42.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:42 smithi117 bash[21480]: cluster 2024-03-04T16:00:40.992357+0000 mgr.y (mgr.24428) 433 : cluster 0 pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 575 B/s rd, 0 op/s 2024-03-04T16:00:42.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:42 smithi117 bash[21480]: audit 2024-03-04T16:00:41.213685+0000 mon.c (mon.2) 619 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:00:42.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:42 smithi117 bash[21480]: audit 2024-03-04T16:00:41.740442+0000 mon.a (mon.0) 2646 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:00:42.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:42 smithi117 bash[21480]: audit 2024-03-04T16:00:41.750308+0000 mon.a (mon.0) 2647 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:00:42.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:42 smithi117 bash[28843]: cluster 2024-03-04T16:00:40.992357+0000 mgr.y (mgr.24428) 433 : cluster 0 pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 575 B/s rd, 0 op/s 2024-03-04T16:00:42.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:42 smithi117 bash[28843]: audit 2024-03-04T16:00:41.213685+0000 mon.c (mon.2) 619 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:00:42.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:42 smithi117 bash[28843]: audit 2024-03-04T16:00:41.740442+0000 mon.a (mon.0) 2646 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:00:42.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:42 smithi117 bash[28843]: audit 2024-03-04T16:00:41.750308+0000 mon.a (mon.0) 2647 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:00:42.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:42 smithi151 bash[23756]: cluster 2024-03-04T16:00:40.992357+0000 mgr.y (mgr.24428) 433 : cluster 0 pgmap v647: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 575 B/s rd, 0 op/s 2024-03-04T16:00:42.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:42 smithi151 bash[23756]: audit 2024-03-04T16:00:41.213685+0000 mon.c (mon.2) 619 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:00:42.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:42 smithi151 bash[23756]: audit 2024-03-04T16:00:41.740442+0000 mon.a (mon.0) 2646 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:00:42.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:42 smithi151 bash[23756]: audit 2024-03-04T16:00:41.750308+0000 mon.a (mon.0) 2647 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:00:43.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:43 smithi117 bash[21480]: audit 2024-03-04T16:00:42.142818+0000 mon.c (mon.2) 620 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:00:43.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:43 smithi117 bash[21480]: audit 2024-03-04T16:00:42.143480+0000 mon.a (mon.0) 2648 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:00:43.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:43 smithi117 bash[21480]: audit 2024-03-04T16:00:42.144533+0000 mon.c (mon.2) 621 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-63"}]: dispatch 2024-03-04T16:00:43.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:43 smithi117 bash[21480]: audit 2024-03-04T16:00:42.145139+0000 mon.a (mon.0) 2649 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-63"}]: dispatch 2024-03-04T16:00:43.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:43 smithi117 bash[21480]: audit 2024-03-04T16:00:42.383151+0000 mon.c (mon.2) 622 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:00:43.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:43 smithi117 bash[21480]: audit 2024-03-04T16:00:42.384116+0000 mon.c (mon.2) 623 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:00:43.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:43 smithi117 bash[21480]: audit 2024-03-04T16:00:42.392070+0000 mon.a (mon.0) 2650 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:00:43.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:43 smithi117 bash[28843]: audit 2024-03-04T16:00:42.142818+0000 mon.c (mon.2) 620 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:00:43.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:43 smithi117 bash[28843]: audit 2024-03-04T16:00:42.143480+0000 mon.a (mon.0) 2648 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:00:43.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:43 smithi117 bash[28843]: audit 2024-03-04T16:00:42.144533+0000 mon.c (mon.2) 621 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-63"}]: dispatch 2024-03-04T16:00:43.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:43 smithi117 bash[28843]: audit 2024-03-04T16:00:42.145139+0000 mon.a (mon.0) 2649 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-63"}]: dispatch 2024-03-04T16:00:43.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:43 smithi117 bash[28843]: audit 2024-03-04T16:00:42.383151+0000 mon.c (mon.2) 622 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:00:43.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:43 smithi117 bash[28843]: audit 2024-03-04T16:00:42.384116+0000 mon.c (mon.2) 623 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:00:43.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:43 smithi117 bash[28843]: audit 2024-03-04T16:00:42.392070+0000 mon.a (mon.0) 2650 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:00:43.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:43 smithi151 bash[23756]: audit 2024-03-04T16:00:42.142818+0000 mon.c (mon.2) 620 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:00:43.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:43 smithi151 bash[23756]: audit 2024-03-04T16:00:42.143480+0000 mon.a (mon.0) 2648 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:00:43.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:43 smithi151 bash[23756]: audit 2024-03-04T16:00:42.144533+0000 mon.c (mon.2) 621 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-63"}]: dispatch 2024-03-04T16:00:43.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:43 smithi151 bash[23756]: audit 2024-03-04T16:00:42.145139+0000 mon.a (mon.0) 2649 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-63"}]: dispatch 2024-03-04T16:00:43.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:43 smithi151 bash[23756]: audit 2024-03-04T16:00:42.383151+0000 mon.c (mon.2) 622 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:00:43.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:43 smithi151 bash[23756]: audit 2024-03-04T16:00:42.384116+0000 mon.c (mon.2) 623 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:00:43.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:43 smithi151 bash[23756]: audit 2024-03-04T16:00:42.392070+0000 mon.a (mon.0) 2650 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:00:44.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:44 smithi151 bash[23756]: cluster 2024-03-04T16:00:42.994505+0000 mgr.y (mgr.24428) 434 : cluster 0 pgmap v648: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:00:44.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:44 smithi151 bash[23756]: cluster 2024-03-04T16:00:43.067680+0000 mon.a (mon.0) 2651 : cluster 0 osdmap e416: 8 total, 8 up, 8 in 2024-03-04T16:00:44.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:44 smithi117 bash[21480]: cluster 2024-03-04T16:00:42.994505+0000 mgr.y (mgr.24428) 434 : cluster 0 pgmap v648: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:00:44.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:44 smithi117 bash[21480]: cluster 2024-03-04T16:00:43.067680+0000 mon.a (mon.0) 2651 : cluster 0 osdmap e416: 8 total, 8 up, 8 in 2024-03-04T16:00:44.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:44 smithi117 bash[28843]: cluster 2024-03-04T16:00:42.994505+0000 mgr.y (mgr.24428) 434 : cluster 0 pgmap v648: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:00:44.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:44 smithi117 bash[28843]: cluster 2024-03-04T16:00:43.067680+0000 mon.a (mon.0) 2651 : cluster 0 osdmap e416: 8 total, 8 up, 8 in 2024-03-04T16:00:45.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:45 smithi151 bash[23756]: cluster 2024-03-04T16:00:44.073499+0000 mon.a (mon.0) 2652 : cluster 0 osdmap e417: 8 total, 8 up, 8 in 2024-03-04T16:00:45.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:45 smithi151 bash[23756]: audit 2024-03-04T16:00:44.076762+0000 mon.c (mon.2) 624 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:00:45.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:45 smithi151 bash[23756]: audit 2024-03-04T16:00:44.089137+0000 mon.a (mon.0) 2653 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:00:45.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:45 smithi151 bash[23756]: audit 2024-03-04T16:00:44.151252+0000 mon.c (mon.2) 625 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:00:45.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:45 smithi151 bash[23756]: audit 2024-03-04T16:00:45.069170+0000 mon.a (mon.0) 2654 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:00:45.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:45 smithi151 bash[23756]: cluster 2024-03-04T16:00:45.079071+0000 mon.a (mon.0) 2655 : cluster 0 osdmap e418: 8 total, 8 up, 8 in 2024-03-04T16:00:45.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:45 smithi151 bash[23756]: audit 2024-03-04T16:00:45.084402+0000 mon.c (mon.2) 626 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:00:45.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:45 smithi151 bash[23756]: audit 2024-03-04T16:00:45.097489+0000 mon.c (mon.2) 627 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:00:45.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:45 smithi151 bash[23756]: audit 2024-03-04T16:00:45.099016+0000 mon.a (mon.0) 2656 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:00:45.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:45 smithi117 bash[21480]: cluster 2024-03-04T16:00:44.073499+0000 mon.a (mon.0) 2652 : cluster 0 osdmap e417: 8 total, 8 up, 8 in 2024-03-04T16:00:45.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:45 smithi117 bash[21480]: audit 2024-03-04T16:00:44.076762+0000 mon.c (mon.2) 624 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:00:45.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:45 smithi117 bash[21480]: audit 2024-03-04T16:00:44.089137+0000 mon.a (mon.0) 2653 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:00:45.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:45 smithi117 bash[21480]: audit 2024-03-04T16:00:44.151252+0000 mon.c (mon.2) 625 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:00:45.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:45 smithi117 bash[21480]: audit 2024-03-04T16:00:45.069170+0000 mon.a (mon.0) 2654 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:00:45.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:45 smithi117 bash[21480]: cluster 2024-03-04T16:00:45.079071+0000 mon.a (mon.0) 2655 : cluster 0 osdmap e418: 8 total, 8 up, 8 in 2024-03-04T16:00:45.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:45 smithi117 bash[21480]: audit 2024-03-04T16:00:45.084402+0000 mon.c (mon.2) 626 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:00:45.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:45 smithi117 bash[21480]: audit 2024-03-04T16:00:45.097489+0000 mon.c (mon.2) 627 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:00:45.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:45 smithi117 bash[21480]: audit 2024-03-04T16:00:45.099016+0000 mon.a (mon.0) 2656 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:00:45.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:45 smithi117 bash[28843]: cluster 2024-03-04T16:00:44.073499+0000 mon.a (mon.0) 2652 : cluster 0 osdmap e417: 8 total, 8 up, 8 in 2024-03-04T16:00:45.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:45 smithi117 bash[28843]: audit 2024-03-04T16:00:44.076762+0000 mon.c (mon.2) 624 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:00:45.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:45 smithi117 bash[28843]: audit 2024-03-04T16:00:44.089137+0000 mon.a (mon.0) 2653 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:00:45.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:45 smithi117 bash[28843]: audit 2024-03-04T16:00:44.151252+0000 mon.c (mon.2) 625 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:00:45.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:45 smithi117 bash[28843]: audit 2024-03-04T16:00:45.069170+0000 mon.a (mon.0) 2654 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:00:45.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:45 smithi117 bash[28843]: cluster 2024-03-04T16:00:45.079071+0000 mon.a (mon.0) 2655 : cluster 0 osdmap e418: 8 total, 8 up, 8 in 2024-03-04T16:00:45.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:45 smithi117 bash[28843]: audit 2024-03-04T16:00:45.084402+0000 mon.c (mon.2) 626 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:00:45.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:45 smithi117 bash[28843]: audit 2024-03-04T16:00:45.097489+0000 mon.c (mon.2) 627 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:00:45.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:45 smithi117 bash[28843]: audit 2024-03-04T16:00:45.099016+0000 mon.a (mon.0) 2656 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:00:46.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:46 smithi151 bash[23756]: cluster 2024-03-04T16:00:44.995367+0000 mgr.y (mgr.24428) 435 : cluster 0 pgmap v651: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:00:46.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:46 smithi151 bash[23756]: cluster 2024-03-04T16:00:45.102350+0000 mon.a (mon.0) 2657 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:00:46.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:46 smithi151 bash[23756]: audit 2024-03-04T16:00:46.071945+0000 mon.a (mon.0) 2658 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:00:46.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:46 smithi151 bash[23756]: cluster 2024-03-04T16:00:46.081109+0000 mon.a (mon.0) 2659 : cluster 0 osdmap e419: 8 total, 8 up, 8 in 2024-03-04T16:00:46.567 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:00:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:00:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:00:46.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:46 smithi117 bash[21480]: cluster 2024-03-04T16:00:44.995367+0000 mgr.y (mgr.24428) 435 : cluster 0 pgmap v651: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:00:46.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:46 smithi117 bash[21480]: cluster 2024-03-04T16:00:45.102350+0000 mon.a (mon.0) 2657 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:00:46.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:46 smithi117 bash[21480]: audit 2024-03-04T16:00:46.071945+0000 mon.a (mon.0) 2658 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:00:46.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:46 smithi117 bash[21480]: cluster 2024-03-04T16:00:46.081109+0000 mon.a (mon.0) 2659 : cluster 0 osdmap e419: 8 total, 8 up, 8 in 2024-03-04T16:00:46.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:46 smithi117 bash[28843]: cluster 2024-03-04T16:00:44.995367+0000 mgr.y (mgr.24428) 435 : cluster 0 pgmap v651: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:00:46.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:46 smithi117 bash[28843]: cluster 2024-03-04T16:00:45.102350+0000 mon.a (mon.0) 2657 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:00:46.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:46 smithi117 bash[28843]: audit 2024-03-04T16:00:46.071945+0000 mon.a (mon.0) 2658 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:00:46.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:46 smithi117 bash[28843]: cluster 2024-03-04T16:00:46.081109+0000 mon.a (mon.0) 2659 : cluster 0 osdmap e419: 8 total, 8 up, 8 in 2024-03-04T16:00:47.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:00:46 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:00:48.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:48 smithi151 bash[23756]: audit 2024-03-04T16:00:46.825196+0000 mgr.y (mgr.24428) 436 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:00:48.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:48 smithi151 bash[23756]: cluster 2024-03-04T16:00:46.997588+0000 mgr.y (mgr.24428) 437 : cluster 0 pgmap v654: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:00:48.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:48 smithi151 bash[23756]: cluster 2024-03-04T16:00:47.149073+0000 mon.a (mon.0) 2660 : cluster 0 osdmap e420: 8 total, 8 up, 8 in 2024-03-04T16:00:48.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:48 smithi117 bash[21480]: audit 2024-03-04T16:00:46.825196+0000 mgr.y (mgr.24428) 436 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:00:48.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:48 smithi117 bash[21480]: cluster 2024-03-04T16:00:46.997588+0000 mgr.y (mgr.24428) 437 : cluster 0 pgmap v654: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:00:48.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:48 smithi117 bash[21480]: cluster 2024-03-04T16:00:47.149073+0000 mon.a (mon.0) 2660 : cluster 0 osdmap e420: 8 total, 8 up, 8 in 2024-03-04T16:00:48.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:48 smithi117 bash[28843]: audit 2024-03-04T16:00:46.825196+0000 mgr.y (mgr.24428) 436 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:00:48.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:48 smithi117 bash[28843]: cluster 2024-03-04T16:00:46.997588+0000 mgr.y (mgr.24428) 437 : cluster 0 pgmap v654: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:00:48.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:48 smithi117 bash[28843]: cluster 2024-03-04T16:00:47.149073+0000 mon.a (mon.0) 2660 : cluster 0 osdmap e420: 8 total, 8 up, 8 in 2024-03-04T16:00:49.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:49 smithi151 bash[23756]: cluster 2024-03-04T16:00:48.150943+0000 mon.a (mon.0) 2661 : cluster 0 osdmap e421: 8 total, 8 up, 8 in 2024-03-04T16:00:49.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:49 smithi117 bash[21480]: cluster 2024-03-04T16:00:48.150943+0000 mon.a (mon.0) 2661 : cluster 0 osdmap e421: 8 total, 8 up, 8 in 2024-03-04T16:00:49.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:49 smithi117 bash[28843]: cluster 2024-03-04T16:00:48.150943+0000 mon.a (mon.0) 2661 : cluster 0 osdmap e421: 8 total, 8 up, 8 in 2024-03-04T16:00:50.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:50 smithi151 bash[23756]: cluster 2024-03-04T16:00:48.998411+0000 mgr.y (mgr.24428) 438 : cluster 0 pgmap v657: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-04T16:00:50.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:50 smithi151 bash[23756]: cluster 2024-03-04T16:00:49.174236+0000 mon.a (mon.0) 2662 : cluster 0 osdmap e422: 8 total, 8 up, 8 in 2024-03-04T16:00:50.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:50 smithi117 bash[21480]: cluster 2024-03-04T16:00:48.998411+0000 mgr.y (mgr.24428) 438 : cluster 0 pgmap v657: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-04T16:00:50.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:50 smithi117 bash[21480]: cluster 2024-03-04T16:00:49.174236+0000 mon.a (mon.0) 2662 : cluster 0 osdmap e422: 8 total, 8 up, 8 in 2024-03-04T16:00:50.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:50 smithi117 bash[28843]: cluster 2024-03-04T16:00:48.998411+0000 mgr.y (mgr.24428) 438 : cluster 0 pgmap v657: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-04T16:00:50.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:50 smithi117 bash[28843]: cluster 2024-03-04T16:00:49.174236+0000 mon.a (mon.0) 2662 : cluster 0 osdmap e422: 8 total, 8 up, 8 in 2024-03-04T16:00:52.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:52 smithi151 bash[23756]: cluster 2024-03-04T16:00:50.999234+0000 mgr.y (mgr.24428) 439 : cluster 0 pgmap v659: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-04T16:00:52.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:52 smithi117 bash[21480]: cluster 2024-03-04T16:00:50.999234+0000 mgr.y (mgr.24428) 439 : cluster 0 pgmap v659: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-04T16:00:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:52 smithi117 bash[28843]: cluster 2024-03-04T16:00:50.999234+0000 mgr.y (mgr.24428) 439 : cluster 0 pgmap v659: 228 pgs: 228 active+clean; 8.4 MiB data, 622 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-03-04T16:00:54.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:54 smithi151 bash[23756]: cluster 2024-03-04T16:00:53.001497+0000 mgr.y (mgr.24428) 440 : cluster 0 pgmap v660: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-03-04T16:00:54.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:54 smithi151 bash[23756]: cluster 2024-03-04T16:00:53.782175+0000 mon.a (mon.0) 2663 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:00:54.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:54 smithi151 bash[23756]: cluster 2024-03-04T16:00:53.793823+0000 mon.a (mon.0) 2664 : cluster 0 osdmap e423: 8 total, 8 up, 8 in 2024-03-04T16:00:54.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:54 smithi117 bash[28843]: cluster 2024-03-04T16:00:53.001497+0000 mgr.y (mgr.24428) 440 : cluster 0 pgmap v660: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-03-04T16:00:54.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:54 smithi117 bash[28843]: cluster 2024-03-04T16:00:53.782175+0000 mon.a (mon.0) 2663 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:00:54.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:54 smithi117 bash[28843]: cluster 2024-03-04T16:00:53.793823+0000 mon.a (mon.0) 2664 : cluster 0 osdmap e423: 8 total, 8 up, 8 in 2024-03-04T16:00:54.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:54 smithi117 bash[21480]: cluster 2024-03-04T16:00:53.001497+0000 mgr.y (mgr.24428) 440 : cluster 0 pgmap v660: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 682 B/s wr, 4 op/s 2024-03-04T16:00:54.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:54 smithi117 bash[21480]: cluster 2024-03-04T16:00:53.782175+0000 mon.a (mon.0) 2663 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:00:54.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:54 smithi117 bash[21480]: cluster 2024-03-04T16:00:53.793823+0000 mon.a (mon.0) 2664 : cluster 0 osdmap e423: 8 total, 8 up, 8 in 2024-03-04T16:00:56.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:56 smithi151 bash[23756]: cluster 2024-03-04T16:00:55.002351+0000 mgr.y (mgr.24428) 441 : cluster 0 pgmap v662: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 597 B/s wr, 4 op/s 2024-03-04T16:00:56.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:56 smithi117 bash[21480]: cluster 2024-03-04T16:00:55.002351+0000 mgr.y (mgr.24428) 441 : cluster 0 pgmap v662: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 597 B/s wr, 4 op/s 2024-03-04T16:00:56.567 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:00:56 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:00:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:00:56.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:56 smithi117 bash[28843]: cluster 2024-03-04T16:00:55.002351+0000 mgr.y (mgr.24428) 441 : cluster 0 pgmap v662: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.2 KiB/s rd, 597 B/s wr, 4 op/s 2024-03-04T16:00:57.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:00:56 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:00:58.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:58 smithi117 bash[28843]: audit 2024-03-04T16:00:56.836566+0000 mgr.y (mgr.24428) 442 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:00:58.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:58 smithi117 bash[28843]: cluster 2024-03-04T16:00:57.004503+0000 mgr.y (mgr.24428) 443 : cluster 0 pgmap v663: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-04T16:00:58.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:58 smithi117 bash[21480]: audit 2024-03-04T16:00:56.836566+0000 mgr.y (mgr.24428) 442 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:00:58.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:58 smithi117 bash[21480]: cluster 2024-03-04T16:00:57.004503+0000 mgr.y (mgr.24428) 443 : cluster 0 pgmap v663: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-04T16:00:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:58 smithi151 bash[23756]: audit 2024-03-04T16:00:56.836566+0000 mgr.y (mgr.24428) 442 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:00:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:58 smithi151 bash[23756]: cluster 2024-03-04T16:00:57.004503+0000 mgr.y (mgr.24428) 443 : cluster 0 pgmap v663: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-03-04T16:00:59.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:00:59 smithi117 bash[21480]: audit 2024-03-04T16:00:59.184903+0000 mon.c (mon.2) 628 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:00:59.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:00:59 smithi117 bash[28843]: audit 2024-03-04T16:00:59.184903+0000 mon.c (mon.2) 628 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:00:59.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:00:59 smithi151 bash[23756]: audit 2024-03-04T16:00:59.184903+0000 mon.c (mon.2) 628 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:01:00.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:00 smithi117 bash[21480]: cluster 2024-03-04T16:00:59.005331+0000 mgr.y (mgr.24428) 444 : cluster 0 pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 416 B/s wr, 3 op/s 2024-03-04T16:01:00.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:00 smithi117 bash[21480]: audit 2024-03-04T16:00:59.218918+0000 mon.c (mon.2) 629 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:00.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:00 smithi117 bash[21480]: audit 2024-03-04T16:00:59.219881+0000 mon.a (mon.0) 2665 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:00.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:00 smithi117 bash[21480]: audit 2024-03-04T16:00:59.221347+0000 mon.c (mon.2) 630 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-65"}]: dispatch 2024-03-04T16:01:00.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:00 smithi117 bash[21480]: audit 2024-03-04T16:00:59.222236+0000 mon.a (mon.0) 2666 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-65"}]: dispatch 2024-03-04T16:01:00.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:00 smithi117 bash[28843]: cluster 2024-03-04T16:00:59.005331+0000 mgr.y (mgr.24428) 444 : cluster 0 pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 416 B/s wr, 3 op/s 2024-03-04T16:01:00.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:00 smithi117 bash[28843]: audit 2024-03-04T16:00:59.218918+0000 mon.c (mon.2) 629 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:00.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:00 smithi117 bash[28843]: audit 2024-03-04T16:00:59.219881+0000 mon.a (mon.0) 2665 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:00.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:00 smithi117 bash[28843]: audit 2024-03-04T16:00:59.221347+0000 mon.c (mon.2) 630 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-65"}]: dispatch 2024-03-04T16:01:00.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:00 smithi117 bash[28843]: audit 2024-03-04T16:00:59.222236+0000 mon.a (mon.0) 2666 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-65"}]: dispatch 2024-03-04T16:01:00.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:00 smithi151 bash[23756]: cluster 2024-03-04T16:00:59.005331+0000 mgr.y (mgr.24428) 444 : cluster 0 pgmap v664: 228 pgs: 228 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 416 B/s wr, 3 op/s 2024-03-04T16:01:00.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:00 smithi151 bash[23756]: audit 2024-03-04T16:00:59.218918+0000 mon.c (mon.2) 629 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:00 smithi151 bash[23756]: audit 2024-03-04T16:00:59.219881+0000 mon.a (mon.0) 2665 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:00 smithi151 bash[23756]: audit 2024-03-04T16:00:59.221347+0000 mon.c (mon.2) 630 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-65"}]: dispatch 2024-03-04T16:01:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:00 smithi151 bash[23756]: audit 2024-03-04T16:00:59.222236+0000 mon.a (mon.0) 2666 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-65"}]: dispatch 2024-03-04T16:01:01.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:01 smithi117 bash[21480]: cluster 2024-03-04T16:01:00.221649+0000 mon.a (mon.0) 2667 : cluster 0 osdmap e424: 8 total, 8 up, 8 in 2024-03-04T16:01:01.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:01 smithi117 bash[28843]: cluster 2024-03-04T16:01:00.221649+0000 mon.a (mon.0) 2667 : cluster 0 osdmap e424: 8 total, 8 up, 8 in 2024-03-04T16:01:01.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:01 smithi151 bash[23756]: cluster 2024-03-04T16:01:00.221649+0000 mon.a (mon.0) 2667 : cluster 0 osdmap e424: 8 total, 8 up, 8 in 2024-03-04T16:01:02.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:02 smithi117 bash[21480]: cluster 2024-03-04T16:01:01.006054+0000 mgr.y (mgr.24428) 445 : cluster 0 pgmap v666: 196 pgs: 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:01:02.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:02 smithi117 bash[21480]: cluster 2024-03-04T16:01:01.236332+0000 mon.a (mon.0) 2668 : cluster 0 osdmap e425: 8 total, 8 up, 8 in 2024-03-04T16:01:02.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:02 smithi117 bash[21480]: audit 2024-03-04T16:01:01.239597+0000 mon.c (mon.2) 631 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:02.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:02 smithi117 bash[21480]: audit 2024-03-04T16:01:01.241842+0000 mon.a (mon.0) 2669 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:02.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:02 smithi117 bash[28843]: cluster 2024-03-04T16:01:01.006054+0000 mgr.y (mgr.24428) 445 : cluster 0 pgmap v666: 196 pgs: 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:01:02.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:02 smithi117 bash[28843]: cluster 2024-03-04T16:01:01.236332+0000 mon.a (mon.0) 2668 : cluster 0 osdmap e425: 8 total, 8 up, 8 in 2024-03-04T16:01:02.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:02 smithi117 bash[28843]: audit 2024-03-04T16:01:01.239597+0000 mon.c (mon.2) 631 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:02.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:02 smithi117 bash[28843]: audit 2024-03-04T16:01:01.241842+0000 mon.a (mon.0) 2669 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:02.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:02 smithi151 bash[23756]: cluster 2024-03-04T16:01:01.006054+0000 mgr.y (mgr.24428) 445 : cluster 0 pgmap v666: 196 pgs: 196 active+clean; 8.4 MiB data, 623 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:01:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:02 smithi151 bash[23756]: cluster 2024-03-04T16:01:01.236332+0000 mon.a (mon.0) 2668 : cluster 0 osdmap e425: 8 total, 8 up, 8 in 2024-03-04T16:01:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:02 smithi151 bash[23756]: audit 2024-03-04T16:01:01.239597+0000 mon.c (mon.2) 631 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:02 smithi151 bash[23756]: audit 2024-03-04T16:01:01.241842+0000 mon.a (mon.0) 2669 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:03.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:03 smithi117 bash[21480]: audit 2024-03-04T16:01:02.240842+0000 mon.a (mon.0) 2670 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:03.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:03 smithi117 bash[21480]: cluster 2024-03-04T16:01:02.251023+0000 mon.a (mon.0) 2671 : cluster 0 osdmap e426: 8 total, 8 up, 8 in 2024-03-04T16:01:03.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:03 smithi117 bash[21480]: audit 2024-03-04T16:01:02.260507+0000 mon.c (mon.2) 632 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:03.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:03 smithi117 bash[28843]: audit 2024-03-04T16:01:02.240842+0000 mon.a (mon.0) 2670 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:03.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:03 smithi117 bash[28843]: cluster 2024-03-04T16:01:02.251023+0000 mon.a (mon.0) 2671 : cluster 0 osdmap e426: 8 total, 8 up, 8 in 2024-03-04T16:01:03.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:03 smithi117 bash[28843]: audit 2024-03-04T16:01:02.260507+0000 mon.c (mon.2) 632 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:03.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:03 smithi151 bash[23756]: audit 2024-03-04T16:01:02.240842+0000 mon.a (mon.0) 2670 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:03 smithi151 bash[23756]: cluster 2024-03-04T16:01:02.251023+0000 mon.a (mon.0) 2671 : cluster 0 osdmap e426: 8 total, 8 up, 8 in 2024-03-04T16:01:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:03 smithi151 bash[23756]: audit 2024-03-04T16:01:02.260507+0000 mon.c (mon.2) 632 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:04.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:04 smithi117 bash[21480]: cluster 2024-03-04T16:01:03.006845+0000 mgr.y (mgr.24428) 446 : cluster 0 pgmap v669: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 607 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:01:04.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:04 smithi117 bash[21480]: cluster 2024-03-04T16:01:03.263356+0000 mon.a (mon.0) 2672 : cluster 0 osdmap e427: 8 total, 8 up, 8 in 2024-03-04T16:01:04.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:04 smithi117 bash[21480]: audit 2024-03-04T16:01:03.340952+0000 mon.c (mon.2) 633 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:04.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:04 smithi117 bash[21480]: audit 2024-03-04T16:01:03.341730+0000 mon.a (mon.0) 2673 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:04.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:04 smithi117 bash[21480]: audit 2024-03-04T16:01:03.343136+0000 mon.c (mon.2) 634 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-67"}]: dispatch 2024-03-04T16:01:04.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:04 smithi117 bash[21480]: audit 2024-03-04T16:01:03.343809+0000 mon.a (mon.0) 2674 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-67"}]: dispatch 2024-03-04T16:01:04.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:04 smithi117 bash[28843]: cluster 2024-03-04T16:01:03.006845+0000 mgr.y (mgr.24428) 446 : cluster 0 pgmap v669: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 607 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:01:04.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:04 smithi117 bash[28843]: cluster 2024-03-04T16:01:03.263356+0000 mon.a (mon.0) 2672 : cluster 0 osdmap e427: 8 total, 8 up, 8 in 2024-03-04T16:01:04.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:04 smithi117 bash[28843]: audit 2024-03-04T16:01:03.340952+0000 mon.c (mon.2) 633 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:04.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:04 smithi117 bash[28843]: audit 2024-03-04T16:01:03.341730+0000 mon.a (mon.0) 2673 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:04.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:04 smithi117 bash[28843]: audit 2024-03-04T16:01:03.343136+0000 mon.c (mon.2) 634 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-67"}]: dispatch 2024-03-04T16:01:04.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:04 smithi117 bash[28843]: audit 2024-03-04T16:01:03.343809+0000 mon.a (mon.0) 2674 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-67"}]: dispatch 2024-03-04T16:01:04.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:04 smithi151 bash[23756]: cluster 2024-03-04T16:01:03.006845+0000 mgr.y (mgr.24428) 446 : cluster 0 pgmap v669: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 607 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:01:04.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:04 smithi151 bash[23756]: cluster 2024-03-04T16:01:03.263356+0000 mon.a (mon.0) 2672 : cluster 0 osdmap e427: 8 total, 8 up, 8 in 2024-03-04T16:01:04.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:04 smithi151 bash[23756]: audit 2024-03-04T16:01:03.340952+0000 mon.c (mon.2) 633 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:04.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:04 smithi151 bash[23756]: audit 2024-03-04T16:01:03.341730+0000 mon.a (mon.0) 2673 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:04.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:04 smithi151 bash[23756]: audit 2024-03-04T16:01:03.343136+0000 mon.c (mon.2) 634 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-67"}]: dispatch 2024-03-04T16:01:04.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:04 smithi151 bash[23756]: audit 2024-03-04T16:01:03.343809+0000 mon.a (mon.0) 2674 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-67"}]: dispatch 2024-03-04T16:01:05.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:05 smithi117 bash[21480]: cluster 2024-03-04T16:01:04.273235+0000 mon.a (mon.0) 2675 : cluster 0 osdmap e428: 8 total, 8 up, 8 in 2024-03-04T16:01:05.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:05 smithi117 bash[28843]: cluster 2024-03-04T16:01:04.273235+0000 mon.a (mon.0) 2675 : cluster 0 osdmap e428: 8 total, 8 up, 8 in 2024-03-04T16:01:05.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:05 smithi151 bash[23756]: cluster 2024-03-04T16:01:04.273235+0000 mon.a (mon.0) 2675 : cluster 0 osdmap e428: 8 total, 8 up, 8 in 2024-03-04T16:01:06.566 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:01:06 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:01:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:01:06.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:06 smithi117 bash[28843]: cluster 2024-03-04T16:01:05.007666+0000 mgr.y (mgr.24428) 447 : cluster 0 pgmap v672: 196 pgs: 196 active+clean; 8.4 MiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:01:06.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:06 smithi117 bash[28843]: cluster 2024-03-04T16:01:05.286518+0000 mon.a (mon.0) 2676 : cluster 0 osdmap e429: 8 total, 8 up, 8 in 2024-03-04T16:01:06.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:06 smithi117 bash[28843]: audit 2024-03-04T16:01:05.298389+0000 mon.c (mon.2) 635 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:06.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:06 smithi117 bash[28843]: audit 2024-03-04T16:01:05.300026+0000 mon.a (mon.0) 2677 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:06.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:06 smithi117 bash[21480]: cluster 2024-03-04T16:01:05.007666+0000 mgr.y (mgr.24428) 447 : cluster 0 pgmap v672: 196 pgs: 196 active+clean; 8.4 MiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:01:06.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:06 smithi117 bash[21480]: cluster 2024-03-04T16:01:05.286518+0000 mon.a (mon.0) 2676 : cluster 0 osdmap e429: 8 total, 8 up, 8 in 2024-03-04T16:01:06.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:06 smithi117 bash[21480]: audit 2024-03-04T16:01:05.298389+0000 mon.c (mon.2) 635 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:06.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:06 smithi117 bash[21480]: audit 2024-03-04T16:01:05.300026+0000 mon.a (mon.0) 2677 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:06.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:06 smithi151 bash[23756]: cluster 2024-03-04T16:01:05.007666+0000 mgr.y (mgr.24428) 447 : cluster 0 pgmap v672: 196 pgs: 196 active+clean; 8.4 MiB data, 607 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:01:06.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:06 smithi151 bash[23756]: cluster 2024-03-04T16:01:05.286518+0000 mon.a (mon.0) 2676 : cluster 0 osdmap e429: 8 total, 8 up, 8 in 2024-03-04T16:01:06.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:06 smithi151 bash[23756]: audit 2024-03-04T16:01:05.298389+0000 mon.c (mon.2) 635 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:06.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:06 smithi151 bash[23756]: audit 2024-03-04T16:01:05.300026+0000 mon.a (mon.0) 2677 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:07.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:01:06 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:01:07.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:07 smithi117 bash[28843]: audit 2024-03-04T16:01:06.281221+0000 mon.a (mon.0) 2678 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:07.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:07 smithi117 bash[28843]: cluster 2024-03-04T16:01:06.291732+0000 mon.a (mon.0) 2679 : cluster 0 osdmap e430: 8 total, 8 up, 8 in 2024-03-04T16:01:07.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:07 smithi117 bash[28843]: audit 2024-03-04T16:01:06.332408+0000 mon.c (mon.2) 636 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:07.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:07 smithi117 bash[28843]: audit 2024-03-04T16:01:06.384051+0000 mon.c (mon.2) 637 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:07.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:07 smithi117 bash[28843]: audit 2024-03-04T16:01:06.384806+0000 mon.a (mon.0) 2680 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:07.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:07 smithi117 bash[28843]: audit 2024-03-04T16:01:06.385784+0000 mon.c (mon.2) 638 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-69"}]: dispatch 2024-03-04T16:01:07.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:07 smithi117 bash[28843]: audit 2024-03-04T16:01:06.386234+0000 mon.a (mon.0) 2681 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-69"}]: dispatch 2024-03-04T16:01:07.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:07 smithi117 bash[21480]: audit 2024-03-04T16:01:06.281221+0000 mon.a (mon.0) 2678 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:07.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:07 smithi117 bash[21480]: cluster 2024-03-04T16:01:06.291732+0000 mon.a (mon.0) 2679 : cluster 0 osdmap e430: 8 total, 8 up, 8 in 2024-03-04T16:01:07.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:07 smithi117 bash[21480]: audit 2024-03-04T16:01:06.332408+0000 mon.c (mon.2) 636 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:07.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:07 smithi117 bash[21480]: audit 2024-03-04T16:01:06.384051+0000 mon.c (mon.2) 637 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:07.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:07 smithi117 bash[21480]: audit 2024-03-04T16:01:06.384806+0000 mon.a (mon.0) 2680 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:07.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:07 smithi117 bash[21480]: audit 2024-03-04T16:01:06.385784+0000 mon.c (mon.2) 638 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-69"}]: dispatch 2024-03-04T16:01:07.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:07 smithi117 bash[21480]: audit 2024-03-04T16:01:06.386234+0000 mon.a (mon.0) 2681 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-69"}]: dispatch 2024-03-04T16:01:07.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:07 smithi151 bash[23756]: audit 2024-03-04T16:01:06.281221+0000 mon.a (mon.0) 2678 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:07.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:07 smithi151 bash[23756]: cluster 2024-03-04T16:01:06.291732+0000 mon.a (mon.0) 2679 : cluster 0 osdmap e430: 8 total, 8 up, 8 in 2024-03-04T16:01:07.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:07 smithi151 bash[23756]: audit 2024-03-04T16:01:06.332408+0000 mon.c (mon.2) 636 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:07.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:07 smithi151 bash[23756]: audit 2024-03-04T16:01:06.384051+0000 mon.c (mon.2) 637 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:07.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:07 smithi151 bash[23756]: audit 2024-03-04T16:01:06.384806+0000 mon.a (mon.0) 2680 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:07.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:07 smithi151 bash[23756]: audit 2024-03-04T16:01:06.385784+0000 mon.c (mon.2) 638 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-69"}]: dispatch 2024-03-04T16:01:07.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:07 smithi151 bash[23756]: audit 2024-03-04T16:01:06.386234+0000 mon.a (mon.0) 2681 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-69"}]: dispatch 2024-03-04T16:01:08.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:08 smithi117 bash[28843]: audit 2024-03-04T16:01:06.848013+0000 mgr.y (mgr.24428) 448 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:01:08.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:08 smithi117 bash[28843]: cluster 2024-03-04T16:01:07.009834+0000 mgr.y (mgr.24428) 449 : cluster 0 pgmap v675: 228 pgs: 228 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-04T16:01:08.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:08 smithi117 bash[28843]: cluster 2024-03-04T16:01:07.321300+0000 mon.a (mon.0) 2682 : cluster 0 osdmap e431: 8 total, 8 up, 8 in 2024-03-04T16:01:08.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:08 smithi117 bash[21480]: audit 2024-03-04T16:01:06.848013+0000 mgr.y (mgr.24428) 448 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:01:08.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:08 smithi117 bash[21480]: cluster 2024-03-04T16:01:07.009834+0000 mgr.y (mgr.24428) 449 : cluster 0 pgmap v675: 228 pgs: 228 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-04T16:01:08.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:08 smithi117 bash[21480]: cluster 2024-03-04T16:01:07.321300+0000 mon.a (mon.0) 2682 : cluster 0 osdmap e431: 8 total, 8 up, 8 in 2024-03-04T16:01:08.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:08 smithi151 bash[23756]: audit 2024-03-04T16:01:06.848013+0000 mgr.y (mgr.24428) 448 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:01:08.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:08 smithi151 bash[23756]: cluster 2024-03-04T16:01:07.009834+0000 mgr.y (mgr.24428) 449 : cluster 0 pgmap v675: 228 pgs: 228 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-03-04T16:01:08.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:08 smithi151 bash[23756]: cluster 2024-03-04T16:01:07.321300+0000 mon.a (mon.0) 2682 : cluster 0 osdmap e431: 8 total, 8 up, 8 in 2024-03-04T16:01:09.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:09 smithi151 bash[23756]: cluster 2024-03-04T16:01:08.334038+0000 mon.a (mon.0) 2683 : cluster 0 osdmap e432: 8 total, 8 up, 8 in 2024-03-04T16:01:09.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:09 smithi151 bash[23756]: audit 2024-03-04T16:01:08.340007+0000 mon.c (mon.2) 639 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:09.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:09 smithi151 bash[23756]: audit 2024-03-04T16:01:08.344335+0000 mon.a (mon.0) 2684 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:09.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:09 smithi151 bash[23756]: cluster 2024-03-04T16:01:09.010643+0000 mgr.y (mgr.24428) 450 : cluster 0 pgmap v678: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T16:01:09.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:09 smithi117 bash[21480]: cluster 2024-03-04T16:01:08.334038+0000 mon.a (mon.0) 2683 : cluster 0 osdmap e432: 8 total, 8 up, 8 in 2024-03-04T16:01:09.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:09 smithi117 bash[21480]: audit 2024-03-04T16:01:08.340007+0000 mon.c (mon.2) 639 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:09.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:09 smithi117 bash[21480]: audit 2024-03-04T16:01:08.344335+0000 mon.a (mon.0) 2684 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:09.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:09 smithi117 bash[21480]: cluster 2024-03-04T16:01:09.010643+0000 mgr.y (mgr.24428) 450 : cluster 0 pgmap v678: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T16:01:09.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:09 smithi117 bash[28843]: cluster 2024-03-04T16:01:08.334038+0000 mon.a (mon.0) 2683 : cluster 0 osdmap e432: 8 total, 8 up, 8 in 2024-03-04T16:01:09.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:09 smithi117 bash[28843]: audit 2024-03-04T16:01:08.340007+0000 mon.c (mon.2) 639 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:09.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:09 smithi117 bash[28843]: audit 2024-03-04T16:01:08.344335+0000 mon.a (mon.0) 2684 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:09.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:09 smithi117 bash[28843]: cluster 2024-03-04T16:01:09.010643+0000 mgr.y (mgr.24428) 450 : cluster 0 pgmap v678: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T16:01:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:10 smithi151 bash[23756]: cluster 2024-03-04T16:01:09.327491+0000 mon.a (mon.0) 2685 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:01:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:10 smithi151 bash[23756]: audit 2024-03-04T16:01:09.329195+0000 mon.a (mon.0) 2686 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:10 smithi151 bash[23756]: cluster 2024-03-04T16:01:09.337442+0000 mon.a (mon.0) 2687 : cluster 0 osdmap e433: 8 total, 8 up, 8 in 2024-03-04T16:01:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:10 smithi151 bash[23756]: audit 2024-03-04T16:01:09.381079+0000 mon.c (mon.2) 640 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:10 smithi151 bash[23756]: audit 2024-03-04T16:01:09.388747+0000 mon.c (mon.2) 641 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:01:10.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:10 smithi151 bash[23756]: audit 2024-03-04T16:01:09.389600+0000 mon.a (mon.0) 2688 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:01:10.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:10 smithi117 bash[21480]: cluster 2024-03-04T16:01:09.327491+0000 mon.a (mon.0) 2685 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:01:10.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:10 smithi117 bash[21480]: audit 2024-03-04T16:01:09.329195+0000 mon.a (mon.0) 2686 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:10.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:10 smithi117 bash[21480]: cluster 2024-03-04T16:01:09.337442+0000 mon.a (mon.0) 2687 : cluster 0 osdmap e433: 8 total, 8 up, 8 in 2024-03-04T16:01:10.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:10 smithi117 bash[21480]: audit 2024-03-04T16:01:09.381079+0000 mon.c (mon.2) 640 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:10.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:10 smithi117 bash[21480]: audit 2024-03-04T16:01:09.388747+0000 mon.c (mon.2) 641 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:01:10.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:10 smithi117 bash[21480]: audit 2024-03-04T16:01:09.389600+0000 mon.a (mon.0) 2688 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:01:10.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:10 smithi117 bash[28843]: cluster 2024-03-04T16:01:09.327491+0000 mon.a (mon.0) 2685 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:01:10.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:10 smithi117 bash[28843]: audit 2024-03-04T16:01:09.329195+0000 mon.a (mon.0) 2686 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:10.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:10 smithi117 bash[28843]: cluster 2024-03-04T16:01:09.337442+0000 mon.a (mon.0) 2687 : cluster 0 osdmap e433: 8 total, 8 up, 8 in 2024-03-04T16:01:10.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:10 smithi117 bash[28843]: audit 2024-03-04T16:01:09.381079+0000 mon.c (mon.2) 640 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:10.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:10 smithi117 bash[28843]: audit 2024-03-04T16:01:09.388747+0000 mon.c (mon.2) 641 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:01:10.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:10 smithi117 bash[28843]: audit 2024-03-04T16:01:09.389600+0000 mon.a (mon.0) 2688 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:01:11.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:11 smithi151 bash[23756]: audit 2024-03-04T16:01:10.361210+0000 mon.a (mon.0) 2689 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:01:11.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:11 smithi151 bash[23756]: cluster 2024-03-04T16:01:10.367878+0000 mon.a (mon.0) 2690 : cluster 0 osdmap e434: 8 total, 8 up, 8 in 2024-03-04T16:01:11.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:11 smithi151 bash[23756]: cluster 2024-03-04T16:01:11.011399+0000 mgr.y (mgr.24428) 451 : cluster 0 pgmap v681: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:01:11.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:11 smithi117 bash[21480]: audit 2024-03-04T16:01:10.361210+0000 mon.a (mon.0) 2689 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:01:11.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:11 smithi117 bash[21480]: cluster 2024-03-04T16:01:10.367878+0000 mon.a (mon.0) 2690 : cluster 0 osdmap e434: 8 total, 8 up, 8 in 2024-03-04T16:01:11.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:11 smithi117 bash[21480]: cluster 2024-03-04T16:01:11.011399+0000 mgr.y (mgr.24428) 451 : cluster 0 pgmap v681: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:01:11.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:11 smithi117 bash[28843]: audit 2024-03-04T16:01:10.361210+0000 mon.a (mon.0) 2689 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:01:11.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:11 smithi117 bash[28843]: cluster 2024-03-04T16:01:10.367878+0000 mon.a (mon.0) 2690 : cluster 0 osdmap e434: 8 total, 8 up, 8 in 2024-03-04T16:01:11.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:11 smithi117 bash[28843]: cluster 2024-03-04T16:01:11.011399+0000 mgr.y (mgr.24428) 451 : cluster 0 pgmap v681: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 612 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:01:12.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:12 smithi151 bash[23756]: cluster 2024-03-04T16:01:11.392546+0000 mon.a (mon.0) 2691 : cluster 0 osdmap e435: 8 total, 8 up, 8 in 2024-03-04T16:01:12.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:12 smithi117 bash[28843]: cluster 2024-03-04T16:01:11.392546+0000 mon.a (mon.0) 2691 : cluster 0 osdmap e435: 8 total, 8 up, 8 in 2024-03-04T16:01:12.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:12 smithi117 bash[21480]: cluster 2024-03-04T16:01:11.392546+0000 mon.a (mon.0) 2691 : cluster 0 osdmap e435: 8 total, 8 up, 8 in 2024-03-04T16:01:13.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:13 smithi151 bash[23756]: cluster 2024-03-04T16:01:13.013534+0000 mgr.y (mgr.24428) 452 : cluster 0 pgmap v683: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.1 KiB/s wr, 4 op/s 2024-03-04T16:01:13.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:13 smithi117 bash[21480]: cluster 2024-03-04T16:01:13.013534+0000 mgr.y (mgr.24428) 452 : cluster 0 pgmap v683: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.1 KiB/s wr, 4 op/s 2024-03-04T16:01:13.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:13 smithi117 bash[28843]: cluster 2024-03-04T16:01:13.013534+0000 mgr.y (mgr.24428) 452 : cluster 0 pgmap v683: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.1 KiB/s wr, 4 op/s 2024-03-04T16:01:14.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:14 smithi151 bash[23756]: audit 2024-03-04T16:01:14.224914+0000 mon.c (mon.2) 642 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:01:14.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:14 smithi117 bash[21480]: audit 2024-03-04T16:01:14.224914+0000 mon.c (mon.2) 642 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:01:14.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:14 smithi117 bash[28843]: audit 2024-03-04T16:01:14.224914+0000 mon.c (mon.2) 642 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:01:15.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:15 smithi151 bash[23756]: cluster 2024-03-04T16:01:15.014354+0000 mgr.y (mgr.24428) 453 : cluster 0 pgmap v684: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-04T16:01:15.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:15 smithi117 bash[21480]: cluster 2024-03-04T16:01:15.014354+0000 mgr.y (mgr.24428) 453 : cluster 0 pgmap v684: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-04T16:01:15.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:15 smithi117 bash[28843]: cluster 2024-03-04T16:01:15.014354+0000 mgr.y (mgr.24428) 453 : cluster 0 pgmap v684: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-04T16:01:16.566 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:01:16 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:01:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:01:17.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:01:16 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:01:18.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:18 smithi151 bash[23756]: audit 2024-03-04T16:01:16.852347+0000 mgr.y (mgr.24428) 454 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:01:18.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:18 smithi151 bash[23756]: cluster 2024-03-04T16:01:17.016579+0000 mgr.y (mgr.24428) 455 : cluster 0 pgmap v685: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-04T16:01:18.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:18 smithi117 bash[21480]: audit 2024-03-04T16:01:16.852347+0000 mgr.y (mgr.24428) 454 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:01:18.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:18 smithi117 bash[21480]: cluster 2024-03-04T16:01:17.016579+0000 mgr.y (mgr.24428) 455 : cluster 0 pgmap v685: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-04T16:01:18.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:18 smithi117 bash[28843]: audit 2024-03-04T16:01:16.852347+0000 mgr.y (mgr.24428) 454 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:01:18.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:18 smithi117 bash[28843]: cluster 2024-03-04T16:01:17.016579+0000 mgr.y (mgr.24428) 455 : cluster 0 pgmap v685: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.7 KiB/s wr, 4 op/s 2024-03-04T16:01:19.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:19 smithi151 bash[23756]: cluster 2024-03-04T16:01:18.789486+0000 mon.a (mon.0) 2692 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:01:19.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:19 smithi117 bash[21480]: cluster 2024-03-04T16:01:18.789486+0000 mon.a (mon.0) 2692 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:01:19.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:19 smithi117 bash[28843]: cluster 2024-03-04T16:01:18.789486+0000 mon.a (mon.0) 2692 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:01:20.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:20 smithi151 bash[23756]: cluster 2024-03-04T16:01:19.017402+0000 mgr.y (mgr.24428) 456 : cluster 0 pgmap v686: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-04T16:01:20.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:20 smithi117 bash[28843]: cluster 2024-03-04T16:01:19.017402+0000 mgr.y (mgr.24428) 456 : cluster 0 pgmap v686: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-04T16:01:20.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:20 smithi117 bash[21480]: cluster 2024-03-04T16:01:19.017402+0000 mgr.y (mgr.24428) 456 : cluster 0 pgmap v686: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-04T16:01:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:22 smithi151 bash[23756]: cluster 2024-03-04T16:01:21.018155+0000 mgr.y (mgr.24428) 457 : cluster 0 pgmap v687: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-03-04T16:01:22.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:22 smithi117 bash[21480]: cluster 2024-03-04T16:01:21.018155+0000 mgr.y (mgr.24428) 457 : cluster 0 pgmap v687: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-03-04T16:01:22.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:22 smithi117 bash[28843]: cluster 2024-03-04T16:01:21.018155+0000 mgr.y (mgr.24428) 457 : cluster 0 pgmap v687: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 3 op/s 2024-03-04T16:01:24.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:24 smithi151 bash[23756]: cluster 2024-03-04T16:01:23.020341+0000 mgr.y (mgr.24428) 458 : cluster 0 pgmap v688: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-03-04T16:01:24.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:24 smithi117 bash[21480]: cluster 2024-03-04T16:01:23.020341+0000 mgr.y (mgr.24428) 458 : cluster 0 pgmap v688: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-03-04T16:01:24.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:24 smithi117 bash[28843]: cluster 2024-03-04T16:01:23.020341+0000 mgr.y (mgr.24428) 458 : cluster 0 pgmap v688: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-03-04T16:01:26.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:26 smithi151 bash[23756]: cluster 2024-03-04T16:01:25.021119+0000 mgr.y (mgr.24428) 459 : cluster 0 pgmap v689: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 937 B/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:01:26.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:26 smithi117 bash[21480]: cluster 2024-03-04T16:01:25.021119+0000 mgr.y (mgr.24428) 459 : cluster 0 pgmap v689: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 937 B/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:01:26.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:26 smithi117 bash[28843]: cluster 2024-03-04T16:01:25.021119+0000 mgr.y (mgr.24428) 459 : cluster 0 pgmap v689: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 937 B/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:01:26.567 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:01:26 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:01:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:01:27.186 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:01:26 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:01:28.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:28 smithi151 bash[23756]: audit 2024-03-04T16:01:26.860392+0000 mgr.y (mgr.24428) 460 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:01:28.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:28 smithi151 bash[23756]: cluster 2024-03-04T16:01:27.023388+0000 mgr.y (mgr.24428) 461 : cluster 0 pgmap v690: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-04T16:01:28.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:28 smithi117 bash[21480]: audit 2024-03-04T16:01:26.860392+0000 mgr.y (mgr.24428) 460 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:01:28.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:28 smithi117 bash[21480]: cluster 2024-03-04T16:01:27.023388+0000 mgr.y (mgr.24428) 461 : cluster 0 pgmap v690: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-04T16:01:28.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:28 smithi117 bash[28843]: audit 2024-03-04T16:01:26.860392+0000 mgr.y (mgr.24428) 460 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:01:28.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:28 smithi117 bash[28843]: cluster 2024-03-04T16:01:27.023388+0000 mgr.y (mgr.24428) 461 : cluster 0 pgmap v690: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-03-04T16:01:30.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:30 smithi151 bash[23756]: cluster 2024-03-04T16:01:29.024224+0000 mgr.y (mgr.24428) 462 : cluster 0 pgmap v691: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T16:01:30.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:30 smithi151 bash[23756]: audit 2024-03-04T16:01:29.258386+0000 mon.c (mon.2) 643 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:01:30.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:30 smithi117 bash[21480]: cluster 2024-03-04T16:01:29.024224+0000 mgr.y (mgr.24428) 462 : cluster 0 pgmap v691: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T16:01:30.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:30 smithi117 bash[21480]: audit 2024-03-04T16:01:29.258386+0000 mon.c (mon.2) 643 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:01:30.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:30 smithi117 bash[28843]: cluster 2024-03-04T16:01:29.024224+0000 mgr.y (mgr.24428) 462 : cluster 0 pgmap v691: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T16:01:30.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:30 smithi117 bash[28843]: audit 2024-03-04T16:01:29.258386+0000 mon.c (mon.2) 643 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:01:32.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:32 smithi151 bash[23756]: cluster 2024-03-04T16:01:31.025026+0000 mgr.y (mgr.24428) 463 : cluster 0 pgmap v692: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T16:01:32.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:32 smithi151 bash[23756]: audit 2024-03-04T16:01:31.495682+0000 mon.c (mon.2) 644 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:32.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:32 smithi151 bash[23756]: audit 2024-03-04T16:01:31.496786+0000 mon.a (mon.0) 2693 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:32.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:32 smithi151 bash[23756]: audit 2024-03-04T16:01:31.498112+0000 mon.c (mon.2) 645 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-71"}]: dispatch 2024-03-04T16:01:32.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:32 smithi151 bash[23756]: audit 2024-03-04T16:01:31.498804+0000 mon.a (mon.0) 2694 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-71"}]: dispatch 2024-03-04T16:01:32.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:32 smithi117 bash[21480]: cluster 2024-03-04T16:01:31.025026+0000 mgr.y (mgr.24428) 463 : cluster 0 pgmap v692: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T16:01:32.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:32 smithi117 bash[21480]: audit 2024-03-04T16:01:31.495682+0000 mon.c (mon.2) 644 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:32.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:32 smithi117 bash[21480]: audit 2024-03-04T16:01:31.496786+0000 mon.a (mon.0) 2693 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:32.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:32 smithi117 bash[21480]: audit 2024-03-04T16:01:31.498112+0000 mon.c (mon.2) 645 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-71"}]: dispatch 2024-03-04T16:01:32.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:32 smithi117 bash[21480]: audit 2024-03-04T16:01:31.498804+0000 mon.a (mon.0) 2694 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-71"}]: dispatch 2024-03-04T16:01:32.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:32 smithi117 bash[28843]: cluster 2024-03-04T16:01:31.025026+0000 mgr.y (mgr.24428) 463 : cluster 0 pgmap v692: 228 pgs: 228 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T16:01:32.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:32 smithi117 bash[28843]: audit 2024-03-04T16:01:31.495682+0000 mon.c (mon.2) 644 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:32.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:32 smithi117 bash[28843]: audit 2024-03-04T16:01:31.496786+0000 mon.a (mon.0) 2693 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:32.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:32 smithi117 bash[28843]: audit 2024-03-04T16:01:31.498112+0000 mon.c (mon.2) 645 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-71"}]: dispatch 2024-03-04T16:01:32.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:32 smithi117 bash[28843]: audit 2024-03-04T16:01:31.498804+0000 mon.a (mon.0) 2694 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-71"}]: dispatch 2024-03-04T16:01:33.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:33 smithi151 bash[23756]: cluster 2024-03-04T16:01:32.129829+0000 mon.a (mon.0) 2695 : cluster 0 osdmap e436: 8 total, 8 up, 8 in 2024-03-04T16:01:33.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:33 smithi117 bash[21480]: cluster 2024-03-04T16:01:32.129829+0000 mon.a (mon.0) 2695 : cluster 0 osdmap e436: 8 total, 8 up, 8 in 2024-03-04T16:01:33.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:33 smithi117 bash[28843]: cluster 2024-03-04T16:01:32.129829+0000 mon.a (mon.0) 2695 : cluster 0 osdmap e436: 8 total, 8 up, 8 in 2024-03-04T16:01:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:34 smithi151 bash[23756]: cluster 2024-03-04T16:01:33.025913+0000 mgr.y (mgr.24428) 464 : cluster 0 pgmap v694: 196 pgs: 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-03-04T16:01:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:34 smithi151 bash[23756]: cluster 2024-03-04T16:01:33.140132+0000 mon.a (mon.0) 2696 : cluster 0 osdmap e437: 8 total, 8 up, 8 in 2024-03-04T16:01:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:34 smithi151 bash[23756]: audit 2024-03-04T16:01:33.146232+0000 mon.c (mon.2) 646 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:34 smithi151 bash[23756]: audit 2024-03-04T16:01:33.148098+0000 mon.a (mon.0) 2697 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:34.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:34 smithi117 bash[21480]: cluster 2024-03-04T16:01:33.025913+0000 mgr.y (mgr.24428) 464 : cluster 0 pgmap v694: 196 pgs: 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-03-04T16:01:34.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:34 smithi117 bash[21480]: cluster 2024-03-04T16:01:33.140132+0000 mon.a (mon.0) 2696 : cluster 0 osdmap e437: 8 total, 8 up, 8 in 2024-03-04T16:01:34.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:34 smithi117 bash[21480]: audit 2024-03-04T16:01:33.146232+0000 mon.c (mon.2) 646 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:34.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:34 smithi117 bash[21480]: audit 2024-03-04T16:01:33.148098+0000 mon.a (mon.0) 2697 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:34.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:34 smithi117 bash[28843]: cluster 2024-03-04T16:01:33.025913+0000 mgr.y (mgr.24428) 464 : cluster 0 pgmap v694: 196 pgs: 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-03-04T16:01:34.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:34 smithi117 bash[28843]: cluster 2024-03-04T16:01:33.140132+0000 mon.a (mon.0) 2696 : cluster 0 osdmap e437: 8 total, 8 up, 8 in 2024-03-04T16:01:34.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:34 smithi117 bash[28843]: audit 2024-03-04T16:01:33.146232+0000 mon.c (mon.2) 646 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:34.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:34 smithi117 bash[28843]: audit 2024-03-04T16:01:33.148098+0000 mon.a (mon.0) 2697 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:35.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:35 smithi151 bash[23756]: audit 2024-03-04T16:01:34.134766+0000 mon.a (mon.0) 2698 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:35.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:35 smithi151 bash[23756]: cluster 2024-03-04T16:01:34.144502+0000 mon.a (mon.0) 2699 : cluster 0 osdmap e438: 8 total, 8 up, 8 in 2024-03-04T16:01:35.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:35 smithi151 bash[23756]: audit 2024-03-04T16:01:34.149717+0000 mon.c (mon.2) 647 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:35.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:35 smithi117 bash[21480]: audit 2024-03-04T16:01:34.134766+0000 mon.a (mon.0) 2698 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:35.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:35 smithi117 bash[21480]: cluster 2024-03-04T16:01:34.144502+0000 mon.a (mon.0) 2699 : cluster 0 osdmap e438: 8 total, 8 up, 8 in 2024-03-04T16:01:35.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:35 smithi117 bash[21480]: audit 2024-03-04T16:01:34.149717+0000 mon.c (mon.2) 647 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:35.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:35 smithi117 bash[28843]: audit 2024-03-04T16:01:34.134766+0000 mon.a (mon.0) 2698 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:35.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:35 smithi117 bash[28843]: cluster 2024-03-04T16:01:34.144502+0000 mon.a (mon.0) 2699 : cluster 0 osdmap e438: 8 total, 8 up, 8 in 2024-03-04T16:01:35.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:35 smithi117 bash[28843]: audit 2024-03-04T16:01:34.149717+0000 mon.c (mon.2) 647 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:36.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:36 smithi151 bash[23756]: cluster 2024-03-04T16:01:35.026738+0000 mgr.y (mgr.24428) 465 : cluster 0 pgmap v697: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:01:36.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:36 smithi151 bash[23756]: cluster 2024-03-04T16:01:35.170242+0000 mon.a (mon.0) 2700 : cluster 0 osdmap e439: 8 total, 8 up, 8 in 2024-03-04T16:01:36.566 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:01:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:01:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:01:36.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:36 smithi117 bash[21480]: cluster 2024-03-04T16:01:35.026738+0000 mgr.y (mgr.24428) 465 : cluster 0 pgmap v697: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:01:36.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:36 smithi117 bash[21480]: cluster 2024-03-04T16:01:35.170242+0000 mon.a (mon.0) 2700 : cluster 0 osdmap e439: 8 total, 8 up, 8 in 2024-03-04T16:01:36.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:36 smithi117 bash[28843]: cluster 2024-03-04T16:01:35.026738+0000 mgr.y (mgr.24428) 465 : cluster 0 pgmap v697: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 616 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:01:36.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:36 smithi117 bash[28843]: cluster 2024-03-04T16:01:35.170242+0000 mon.a (mon.0) 2700 : cluster 0 osdmap e439: 8 total, 8 up, 8 in 2024-03-04T16:01:37.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:01:36 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:01:37.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:37 smithi117 bash[21480]: cluster 2024-03-04T16:01:36.171699+0000 mon.a (mon.0) 2701 : cluster 0 osdmap e440: 8 total, 8 up, 8 in 2024-03-04T16:01:37.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:37 smithi117 bash[28843]: cluster 2024-03-04T16:01:36.171699+0000 mon.a (mon.0) 2701 : cluster 0 osdmap e440: 8 total, 8 up, 8 in 2024-03-04T16:01:37.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:37 smithi151 bash[23756]: cluster 2024-03-04T16:01:36.171699+0000 mon.a (mon.0) 2701 : cluster 0 osdmap e440: 8 total, 8 up, 8 in 2024-03-04T16:01:38.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:38 smithi117 bash[21480]: audit 2024-03-04T16:01:36.871820+0000 mgr.y (mgr.24428) 466 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:01:38.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:38 smithi117 bash[21480]: cluster 2024-03-04T16:01:37.028931+0000 mgr.y (mgr.24428) 467 : cluster 0 pgmap v700: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-04T16:01:38.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:38 smithi117 bash[28843]: audit 2024-03-04T16:01:36.871820+0000 mgr.y (mgr.24428) 466 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:01:38.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:38 smithi117 bash[28843]: cluster 2024-03-04T16:01:37.028931+0000 mgr.y (mgr.24428) 467 : cluster 0 pgmap v700: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-04T16:01:38.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:38 smithi151 bash[23756]: audit 2024-03-04T16:01:36.871820+0000 mgr.y (mgr.24428) 466 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:01:38.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:38 smithi151 bash[23756]: cluster 2024-03-04T16:01:37.028931+0000 mgr.y (mgr.24428) 467 : cluster 0 pgmap v700: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-04T16:01:40.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:40 smithi117 bash[28843]: cluster 2024-03-04T16:01:39.029873+0000 mgr.y (mgr.24428) 468 : cluster 0 pgmap v701: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-03-04T16:01:40.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:40 smithi117 bash[21480]: cluster 2024-03-04T16:01:39.029873+0000 mgr.y (mgr.24428) 468 : cluster 0 pgmap v701: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-03-04T16:01:40.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:40 smithi151 bash[23756]: cluster 2024-03-04T16:01:39.029873+0000 mgr.y (mgr.24428) 468 : cluster 0 pgmap v701: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-03-04T16:01:42.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:42 smithi117 bash[21480]: cluster 2024-03-04T16:01:41.030699+0000 mgr.y (mgr.24428) 469 : cluster 0 pgmap v702: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-03-04T16:01:42.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:42 smithi117 bash[28843]: cluster 2024-03-04T16:01:41.030699+0000 mgr.y (mgr.24428) 469 : cluster 0 pgmap v702: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-03-04T16:01:42.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:42 smithi151 bash[23756]: cluster 2024-03-04T16:01:41.030699+0000 mgr.y (mgr.24428) 469 : cluster 0 pgmap v702: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.0 KiB/s wr, 3 op/s 2024-03-04T16:01:43.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[28843]: audit 2024-03-04T16:01:42.466192+0000 mon.c (mon.2) 648 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:01:43.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[28843]: audit 2024-03-04T16:01:42.993100+0000 mon.a (mon.0) 2702 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:01:43.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[28843]: audit 2024-03-04T16:01:43.005258+0000 mon.a (mon.0) 2703 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:01:43.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[28843]: audit 2024-03-04T16:01:43.150858+0000 mon.c (mon.2) 649 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[28843]: audit 2024-03-04T16:01:43.151576+0000 mon.a (mon.0) 2704 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[28843]: audit 2024-03-04T16:01:43.153898+0000 mon.c (mon.2) 650 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[28843]: audit 2024-03-04T16:01:43.154503+0000 mon.a (mon.0) 2705 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[28843]: audit 2024-03-04T16:01:43.156743+0000 mon.c (mon.2) 651 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[28843]: audit 2024-03-04T16:01:43.157351+0000 mon.a (mon.0) 2706 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[28843]: audit 2024-03-04T16:01:43.159267+0000 mon.c (mon.2) 652 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[28843]: audit 2024-03-04T16:01:43.159868+0000 mon.a (mon.0) 2707 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[28843]: audit 2024-03-04T16:01:43.170215+0000 mon.a (mon.0) 2708 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:01:43.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[21480]: audit 2024-03-04T16:01:42.466192+0000 mon.c (mon.2) 648 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:01:43.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[21480]: audit 2024-03-04T16:01:42.993100+0000 mon.a (mon.0) 2702 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:01:43.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[21480]: audit 2024-03-04T16:01:43.005258+0000 mon.a (mon.0) 2703 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:01:43.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[21480]: audit 2024-03-04T16:01:43.150858+0000 mon.c (mon.2) 649 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[21480]: audit 2024-03-04T16:01:43.151576+0000 mon.a (mon.0) 2704 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[21480]: audit 2024-03-04T16:01:43.153898+0000 mon.c (mon.2) 650 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[21480]: audit 2024-03-04T16:01:43.154503+0000 mon.a (mon.0) 2705 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[21480]: audit 2024-03-04T16:01:43.156743+0000 mon.c (mon.2) 651 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[21480]: audit 2024-03-04T16:01:43.157351+0000 mon.a (mon.0) 2706 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[21480]: audit 2024-03-04T16:01:43.159267+0000 mon.c (mon.2) 652 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[21480]: audit 2024-03-04T16:01:43.159868+0000 mon.a (mon.0) 2707 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:43 smithi117 bash[21480]: audit 2024-03-04T16:01:43.170215+0000 mon.a (mon.0) 2708 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:01:43.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:43 smithi151 bash[23756]: audit 2024-03-04T16:01:42.466192+0000 mon.c (mon.2) 648 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:01:43.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:43 smithi151 bash[23756]: audit 2024-03-04T16:01:42.993100+0000 mon.a (mon.0) 2702 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:01:43.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:43 smithi151 bash[23756]: audit 2024-03-04T16:01:43.005258+0000 mon.a (mon.0) 2703 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:01:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:43 smithi151 bash[23756]: audit 2024-03-04T16:01:43.150858+0000 mon.c (mon.2) 649 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:43 smithi151 bash[23756]: audit 2024-03-04T16:01:43.151576+0000 mon.a (mon.0) 2704 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:43 smithi151 bash[23756]: audit 2024-03-04T16:01:43.153898+0000 mon.c (mon.2) 650 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:43 smithi151 bash[23756]: audit 2024-03-04T16:01:43.154503+0000 mon.a (mon.0) 2705 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:43 smithi151 bash[23756]: audit 2024-03-04T16:01:43.156743+0000 mon.c (mon.2) 651 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:43 smithi151 bash[23756]: audit 2024-03-04T16:01:43.157351+0000 mon.a (mon.0) 2706 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:43 smithi151 bash[23756]: audit 2024-03-04T16:01:43.159267+0000 mon.c (mon.2) 652 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:43 smithi151 bash[23756]: audit 2024-03-04T16:01:43.159868+0000 mon.a (mon.0) 2707 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:43 smithi151 bash[23756]: audit 2024-03-04T16:01:43.170215+0000 mon.a (mon.0) 2708 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:01:44.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[21480]: cluster 2024-03-04T16:01:43.031982+0000 mgr.y (mgr.24428) 470 : cluster 0 pgmap v703: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-03-04T16:01:44.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[21480]: cephadm 2024-03-04T16:01:43.161317+0000 mgr.y (mgr.24428) 471 : cephadm 1 Adjusting osd_memory_target on smithi117 to 3292M 2024-03-04T16:01:44.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[21480]: audit 2024-03-04T16:01:43.749769+0000 mon.c (mon.2) 653 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[21480]: audit 2024-03-04T16:01:43.750397+0000 mon.a (mon.0) 2709 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[21480]: audit 2024-03-04T16:01:43.752260+0000 mon.c (mon.2) 654 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[21480]: audit 2024-03-04T16:01:43.752801+0000 mon.a (mon.0) 2710 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[21480]: audit 2024-03-04T16:01:43.754748+0000 mon.c (mon.2) 655 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[21480]: audit 2024-03-04T16:01:43.755385+0000 mon.a (mon.0) 2711 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[21480]: audit 2024-03-04T16:01:43.757404+0000 mon.c (mon.2) 656 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[21480]: audit 2024-03-04T16:01:43.757967+0000 mon.a (mon.0) 2712 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[21480]: audit 2024-03-04T16:01:43.767962+0000 mon.a (mon.0) 2713 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:01:44.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[21480]: audit 2024-03-04T16:01:43.773660+0000 mon.c (mon.2) 657 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:01:44.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[21480]: audit 2024-03-04T16:01:43.775491+0000 mon.c (mon.2) 658 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:01:44.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[21480]: audit 2024-03-04T16:01:43.789559+0000 mon.a (mon.0) 2714 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:01:44.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[28843]: cluster 2024-03-04T16:01:43.031982+0000 mgr.y (mgr.24428) 470 : cluster 0 pgmap v703: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-03-04T16:01:44.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[28843]: cephadm 2024-03-04T16:01:43.161317+0000 mgr.y (mgr.24428) 471 : cephadm 1 Adjusting osd_memory_target on smithi117 to 3292M 2024-03-04T16:01:44.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[28843]: audit 2024-03-04T16:01:43.749769+0000 mon.c (mon.2) 653 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[28843]: audit 2024-03-04T16:01:43.750397+0000 mon.a (mon.0) 2709 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[28843]: audit 2024-03-04T16:01:43.752260+0000 mon.c (mon.2) 654 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[28843]: audit 2024-03-04T16:01:43.752801+0000 mon.a (mon.0) 2710 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[28843]: audit 2024-03-04T16:01:43.754748+0000 mon.c (mon.2) 655 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[28843]: audit 2024-03-04T16:01:43.755385+0000 mon.a (mon.0) 2711 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[28843]: audit 2024-03-04T16:01:43.757404+0000 mon.c (mon.2) 656 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[28843]: audit 2024-03-04T16:01:43.757967+0000 mon.a (mon.0) 2712 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[28843]: audit 2024-03-04T16:01:43.767962+0000 mon.a (mon.0) 2713 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:01:44.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[28843]: audit 2024-03-04T16:01:43.773660+0000 mon.c (mon.2) 657 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:01:44.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[28843]: audit 2024-03-04T16:01:43.775491+0000 mon.c (mon.2) 658 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:01:44.570 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:44 smithi117 bash[28843]: audit 2024-03-04T16:01:43.789559+0000 mon.a (mon.0) 2714 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:01:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:44 smithi151 bash[23756]: cluster 2024-03-04T16:01:43.031982+0000 mgr.y (mgr.24428) 470 : cluster 0 pgmap v703: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 895 B/s wr, 3 op/s 2024-03-04T16:01:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:44 smithi151 bash[23756]: cephadm 2024-03-04T16:01:43.161317+0000 mgr.y (mgr.24428) 471 : cephadm 1 Adjusting osd_memory_target on smithi117 to 3292M 2024-03-04T16:01:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:44 smithi151 bash[23756]: audit 2024-03-04T16:01:43.749769+0000 mon.c (mon.2) 653 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:44 smithi151 bash[23756]: audit 2024-03-04T16:01:43.750397+0000 mon.a (mon.0) 2709 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:44 smithi151 bash[23756]: audit 2024-03-04T16:01:43.752260+0000 mon.c (mon.2) 654 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:44 smithi151 bash[23756]: audit 2024-03-04T16:01:43.752801+0000 mon.a (mon.0) 2710 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:44 smithi151 bash[23756]: audit 2024-03-04T16:01:43.754748+0000 mon.c (mon.2) 655 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:44 smithi151 bash[23756]: audit 2024-03-04T16:01:43.755385+0000 mon.a (mon.0) 2711 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:44 smithi151 bash[23756]: audit 2024-03-04T16:01:43.757404+0000 mon.c (mon.2) 656 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:44 smithi151 bash[23756]: audit 2024-03-04T16:01:43.757967+0000 mon.a (mon.0) 2712 : audit 1 from='mgr.24428 ' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-03-04T16:01:44.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:44 smithi151 bash[23756]: audit 2024-03-04T16:01:43.767962+0000 mon.a (mon.0) 2713 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:01:44.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:44 smithi151 bash[23756]: audit 2024-03-04T16:01:43.773660+0000 mon.c (mon.2) 657 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:01:44.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:44 smithi151 bash[23756]: audit 2024-03-04T16:01:43.775491+0000 mon.c (mon.2) 658 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:01:44.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:44 smithi151 bash[23756]: audit 2024-03-04T16:01:43.789559+0000 mon.a (mon.0) 2714 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:01:45.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:45 smithi117 bash[28843]: cephadm 2024-03-04T16:01:43.759411+0000 mgr.y (mgr.24428) 472 : cephadm 1 Adjusting osd_memory_target on smithi151 to 3292M 2024-03-04T16:01:45.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:45 smithi117 bash[28843]: audit 2024-03-04T16:01:44.292756+0000 mon.c (mon.2) 659 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:01:45.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:45 smithi117 bash[21480]: cephadm 2024-03-04T16:01:43.759411+0000 mgr.y (mgr.24428) 472 : cephadm 1 Adjusting osd_memory_target on smithi151 to 3292M 2024-03-04T16:01:45.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:45 smithi117 bash[21480]: audit 2024-03-04T16:01:44.292756+0000 mon.c (mon.2) 659 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:01:45.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:45 smithi151 bash[23756]: cephadm 2024-03-04T16:01:43.759411+0000 mgr.y (mgr.24428) 472 : cephadm 1 Adjusting osd_memory_target on smithi151 to 3292M 2024-03-04T16:01:45.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:45 smithi151 bash[23756]: audit 2024-03-04T16:01:44.292756+0000 mon.c (mon.2) 659 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:01:46.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:46 smithi117 bash[21480]: cluster 2024-03-04T16:01:45.032750+0000 mgr.y (mgr.24428) 473 : cluster 0 pgmap v704: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 726 B/s wr, 3 op/s 2024-03-04T16:01:46.566 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:01:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:01:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:01:46.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:46 smithi117 bash[28843]: cluster 2024-03-04T16:01:45.032750+0000 mgr.y (mgr.24428) 473 : cluster 0 pgmap v704: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 726 B/s wr, 3 op/s 2024-03-04T16:01:46.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:46 smithi151 bash[23756]: cluster 2024-03-04T16:01:45.032750+0000 mgr.y (mgr.24428) 473 : cluster 0 pgmap v704: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 726 B/s wr, 3 op/s 2024-03-04T16:01:47.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:01:46 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:01:47.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:47 smithi117 bash[28843]: audit 2024-03-04T16:01:46.270640+0000 mon.c (mon.2) 660 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:47.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:47 smithi117 bash[28843]: audit 2024-03-04T16:01:46.271339+0000 mon.a (mon.0) 2715 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:47.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:47 smithi117 bash[28843]: audit 2024-03-04T16:01:46.272506+0000 mon.c (mon.2) 661 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-73"}]: dispatch 2024-03-04T16:01:47.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:47 smithi117 bash[28843]: audit 2024-03-04T16:01:46.273132+0000 mon.a (mon.0) 2716 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-73"}]: dispatch 2024-03-04T16:01:47.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:47 smithi117 bash[21480]: audit 2024-03-04T16:01:46.270640+0000 mon.c (mon.2) 660 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:47.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:47 smithi117 bash[21480]: audit 2024-03-04T16:01:46.271339+0000 mon.a (mon.0) 2715 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:47.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:47 smithi117 bash[21480]: audit 2024-03-04T16:01:46.272506+0000 mon.c (mon.2) 661 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-73"}]: dispatch 2024-03-04T16:01:47.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:47 smithi117 bash[21480]: audit 2024-03-04T16:01:46.273132+0000 mon.a (mon.0) 2716 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-73"}]: dispatch 2024-03-04T16:01:47.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:47 smithi151 bash[23756]: audit 2024-03-04T16:01:46.270640+0000 mon.c (mon.2) 660 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:47.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:47 smithi151 bash[23756]: audit 2024-03-04T16:01:46.271339+0000 mon.a (mon.0) 2715 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:47.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:47 smithi151 bash[23756]: audit 2024-03-04T16:01:46.272506+0000 mon.c (mon.2) 661 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-73"}]: dispatch 2024-03-04T16:01:47.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:47 smithi151 bash[23756]: audit 2024-03-04T16:01:46.273132+0000 mon.a (mon.0) 2716 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-73"}]: dispatch 2024-03-04T16:01:48.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:48 smithi117 bash[21480]: audit 2024-03-04T16:01:46.883207+0000 mgr.y (mgr.24428) 474 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:01:48.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:48 smithi117 bash[21480]: cluster 2024-03-04T16:01:47.035082+0000 mgr.y (mgr.24428) 475 : cluster 0 pgmap v705: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 659 B/s wr, 3 op/s 2024-03-04T16:01:48.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:48 smithi117 bash[21480]: cluster 2024-03-04T16:01:47.228972+0000 mon.a (mon.0) 2717 : cluster 0 osdmap e441: 8 total, 8 up, 8 in 2024-03-04T16:01:48.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:48 smithi117 bash[28843]: audit 2024-03-04T16:01:46.883207+0000 mgr.y (mgr.24428) 474 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:01:48.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:48 smithi117 bash[28843]: cluster 2024-03-04T16:01:47.035082+0000 mgr.y (mgr.24428) 475 : cluster 0 pgmap v705: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 659 B/s wr, 3 op/s 2024-03-04T16:01:48.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:48 smithi117 bash[28843]: cluster 2024-03-04T16:01:47.228972+0000 mon.a (mon.0) 2717 : cluster 0 osdmap e441: 8 total, 8 up, 8 in 2024-03-04T16:01:48.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:48 smithi151 bash[23756]: audit 2024-03-04T16:01:46.883207+0000 mgr.y (mgr.24428) 474 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:01:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:48 smithi151 bash[23756]: cluster 2024-03-04T16:01:47.035082+0000 mgr.y (mgr.24428) 475 : cluster 0 pgmap v705: 228 pgs: 228 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 1.9 KiB/s rd, 659 B/s wr, 3 op/s 2024-03-04T16:01:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:48 smithi151 bash[23756]: cluster 2024-03-04T16:01:47.228972+0000 mon.a (mon.0) 2717 : cluster 0 osdmap e441: 8 total, 8 up, 8 in 2024-03-04T16:01:49.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:49 smithi117 bash[21480]: cluster 2024-03-04T16:01:48.261821+0000 mon.a (mon.0) 2718 : cluster 0 osdmap e442: 8 total, 8 up, 8 in 2024-03-04T16:01:49.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:49 smithi117 bash[21480]: audit 2024-03-04T16:01:48.269705+0000 mon.c (mon.2) 662 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:49.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:49 smithi117 bash[21480]: audit 2024-03-04T16:01:48.277062+0000 mon.a (mon.0) 2719 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:49.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:49 smithi117 bash[28843]: cluster 2024-03-04T16:01:48.261821+0000 mon.a (mon.0) 2718 : cluster 0 osdmap e442: 8 total, 8 up, 8 in 2024-03-04T16:01:49.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:49 smithi117 bash[28843]: audit 2024-03-04T16:01:48.269705+0000 mon.c (mon.2) 662 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:49.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:49 smithi117 bash[28843]: audit 2024-03-04T16:01:48.277062+0000 mon.a (mon.0) 2719 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:49.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:49 smithi151 bash[23756]: cluster 2024-03-04T16:01:48.261821+0000 mon.a (mon.0) 2718 : cluster 0 osdmap e442: 8 total, 8 up, 8 in 2024-03-04T16:01:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:49 smithi151 bash[23756]: audit 2024-03-04T16:01:48.269705+0000 mon.c (mon.2) 662 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:49 smithi151 bash[23756]: audit 2024-03-04T16:01:48.277062+0000 mon.a (mon.0) 2719 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:50.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:50 smithi117 bash[21480]: cluster 2024-03-04T16:01:49.035911+0000 mgr.y (mgr.24428) 476 : cluster 0 pgmap v708: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-04T16:01:50.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:50 smithi117 bash[21480]: cluster 2024-03-04T16:01:49.253423+0000 mon.a (mon.0) 2720 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:01:50.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:50 smithi117 bash[21480]: audit 2024-03-04T16:01:49.265132+0000 mon.a (mon.0) 2721 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:50.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:50 smithi117 bash[21480]: cluster 2024-03-04T16:01:49.275933+0000 mon.a (mon.0) 2722 : cluster 0 osdmap e443: 8 total, 8 up, 8 in 2024-03-04T16:01:50.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:50 smithi117 bash[21480]: audit 2024-03-04T16:01:49.279916+0000 mon.c (mon.2) 663 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:50.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:50 smithi117 bash[28843]: cluster 2024-03-04T16:01:49.035911+0000 mgr.y (mgr.24428) 476 : cluster 0 pgmap v708: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-04T16:01:50.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:50 smithi117 bash[28843]: cluster 2024-03-04T16:01:49.253423+0000 mon.a (mon.0) 2720 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:01:50.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:50 smithi117 bash[28843]: audit 2024-03-04T16:01:49.265132+0000 mon.a (mon.0) 2721 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:50.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:50 smithi117 bash[28843]: cluster 2024-03-04T16:01:49.275933+0000 mon.a (mon.0) 2722 : cluster 0 osdmap e443: 8 total, 8 up, 8 in 2024-03-04T16:01:50.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:50 smithi117 bash[28843]: audit 2024-03-04T16:01:49.279916+0000 mon.c (mon.2) 663 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:50.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:50 smithi151 bash[23756]: cluster 2024-03-04T16:01:49.035911+0000 mgr.y (mgr.24428) 476 : cluster 0 pgmap v708: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-04T16:01:50.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:50 smithi151 bash[23756]: cluster 2024-03-04T16:01:49.253423+0000 mon.a (mon.0) 2720 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:01:50.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:50 smithi151 bash[23756]: audit 2024-03-04T16:01:49.265132+0000 mon.a (mon.0) 2721 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:50.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:50 smithi151 bash[23756]: cluster 2024-03-04T16:01:49.275933+0000 mon.a (mon.0) 2722 : cluster 0 osdmap e443: 8 total, 8 up, 8 in 2024-03-04T16:01:50.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:50 smithi151 bash[23756]: audit 2024-03-04T16:01:49.279916+0000 mon.c (mon.2) 663 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:51.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:51 smithi117 bash[21480]: cluster 2024-03-04T16:01:50.277047+0000 mon.a (mon.0) 2723 : cluster 0 osdmap e444: 8 total, 8 up, 8 in 2024-03-04T16:01:51.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:51 smithi117 bash[28843]: cluster 2024-03-04T16:01:50.277047+0000 mon.a (mon.0) 2723 : cluster 0 osdmap e444: 8 total, 8 up, 8 in 2024-03-04T16:01:51.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:51 smithi151 bash[23756]: cluster 2024-03-04T16:01:50.277047+0000 mon.a (mon.0) 2723 : cluster 0 osdmap e444: 8 total, 8 up, 8 in 2024-03-04T16:01:52.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:52 smithi117 bash[21480]: cluster 2024-03-04T16:01:51.036837+0000 mgr.y (mgr.24428) 477 : cluster 0 pgmap v711: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:01:52.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:52 smithi117 bash[21480]: cluster 2024-03-04T16:01:51.302304+0000 mon.a (mon.0) 2724 : cluster 0 osdmap e445: 8 total, 8 up, 8 in 2024-03-04T16:01:52.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:52 smithi117 bash[21480]: audit 2024-03-04T16:01:51.416731+0000 mon.c (mon.2) 664 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:52.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:52 smithi117 bash[21480]: audit 2024-03-04T16:01:51.417458+0000 mon.a (mon.0) 2725 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:52.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:52 smithi117 bash[21480]: audit 2024-03-04T16:01:51.418756+0000 mon.c (mon.2) 665 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-75"}]: dispatch 2024-03-04T16:01:52.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:52 smithi117 bash[21480]: audit 2024-03-04T16:01:51.419423+0000 mon.a (mon.0) 2726 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-75"}]: dispatch 2024-03-04T16:01:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:52 smithi117 bash[28843]: cluster 2024-03-04T16:01:51.036837+0000 mgr.y (mgr.24428) 477 : cluster 0 pgmap v711: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:01:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:52 smithi117 bash[28843]: cluster 2024-03-04T16:01:51.302304+0000 mon.a (mon.0) 2724 : cluster 0 osdmap e445: 8 total, 8 up, 8 in 2024-03-04T16:01:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:52 smithi117 bash[28843]: audit 2024-03-04T16:01:51.416731+0000 mon.c (mon.2) 664 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:52 smithi117 bash[28843]: audit 2024-03-04T16:01:51.417458+0000 mon.a (mon.0) 2725 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:52 smithi117 bash[28843]: audit 2024-03-04T16:01:51.418756+0000 mon.c (mon.2) 665 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-75"}]: dispatch 2024-03-04T16:01:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:52 smithi117 bash[28843]: audit 2024-03-04T16:01:51.419423+0000 mon.a (mon.0) 2726 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-75"}]: dispatch 2024-03-04T16:01:52.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:52 smithi151 bash[23756]: cluster 2024-03-04T16:01:51.036837+0000 mgr.y (mgr.24428) 477 : cluster 0 pgmap v711: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 617 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:01:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:52 smithi151 bash[23756]: cluster 2024-03-04T16:01:51.302304+0000 mon.a (mon.0) 2724 : cluster 0 osdmap e445: 8 total, 8 up, 8 in 2024-03-04T16:01:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:52 smithi151 bash[23756]: audit 2024-03-04T16:01:51.416731+0000 mon.c (mon.2) 664 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:52 smithi151 bash[23756]: audit 2024-03-04T16:01:51.417458+0000 mon.a (mon.0) 2725 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:52 smithi151 bash[23756]: audit 2024-03-04T16:01:51.418756+0000 mon.c (mon.2) 665 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-75"}]: dispatch 2024-03-04T16:01:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:52 smithi151 bash[23756]: audit 2024-03-04T16:01:51.419423+0000 mon.a (mon.0) 2726 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-75"}]: dispatch 2024-03-04T16:01:53.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:53 smithi117 bash[21480]: cluster 2024-03-04T16:01:52.301650+0000 mon.a (mon.0) 2727 : cluster 0 osdmap e446: 8 total, 8 up, 8 in 2024-03-04T16:01:53.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:53 smithi117 bash[28843]: cluster 2024-03-04T16:01:52.301650+0000 mon.a (mon.0) 2727 : cluster 0 osdmap e446: 8 total, 8 up, 8 in 2024-03-04T16:01:53.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:53 smithi151 bash[23756]: cluster 2024-03-04T16:01:52.301650+0000 mon.a (mon.0) 2727 : cluster 0 osdmap e446: 8 total, 8 up, 8 in 2024-03-04T16:01:54.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:54 smithi117 bash[28843]: cluster 2024-03-04T16:01:53.037752+0000 mgr.y (mgr.24428) 478 : cluster 0 pgmap v714: 196 pgs: 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T16:01:54.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:54 smithi117 bash[28843]: cluster 2024-03-04T16:01:53.326839+0000 mon.a (mon.0) 2728 : cluster 0 osdmap e447: 8 total, 8 up, 8 in 2024-03-04T16:01:54.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:54 smithi117 bash[28843]: audit 2024-03-04T16:01:53.331399+0000 mon.c (mon.2) 666 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:54.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:54 smithi117 bash[28843]: audit 2024-03-04T16:01:53.345683+0000 mon.a (mon.0) 2729 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:54.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:54 smithi117 bash[21480]: cluster 2024-03-04T16:01:53.037752+0000 mgr.y (mgr.24428) 478 : cluster 0 pgmap v714: 196 pgs: 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T16:01:54.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:54 smithi117 bash[21480]: cluster 2024-03-04T16:01:53.326839+0000 mon.a (mon.0) 2728 : cluster 0 osdmap e447: 8 total, 8 up, 8 in 2024-03-04T16:01:54.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:54 smithi117 bash[21480]: audit 2024-03-04T16:01:53.331399+0000 mon.c (mon.2) 666 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:54.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:54 smithi117 bash[21480]: audit 2024-03-04T16:01:53.345683+0000 mon.a (mon.0) 2729 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:54.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:54 smithi151 bash[23756]: cluster 2024-03-04T16:01:53.037752+0000 mgr.y (mgr.24428) 478 : cluster 0 pgmap v714: 196 pgs: 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T16:01:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:54 smithi151 bash[23756]: cluster 2024-03-04T16:01:53.326839+0000 mon.a (mon.0) 2728 : cluster 0 osdmap e447: 8 total, 8 up, 8 in 2024-03-04T16:01:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:54 smithi151 bash[23756]: audit 2024-03-04T16:01:53.331399+0000 mon.c (mon.2) 666 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:54 smithi151 bash[23756]: audit 2024-03-04T16:01:53.345683+0000 mon.a (mon.0) 2729 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:55.681 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 16:01:55 smithi151 bash[51061]: logger=cleanup t=2024-03-04T16:01:55.531090104Z level=info msg="Completed cleanup jobs" duration=181.586424ms 2024-03-04T16:01:55.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:55 smithi151 bash[23756]: audit 2024-03-04T16:01:54.328970+0000 mon.a (mon.0) 2730 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:55.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:55 smithi151 bash[23756]: cluster 2024-03-04T16:01:54.336144+0000 mon.a (mon.0) 2731 : cluster 0 osdmap e448: 8 total, 8 up, 8 in 2024-03-04T16:01:55.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:55 smithi151 bash[23756]: audit 2024-03-04T16:01:54.343952+0000 mon.c (mon.2) 667 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:55.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:55 smithi151 bash[23756]: cluster 2024-03-04T16:01:55.038745+0000 mgr.y (mgr.24428) 479 : cluster 0 pgmap v717: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T16:01:55.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:55 smithi117 bash[28843]: audit 2024-03-04T16:01:54.328970+0000 mon.a (mon.0) 2730 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:55.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:55 smithi117 bash[28843]: cluster 2024-03-04T16:01:54.336144+0000 mon.a (mon.0) 2731 : cluster 0 osdmap e448: 8 total, 8 up, 8 in 2024-03-04T16:01:55.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:55 smithi117 bash[28843]: audit 2024-03-04T16:01:54.343952+0000 mon.c (mon.2) 667 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:55.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:55 smithi117 bash[28843]: cluster 2024-03-04T16:01:55.038745+0000 mgr.y (mgr.24428) 479 : cluster 0 pgmap v717: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T16:01:55.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:55 smithi117 bash[21480]: audit 2024-03-04T16:01:54.328970+0000 mon.a (mon.0) 2730 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:55.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:55 smithi117 bash[21480]: cluster 2024-03-04T16:01:54.336144+0000 mon.a (mon.0) 2731 : cluster 0 osdmap e448: 8 total, 8 up, 8 in 2024-03-04T16:01:55.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:55 smithi117 bash[21480]: audit 2024-03-04T16:01:54.343952+0000 mon.c (mon.2) 667 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:55.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:55 smithi117 bash[21480]: cluster 2024-03-04T16:01:55.038745+0000 mgr.y (mgr.24428) 479 : cluster 0 pgmap v717: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T16:01:56.566 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:01:56 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:01:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:01:56.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:56 smithi117 bash[21480]: cluster 2024-03-04T16:01:55.349014+0000 mon.a (mon.0) 2732 : cluster 0 osdmap e449: 8 total, 8 up, 8 in 2024-03-04T16:01:56.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:56 smithi117 bash[21480]: audit 2024-03-04T16:01:55.462658+0000 mon.c (mon.2) 668 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:56.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:56 smithi117 bash[21480]: audit 2024-03-04T16:01:55.463375+0000 mon.a (mon.0) 2733 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:56.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:56 smithi117 bash[21480]: audit 2024-03-04T16:01:55.464556+0000 mon.c (mon.2) 669 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-77"}]: dispatch 2024-03-04T16:01:56.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:56 smithi117 bash[21480]: audit 2024-03-04T16:01:55.465195+0000 mon.a (mon.0) 2734 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-77"}]: dispatch 2024-03-04T16:01:56.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:56 smithi117 bash[28843]: cluster 2024-03-04T16:01:55.349014+0000 mon.a (mon.0) 2732 : cluster 0 osdmap e449: 8 total, 8 up, 8 in 2024-03-04T16:01:56.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:56 smithi117 bash[28843]: audit 2024-03-04T16:01:55.462658+0000 mon.c (mon.2) 668 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:56.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:56 smithi117 bash[28843]: audit 2024-03-04T16:01:55.463375+0000 mon.a (mon.0) 2733 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:56.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:56 smithi117 bash[28843]: audit 2024-03-04T16:01:55.464556+0000 mon.c (mon.2) 669 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-77"}]: dispatch 2024-03-04T16:01:56.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:56 smithi117 bash[28843]: audit 2024-03-04T16:01:55.465195+0000 mon.a (mon.0) 2734 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-77"}]: dispatch 2024-03-04T16:01:56.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:56 smithi151 bash[23756]: cluster 2024-03-04T16:01:55.349014+0000 mon.a (mon.0) 2732 : cluster 0 osdmap e449: 8 total, 8 up, 8 in 2024-03-04T16:01:56.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:56 smithi151 bash[23756]: audit 2024-03-04T16:01:55.462658+0000 mon.c (mon.2) 668 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:56.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:56 smithi151 bash[23756]: audit 2024-03-04T16:01:55.463375+0000 mon.a (mon.0) 2733 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:01:56.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:56 smithi151 bash[23756]: audit 2024-03-04T16:01:55.464556+0000 mon.c (mon.2) 669 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-77"}]: dispatch 2024-03-04T16:01:56.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:56 smithi151 bash[23756]: audit 2024-03-04T16:01:55.465195+0000 mon.a (mon.0) 2734 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-77"}]: dispatch 2024-03-04T16:01:57.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:01:56 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:01:57.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:57 smithi151 bash[23756]: cluster 2024-03-04T16:01:56.369748+0000 mon.a (mon.0) 2735 : cluster 0 osdmap e450: 8 total, 8 up, 8 in 2024-03-04T16:01:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:57 smithi151 bash[23756]: audit 2024-03-04T16:01:56.894756+0000 mgr.y (mgr.24428) 480 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:01:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:57 smithi151 bash[23756]: cluster 2024-03-04T16:01:57.040745+0000 mgr.y (mgr.24428) 481 : cluster 0 pgmap v720: 196 pgs: 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T16:01:57.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:57 smithi117 bash[28843]: cluster 2024-03-04T16:01:56.369748+0000 mon.a (mon.0) 2735 : cluster 0 osdmap e450: 8 total, 8 up, 8 in 2024-03-04T16:01:57.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:57 smithi117 bash[28843]: audit 2024-03-04T16:01:56.894756+0000 mgr.y (mgr.24428) 480 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:01:57.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:57 smithi117 bash[28843]: cluster 2024-03-04T16:01:57.040745+0000 mgr.y (mgr.24428) 481 : cluster 0 pgmap v720: 196 pgs: 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T16:01:57.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:57 smithi117 bash[21480]: cluster 2024-03-04T16:01:56.369748+0000 mon.a (mon.0) 2735 : cluster 0 osdmap e450: 8 total, 8 up, 8 in 2024-03-04T16:01:57.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:57 smithi117 bash[21480]: audit 2024-03-04T16:01:56.894756+0000 mgr.y (mgr.24428) 480 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:01:57.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:57 smithi117 bash[21480]: cluster 2024-03-04T16:01:57.040745+0000 mgr.y (mgr.24428) 481 : cluster 0 pgmap v720: 196 pgs: 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T16:01:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:58 smithi151 bash[23756]: cluster 2024-03-04T16:01:57.370841+0000 mon.a (mon.0) 2736 : cluster 0 osdmap e451: 8 total, 8 up, 8 in 2024-03-04T16:01:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:58 smithi151 bash[23756]: audit 2024-03-04T16:01:57.373832+0000 mon.c (mon.2) 670 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:58 smithi151 bash[23756]: audit 2024-03-04T16:01:57.375668+0000 mon.a (mon.0) 2737 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:58.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:58 smithi151 bash[23756]: audit 2024-03-04T16:01:58.363700+0000 mon.a (mon.0) 2738 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:58.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:58 smithi151 bash[23756]: cluster 2024-03-04T16:01:58.371968+0000 mon.a (mon.0) 2739 : cluster 0 osdmap e452: 8 total, 8 up, 8 in 2024-03-04T16:01:58.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:58 smithi117 bash[28843]: cluster 2024-03-04T16:01:57.370841+0000 mon.a (mon.0) 2736 : cluster 0 osdmap e451: 8 total, 8 up, 8 in 2024-03-04T16:01:58.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:58 smithi117 bash[28843]: audit 2024-03-04T16:01:57.373832+0000 mon.c (mon.2) 670 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:58.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:58 smithi117 bash[28843]: audit 2024-03-04T16:01:57.375668+0000 mon.a (mon.0) 2737 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:58.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:58 smithi117 bash[28843]: audit 2024-03-04T16:01:58.363700+0000 mon.a (mon.0) 2738 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:58.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:58 smithi117 bash[28843]: cluster 2024-03-04T16:01:58.371968+0000 mon.a (mon.0) 2739 : cluster 0 osdmap e452: 8 total, 8 up, 8 in 2024-03-04T16:01:58.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:58 smithi117 bash[21480]: cluster 2024-03-04T16:01:57.370841+0000 mon.a (mon.0) 2736 : cluster 0 osdmap e451: 8 total, 8 up, 8 in 2024-03-04T16:01:58.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:58 smithi117 bash[21480]: audit 2024-03-04T16:01:57.373832+0000 mon.c (mon.2) 670 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:58.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:58 smithi117 bash[21480]: audit 2024-03-04T16:01:57.375668+0000 mon.a (mon.0) 2737 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:01:58.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:58 smithi117 bash[21480]: audit 2024-03-04T16:01:58.363700+0000 mon.a (mon.0) 2738 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:01:58.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:58 smithi117 bash[21480]: cluster 2024-03-04T16:01:58.371968+0000 mon.a (mon.0) 2739 : cluster 0 osdmap e452: 8 total, 8 up, 8 in 2024-03-04T16:01:59.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:59 smithi151 bash[23756]: audit 2024-03-04T16:01:58.380169+0000 mon.c (mon.2) 671 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:59 smithi151 bash[23756]: cluster 2024-03-04T16:01:58.801407+0000 mon.a (mon.0) 2740 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:01:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:59 smithi151 bash[23756]: cluster 2024-03-04T16:01:59.041607+0000 mgr.y (mgr.24428) 482 : cluster 0 pgmap v723: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T16:01:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:01:59 smithi151 bash[23756]: audit 2024-03-04T16:01:59.327562+0000 mon.c (mon.2) 672 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:01:59.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:59 smithi117 bash[21480]: audit 2024-03-04T16:01:58.380169+0000 mon.c (mon.2) 671 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:59.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:59 smithi117 bash[21480]: cluster 2024-03-04T16:01:58.801407+0000 mon.a (mon.0) 2740 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:01:59.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:59 smithi117 bash[21480]: cluster 2024-03-04T16:01:59.041607+0000 mgr.y (mgr.24428) 482 : cluster 0 pgmap v723: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T16:01:59.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:01:59 smithi117 bash[21480]: audit 2024-03-04T16:01:59.327562+0000 mon.c (mon.2) 672 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:01:59.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:59 smithi117 bash[28843]: audit 2024-03-04T16:01:58.380169+0000 mon.c (mon.2) 671 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:01:59.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:59 smithi117 bash[28843]: cluster 2024-03-04T16:01:58.801407+0000 mon.a (mon.0) 2740 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:01:59.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:59 smithi117 bash[28843]: cluster 2024-03-04T16:01:59.041607+0000 mgr.y (mgr.24428) 482 : cluster 0 pgmap v723: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T16:01:59.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:01:59 smithi117 bash[28843]: audit 2024-03-04T16:01:59.327562+0000 mon.c (mon.2) 672 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:02:00.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:00 smithi151 bash[23756]: cluster 2024-03-04T16:01:59.423506+0000 mon.a (mon.0) 2741 : cluster 0 osdmap e453: 8 total, 8 up, 8 in 2024-03-04T16:02:00.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:00 smithi117 bash[21480]: cluster 2024-03-04T16:01:59.423506+0000 mon.a (mon.0) 2741 : cluster 0 osdmap e453: 8 total, 8 up, 8 in 2024-03-04T16:02:00.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:00 smithi117 bash[28843]: cluster 2024-03-04T16:01:59.423506+0000 mon.a (mon.0) 2741 : cluster 0 osdmap e453: 8 total, 8 up, 8 in 2024-03-04T16:02:01.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:01 smithi117 bash[21480]: cluster 2024-03-04T16:02:00.423472+0000 mon.a (mon.0) 2742 : cluster 0 osdmap e454: 8 total, 8 up, 8 in 2024-03-04T16:02:01.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:01 smithi117 bash[21480]: audit 2024-03-04T16:02:00.461625+0000 mon.c (mon.2) 673 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.19"}]: dispatch 2024-03-04T16:02:01.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:01 smithi117 bash[21480]: audit 2024-03-04T16:02:00.462046+0000 mgr.y (mgr.24428) 483 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.19"}]: dispatch 2024-03-04T16:02:01.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:01 smithi117 bash[21480]: cluster 2024-03-04T16:02:01.042433+0000 mgr.y (mgr.24428) 484 : cluster 0 pgmap v726: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:02:01.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:01 smithi117 bash[28843]: cluster 2024-03-04T16:02:00.423472+0000 mon.a (mon.0) 2742 : cluster 0 osdmap e454: 8 total, 8 up, 8 in 2024-03-04T16:02:01.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:01 smithi117 bash[28843]: audit 2024-03-04T16:02:00.461625+0000 mon.c (mon.2) 673 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.19"}]: dispatch 2024-03-04T16:02:01.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:01 smithi117 bash[28843]: audit 2024-03-04T16:02:00.462046+0000 mgr.y (mgr.24428) 483 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.19"}]: dispatch 2024-03-04T16:02:01.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:01 smithi117 bash[28843]: cluster 2024-03-04T16:02:01.042433+0000 mgr.y (mgr.24428) 484 : cluster 0 pgmap v726: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:02:01.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:01 smithi151 bash[23756]: cluster 2024-03-04T16:02:00.423472+0000 mon.a (mon.0) 2742 : cluster 0 osdmap e454: 8 total, 8 up, 8 in 2024-03-04T16:02:01.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:01 smithi151 bash[23756]: audit 2024-03-04T16:02:00.461625+0000 mon.c (mon.2) 673 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.19"}]: dispatch 2024-03-04T16:02:01.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:01 smithi151 bash[23756]: audit 2024-03-04T16:02:00.462046+0000 mgr.y (mgr.24428) 483 : audit 0 from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "305.19"}]: dispatch 2024-03-04T16:02:01.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:01 smithi151 bash[23756]: cluster 2024-03-04T16:02:01.042433+0000 mgr.y (mgr.24428) 484 : cluster 0 pgmap v726: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 618 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:02:02.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:02 smithi117 bash[21480]: cluster 2024-03-04T16:02:01.064148+0000 osd.0 (osd.0) 11 : cluster 0 305.19 deep-scrub starts 2024-03-04T16:02:02.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:02 smithi117 bash[21480]: cluster 2024-03-04T16:02:01.071247+0000 osd.0 (osd.0) 12 : cluster 0 305.19 deep-scrub ok 2024-03-04T16:02:02.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:02 smithi117 bash[28843]: cluster 2024-03-04T16:02:01.064148+0000 osd.0 (osd.0) 11 : cluster 0 305.19 deep-scrub starts 2024-03-04T16:02:02.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:02 smithi117 bash[28843]: cluster 2024-03-04T16:02:01.071247+0000 osd.0 (osd.0) 12 : cluster 0 305.19 deep-scrub ok 2024-03-04T16:02:02.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:02 smithi151 bash[23756]: cluster 2024-03-04T16:02:01.064148+0000 osd.0 (osd.0) 11 : cluster 0 305.19 deep-scrub starts 2024-03-04T16:02:02.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:02 smithi151 bash[23756]: cluster 2024-03-04T16:02:01.071247+0000 osd.0 (osd.0) 12 : cluster 0 305.19 deep-scrub ok 2024-03-04T16:02:03.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:03 smithi117 bash[21480]: cluster 2024-03-04T16:02:03.044662+0000 mgr.y (mgr.24428) 485 : cluster 0 pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 901 B/s rd, 901 B/s wr, 2 op/s 2024-03-04T16:02:03.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:03 smithi117 bash[28843]: cluster 2024-03-04T16:02:03.044662+0000 mgr.y (mgr.24428) 485 : cluster 0 pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 901 B/s rd, 901 B/s wr, 2 op/s 2024-03-04T16:02:03.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:03 smithi151 bash[23756]: cluster 2024-03-04T16:02:03.044662+0000 mgr.y (mgr.24428) 485 : cluster 0 pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 901 B/s rd, 901 B/s wr, 2 op/s 2024-03-04T16:02:06.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:06 smithi151 bash[23756]: cluster 2024-03-04T16:02:05.045675+0000 mgr.y (mgr.24428) 486 : cluster 0 pgmap v728: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 766 B/s rd, 766 B/s wr, 2 op/s 2024-03-04T16:02:06.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:06 smithi117 bash[21480]: cluster 2024-03-04T16:02:05.045675+0000 mgr.y (mgr.24428) 486 : cluster 0 pgmap v728: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 766 B/s rd, 766 B/s wr, 2 op/s 2024-03-04T16:02:06.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:06 smithi117 bash[28843]: cluster 2024-03-04T16:02:05.045675+0000 mgr.y (mgr.24428) 486 : cluster 0 pgmap v728: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 766 B/s rd, 766 B/s wr, 2 op/s 2024-03-04T16:02:06.566 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:02:06 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:02:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:02:07.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:02:06 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:02:08.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:08 smithi151 bash[23756]: audit 2024-03-04T16:02:06.906199+0000 mgr.y (mgr.24428) 487 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:02:08.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:08 smithi151 bash[23756]: cluster 2024-03-04T16:02:07.047878+0000 mgr.y (mgr.24428) 488 : cluster 0 pgmap v729: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-03-04T16:02:08.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:08 smithi117 bash[21480]: audit 2024-03-04T16:02:06.906199+0000 mgr.y (mgr.24428) 487 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:02:08.569 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:08 smithi117 bash[21480]: cluster 2024-03-04T16:02:07.047878+0000 mgr.y (mgr.24428) 488 : cluster 0 pgmap v729: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-03-04T16:02:08.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:08 smithi117 bash[28843]: audit 2024-03-04T16:02:06.906199+0000 mgr.y (mgr.24428) 487 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:02:08.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:08 smithi117 bash[28843]: cluster 2024-03-04T16:02:07.047878+0000 mgr.y (mgr.24428) 488 : cluster 0 pgmap v729: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 639 B/s wr, 2 op/s 2024-03-04T16:02:10.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:10 smithi151 bash[23756]: cluster 2024-03-04T16:02:09.048702+0000 mgr.y (mgr.24428) 489 : cluster 0 pgmap v730: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 531 B/s wr, 2 op/s 2024-03-04T16:02:10.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:10 smithi117 bash[21480]: cluster 2024-03-04T16:02:09.048702+0000 mgr.y (mgr.24428) 489 : cluster 0 pgmap v730: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 531 B/s wr, 2 op/s 2024-03-04T16:02:10.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:10 smithi117 bash[28843]: cluster 2024-03-04T16:02:09.048702+0000 mgr.y (mgr.24428) 489 : cluster 0 pgmap v730: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 531 B/s wr, 2 op/s 2024-03-04T16:02:12.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:12 smithi151 bash[23756]: cluster 2024-03-04T16:02:11.049567+0000 mgr.y (mgr.24428) 490 : cluster 0 pgmap v731: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 963 B/s rd, 481 B/s wr, 1 op/s 2024-03-04T16:02:12.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:12 smithi117 bash[21480]: cluster 2024-03-04T16:02:11.049567+0000 mgr.y (mgr.24428) 490 : cluster 0 pgmap v731: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 963 B/s rd, 481 B/s wr, 1 op/s 2024-03-04T16:02:12.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:12 smithi117 bash[28843]: cluster 2024-03-04T16:02:11.049567+0000 mgr.y (mgr.24428) 490 : cluster 0 pgmap v731: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 963 B/s rd, 481 B/s wr, 1 op/s 2024-03-04T16:02:14.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:14 smithi151 bash[23756]: cluster 2024-03-04T16:02:13.051799+0000 mgr.y (mgr.24428) 491 : cluster 0 pgmap v732: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-03-04T16:02:14.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:14 smithi117 bash[21480]: cluster 2024-03-04T16:02:13.051799+0000 mgr.y (mgr.24428) 491 : cluster 0 pgmap v732: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-03-04T16:02:14.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:14 smithi117 bash[28843]: cluster 2024-03-04T16:02:13.051799+0000 mgr.y (mgr.24428) 491 : cluster 0 pgmap v732: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 426 B/s wr, 1 op/s 2024-03-04T16:02:15.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:15 smithi151 bash[23756]: audit 2024-03-04T16:02:14.361719+0000 mon.c (mon.2) 674 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:02:15.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:15 smithi117 bash[21480]: audit 2024-03-04T16:02:14.361719+0000 mon.c (mon.2) 674 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:02:15.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:15 smithi117 bash[28843]: audit 2024-03-04T16:02:14.361719+0000 mon.c (mon.2) 674 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:02:16.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:16 smithi151 bash[23756]: cluster 2024-03-04T16:02:15.052645+0000 mgr.y (mgr.24428) 492 : cluster 0 pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-04T16:02:16.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:16 smithi117 bash[21480]: cluster 2024-03-04T16:02:15.052645+0000 mgr.y (mgr.24428) 492 : cluster 0 pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-04T16:02:16.566 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:02:16 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:02:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:02:16.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:16 smithi117 bash[28843]: cluster 2024-03-04T16:02:15.052645+0000 mgr.y (mgr.24428) 492 : cluster 0 pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-04T16:02:17.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:02:16 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:02:18.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:18 smithi151 bash[23756]: audit 2024-03-04T16:02:16.917543+0000 mgr.y (mgr.24428) 493 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:02:18.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:18 smithi151 bash[23756]: cluster 2024-03-04T16:02:17.054865+0000 mgr.y (mgr.24428) 494 : cluster 0 pgmap v734: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:18.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:18 smithi117 bash[21480]: audit 2024-03-04T16:02:16.917543+0000 mgr.y (mgr.24428) 493 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:02:18.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:18 smithi117 bash[21480]: cluster 2024-03-04T16:02:17.054865+0000 mgr.y (mgr.24428) 494 : cluster 0 pgmap v734: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:18.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:18 smithi117 bash[28843]: audit 2024-03-04T16:02:16.917543+0000 mgr.y (mgr.24428) 493 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:02:18.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:18 smithi117 bash[28843]: cluster 2024-03-04T16:02:17.054865+0000 mgr.y (mgr.24428) 494 : cluster 0 pgmap v734: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:20.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:20 smithi151 bash[23756]: cluster 2024-03-04T16:02:19.055685+0000 mgr.y (mgr.24428) 495 : cluster 0 pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-04T16:02:20.489 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:20 smithi117 bash[21480]: cluster 2024-03-04T16:02:19.055685+0000 mgr.y (mgr.24428) 495 : cluster 0 pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-04T16:02:20.489 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:20 smithi117 bash[28843]: cluster 2024-03-04T16:02:19.055685+0000 mgr.y (mgr.24428) 495 : cluster 0 pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-04T16:02:20.816 INFO:journalctl@ceph.osd.0.smithi117.stdout:Mar 04 16:02:20 smithi117 bash[31848]: debug 2024-03-04T16:02:20.489+0000 7f1be0dbd700 -1 snap_mapper.add_oid found existing snaps mapped on 305:9bb03a75:test-rados-api-smithi117-64061-80::foo:2, removing 2024-03-04T16:02:20.932 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 16:02:20 smithi151 bash[27331]: debug 2024-03-04T16:02:20.483+0000 7fd74e23c700 -1 snap_mapper.add_oid found existing snaps mapped on 305:9bb03a75:test-rados-api-smithi117-64061-80::foo:2, removing 2024-03-04T16:02:20.932 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:02:20 smithi151 bash[38263]: debug 2024-03-04T16:02:20.483+0000 7f12b7194700 -1 snap_mapper.add_oid found existing snaps mapped on 305:9bb03a75:test-rados-api-smithi117-64061-80::foo:2, removing 2024-03-04T16:02:21.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:21 smithi151 bash[23756]: audit 2024-03-04T16:02:20.497527+0000 mon.c (mon.2) 675 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:21.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:21 smithi151 bash[23756]: audit 2024-03-04T16:02:20.498253+0000 mon.a (mon.0) 2743 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:21.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:21 smithi151 bash[23756]: audit 2024-03-04T16:02:20.499402+0000 mon.c (mon.2) 676 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-79"}]: dispatch 2024-03-04T16:02:21.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:21 smithi151 bash[23756]: audit 2024-03-04T16:02:20.500056+0000 mon.a (mon.0) 2744 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-79"}]: dispatch 2024-03-04T16:02:21.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:21 smithi117 bash[21480]: audit 2024-03-04T16:02:20.497527+0000 mon.c (mon.2) 675 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:21.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:21 smithi117 bash[21480]: audit 2024-03-04T16:02:20.498253+0000 mon.a (mon.0) 2743 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:21.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:21 smithi117 bash[21480]: audit 2024-03-04T16:02:20.499402+0000 mon.c (mon.2) 676 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-79"}]: dispatch 2024-03-04T16:02:21.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:21 smithi117 bash[21480]: audit 2024-03-04T16:02:20.500056+0000 mon.a (mon.0) 2744 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-79"}]: dispatch 2024-03-04T16:02:21.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:21 smithi117 bash[28843]: audit 2024-03-04T16:02:20.497527+0000 mon.c (mon.2) 675 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:21.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:21 smithi117 bash[28843]: audit 2024-03-04T16:02:20.498253+0000 mon.a (mon.0) 2743 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:21.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:21 smithi117 bash[28843]: audit 2024-03-04T16:02:20.499402+0000 mon.c (mon.2) 676 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-79"}]: dispatch 2024-03-04T16:02:21.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:21 smithi117 bash[28843]: audit 2024-03-04T16:02:20.500056+0000 mon.a (mon.0) 2744 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-79"}]: dispatch 2024-03-04T16:02:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:22 smithi151 bash[23756]: cluster 2024-03-04T16:02:21.056516+0000 mgr.y (mgr.24428) 496 : cluster 0 pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-04T16:02:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:22 smithi151 bash[23756]: cluster 2024-03-04T16:02:21.167083+0000 mon.a (mon.0) 2745 : cluster 0 osdmap e455: 8 total, 8 up, 8 in 2024-03-04T16:02:22.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:22 smithi117 bash[21480]: cluster 2024-03-04T16:02:21.056516+0000 mgr.y (mgr.24428) 496 : cluster 0 pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-04T16:02:22.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:22 smithi117 bash[21480]: cluster 2024-03-04T16:02:21.167083+0000 mon.a (mon.0) 2745 : cluster 0 osdmap e455: 8 total, 8 up, 8 in 2024-03-04T16:02:22.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:22 smithi117 bash[28843]: cluster 2024-03-04T16:02:21.056516+0000 mgr.y (mgr.24428) 496 : cluster 0 pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-03-04T16:02:22.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:22 smithi117 bash[28843]: cluster 2024-03-04T16:02:21.167083+0000 mon.a (mon.0) 2745 : cluster 0 osdmap e455: 8 total, 8 up, 8 in 2024-03-04T16:02:23.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:23 smithi117 bash[21480]: cluster 2024-03-04T16:02:22.166167+0000 mon.a (mon.0) 2746 : cluster 0 osdmap e456: 8 total, 8 up, 8 in 2024-03-04T16:02:23.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:23 smithi117 bash[21480]: audit 2024-03-04T16:02:22.168551+0000 mon.c (mon.2) 677 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:23.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:23 smithi117 bash[21480]: audit 2024-03-04T16:02:22.184250+0000 mon.a (mon.0) 2747 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:23.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:23 smithi117 bash[21480]: audit 2024-03-04T16:02:23.159545+0000 mon.a (mon.0) 2748 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:02:23.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:23 smithi117 bash[28843]: cluster 2024-03-04T16:02:22.166167+0000 mon.a (mon.0) 2746 : cluster 0 osdmap e456: 8 total, 8 up, 8 in 2024-03-04T16:02:23.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:23 smithi117 bash[28843]: audit 2024-03-04T16:02:22.168551+0000 mon.c (mon.2) 677 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:23.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:23 smithi117 bash[28843]: audit 2024-03-04T16:02:22.184250+0000 mon.a (mon.0) 2747 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:23.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:23 smithi117 bash[28843]: audit 2024-03-04T16:02:23.159545+0000 mon.a (mon.0) 2748 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:02:23.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:23 smithi151 bash[23756]: cluster 2024-03-04T16:02:22.166167+0000 mon.a (mon.0) 2746 : cluster 0 osdmap e456: 8 total, 8 up, 8 in 2024-03-04T16:02:23.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:23 smithi151 bash[23756]: audit 2024-03-04T16:02:22.168551+0000 mon.c (mon.2) 677 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:23.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:23 smithi151 bash[23756]: audit 2024-03-04T16:02:22.184250+0000 mon.a (mon.0) 2747 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:23.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:23 smithi151 bash[23756]: audit 2024-03-04T16:02:23.159545+0000 mon.a (mon.0) 2748 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:02:24.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:24 smithi117 bash[21480]: cluster 2024-03-04T16:02:23.057393+0000 mgr.y (mgr.24428) 497 : cluster 0 pgmap v739: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-03-04T16:02:24.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:24 smithi117 bash[21480]: cluster 2024-03-04T16:02:23.172354+0000 mon.a (mon.0) 2749 : cluster 0 osdmap e457: 8 total, 8 up, 8 in 2024-03-04T16:02:24.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:24 smithi117 bash[21480]: audit 2024-03-04T16:02:23.176554+0000 mon.c (mon.2) 678 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:02:24.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:24 smithi117 bash[21480]: cluster 2024-03-04T16:02:23.177631+0000 mon.a (mon.0) 2750 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:02:24.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:24 smithi117 bash[21480]: audit 2024-03-04T16:02:23.200958+0000 mon.c (mon.2) 679 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:24.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:24 smithi117 bash[21480]: audit 2024-03-04T16:02:23.202329+0000 mon.a (mon.0) 2751 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:24.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:24 smithi117 bash[21480]: audit 2024-03-04T16:02:24.193240+0000 mon.a (mon.0) 2752 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:02:24.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:24 smithi117 bash[21480]: cluster 2024-03-04T16:02:24.200471+0000 mon.a (mon.0) 2753 : cluster 0 osdmap e458: 8 total, 8 up, 8 in 2024-03-04T16:02:24.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:24 smithi117 bash[21480]: audit 2024-03-04T16:02:24.205095+0000 mon.c (mon.2) 680 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:24.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:24 smithi117 bash[21480]: audit 2024-03-04T16:02:24.208061+0000 mon.a (mon.0) 2754 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:24.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:24 smithi117 bash[28843]: cluster 2024-03-04T16:02:23.057393+0000 mgr.y (mgr.24428) 497 : cluster 0 pgmap v739: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-03-04T16:02:24.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:24 smithi117 bash[28843]: cluster 2024-03-04T16:02:23.172354+0000 mon.a (mon.0) 2749 : cluster 0 osdmap e457: 8 total, 8 up, 8 in 2024-03-04T16:02:24.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:24 smithi117 bash[28843]: audit 2024-03-04T16:02:23.176554+0000 mon.c (mon.2) 678 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:02:24.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:24 smithi117 bash[28843]: cluster 2024-03-04T16:02:23.177631+0000 mon.a (mon.0) 2750 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:02:24.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:24 smithi117 bash[28843]: audit 2024-03-04T16:02:23.200958+0000 mon.c (mon.2) 679 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:24.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:24 smithi117 bash[28843]: audit 2024-03-04T16:02:23.202329+0000 mon.a (mon.0) 2751 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:24.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:24 smithi117 bash[28843]: audit 2024-03-04T16:02:24.193240+0000 mon.a (mon.0) 2752 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:02:24.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:24 smithi117 bash[28843]: cluster 2024-03-04T16:02:24.200471+0000 mon.a (mon.0) 2753 : cluster 0 osdmap e458: 8 total, 8 up, 8 in 2024-03-04T16:02:24.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:24 smithi117 bash[28843]: audit 2024-03-04T16:02:24.205095+0000 mon.c (mon.2) 680 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:24.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:24 smithi117 bash[28843]: audit 2024-03-04T16:02:24.208061+0000 mon.a (mon.0) 2754 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:24.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:24 smithi151 bash[23756]: cluster 2024-03-04T16:02:23.057393+0000 mgr.y (mgr.24428) 497 : cluster 0 pgmap v739: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 1 op/s 2024-03-04T16:02:24.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:24 smithi151 bash[23756]: cluster 2024-03-04T16:02:23.172354+0000 mon.a (mon.0) 2749 : cluster 0 osdmap e457: 8 total, 8 up, 8 in 2024-03-04T16:02:24.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:24 smithi151 bash[23756]: audit 2024-03-04T16:02:23.176554+0000 mon.c (mon.2) 678 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:02:24.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:24 smithi151 bash[23756]: cluster 2024-03-04T16:02:23.177631+0000 mon.a (mon.0) 2750 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:02:24.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:24 smithi151 bash[23756]: audit 2024-03-04T16:02:23.200958+0000 mon.c (mon.2) 679 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:24.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:24 smithi151 bash[23756]: audit 2024-03-04T16:02:23.202329+0000 mon.a (mon.0) 2751 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:24.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:24 smithi151 bash[23756]: audit 2024-03-04T16:02:24.193240+0000 mon.a (mon.0) 2752 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:02:24.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:24 smithi151 bash[23756]: cluster 2024-03-04T16:02:24.200471+0000 mon.a (mon.0) 2753 : cluster 0 osdmap e458: 8 total, 8 up, 8 in 2024-03-04T16:02:24.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:24 smithi151 bash[23756]: audit 2024-03-04T16:02:24.205095+0000 mon.c (mon.2) 680 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:24.685 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:24 smithi151 bash[23756]: audit 2024-03-04T16:02:24.208061+0000 mon.a (mon.0) 2754 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:26.566 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:02:26 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:02:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:02:26.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:26 smithi117 bash[28843]: cluster 2024-03-04T16:02:25.058166+0000 mgr.y (mgr.24428) 498 : cluster 0 pgmap v742: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-03-04T16:02:26.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:26 smithi117 bash[28843]: audit 2024-03-04T16:02:25.196133+0000 mon.a (mon.0) 2755 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T16:02:26.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:26 smithi117 bash[28843]: cluster 2024-03-04T16:02:25.204072+0000 mon.a (mon.0) 2756 : cluster 0 osdmap e459: 8 total, 8 up, 8 in 2024-03-04T16:02:26.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:26 smithi117 bash[28843]: audit 2024-03-04T16:02:25.205136+0000 mon.c (mon.2) 681 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:26.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:26 smithi117 bash[28843]: audit 2024-03-04T16:02:25.206568+0000 mon.a (mon.0) 2757 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:26.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:26 smithi117 bash[21480]: cluster 2024-03-04T16:02:25.058166+0000 mgr.y (mgr.24428) 498 : cluster 0 pgmap v742: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-03-04T16:02:26.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:26 smithi117 bash[21480]: audit 2024-03-04T16:02:25.196133+0000 mon.a (mon.0) 2755 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T16:02:26.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:26 smithi117 bash[21480]: cluster 2024-03-04T16:02:25.204072+0000 mon.a (mon.0) 2756 : cluster 0 osdmap e459: 8 total, 8 up, 8 in 2024-03-04T16:02:26.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:26 smithi117 bash[21480]: audit 2024-03-04T16:02:25.205136+0000 mon.c (mon.2) 681 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:26.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:26 smithi117 bash[21480]: audit 2024-03-04T16:02:25.206568+0000 mon.a (mon.0) 2757 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:26.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:26 smithi151 bash[23756]: cluster 2024-03-04T16:02:25.058166+0000 mgr.y (mgr.24428) 498 : cluster 0 pgmap v742: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 619 MiB used, 715 GiB / 715 GiB avail; 2.0 KiB/s rd, 1 op/s 2024-03-04T16:02:26.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:26 smithi151 bash[23756]: audit 2024-03-04T16:02:25.196133+0000 mon.a (mon.0) 2755 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T16:02:26.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:26 smithi151 bash[23756]: cluster 2024-03-04T16:02:25.204072+0000 mon.a (mon.0) 2756 : cluster 0 osdmap e459: 8 total, 8 up, 8 in 2024-03-04T16:02:26.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:26 smithi151 bash[23756]: audit 2024-03-04T16:02:25.205136+0000 mon.c (mon.2) 681 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:26.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:26 smithi151 bash[23756]: audit 2024-03-04T16:02:25.206568+0000 mon.a (mon.0) 2757 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:27.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:02:26 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:02:27.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:27 smithi117 bash[21480]: audit 2024-03-04T16:02:26.207533+0000 mon.a (mon.0) 2758 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-04T16:02:27.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:27 smithi117 bash[21480]: cluster 2024-03-04T16:02:26.215750+0000 mon.a (mon.0) 2759 : cluster 0 osdmap e460: 8 total, 8 up, 8 in 2024-03-04T16:02:27.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:27 smithi117 bash[21480]: audit 2024-03-04T16:02:26.216795+0000 mon.c (mon.2) 682 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:27.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:27 smithi117 bash[21480]: audit 2024-03-04T16:02:26.218330+0000 mon.a (mon.0) 2760 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:27.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:27 smithi117 bash[28843]: audit 2024-03-04T16:02:26.207533+0000 mon.a (mon.0) 2758 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-04T16:02:27.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:27 smithi117 bash[28843]: cluster 2024-03-04T16:02:26.215750+0000 mon.a (mon.0) 2759 : cluster 0 osdmap e460: 8 total, 8 up, 8 in 2024-03-04T16:02:27.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:27 smithi117 bash[28843]: audit 2024-03-04T16:02:26.216795+0000 mon.c (mon.2) 682 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:27.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:27 smithi117 bash[28843]: audit 2024-03-04T16:02:26.218330+0000 mon.a (mon.0) 2760 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:27.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:27 smithi151 bash[23756]: audit 2024-03-04T16:02:26.207533+0000 mon.a (mon.0) 2758 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-04T16:02:27.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:27 smithi151 bash[23756]: cluster 2024-03-04T16:02:26.215750+0000 mon.a (mon.0) 2759 : cluster 0 osdmap e460: 8 total, 8 up, 8 in 2024-03-04T16:02:27.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:27 smithi151 bash[23756]: audit 2024-03-04T16:02:26.216795+0000 mon.c (mon.2) 682 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:27.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:27 smithi151 bash[23756]: audit 2024-03-04T16:02:26.218330+0000 mon.a (mon.0) 2760 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:28.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:28 smithi117 bash[21480]: audit 2024-03-04T16:02:26.929042+0000 mgr.y (mgr.24428) 499 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:02:28.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:28 smithi117 bash[21480]: cluster 2024-03-04T16:02:27.060325+0000 mgr.y (mgr.24428) 500 : cluster 0 pgmap v745: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:28.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:28 smithi117 bash[21480]: audit 2024-03-04T16:02:27.216343+0000 mon.a (mon.0) 2761 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-04T16:02:28.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:28 smithi117 bash[21480]: cluster 2024-03-04T16:02:27.225384+0000 mon.a (mon.0) 2762 : cluster 0 osdmap e461: 8 total, 8 up, 8 in 2024-03-04T16:02:28.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:28 smithi117 bash[21480]: audit 2024-03-04T16:02:27.272413+0000 mon.c (mon.2) 683 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-04T16:02:28.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:28 smithi117 bash[21480]: audit 2024-03-04T16:02:27.273349+0000 mon.a (mon.0) 2763 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-04T16:02:28.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:28 smithi117 bash[28843]: audit 2024-03-04T16:02:26.929042+0000 mgr.y (mgr.24428) 499 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:02:28.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:28 smithi117 bash[28843]: cluster 2024-03-04T16:02:27.060325+0000 mgr.y (mgr.24428) 500 : cluster 0 pgmap v745: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:28.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:28 smithi117 bash[28843]: audit 2024-03-04T16:02:27.216343+0000 mon.a (mon.0) 2761 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-04T16:02:28.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:28 smithi117 bash[28843]: cluster 2024-03-04T16:02:27.225384+0000 mon.a (mon.0) 2762 : cluster 0 osdmap e461: 8 total, 8 up, 8 in 2024-03-04T16:02:28.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:28 smithi117 bash[28843]: audit 2024-03-04T16:02:27.272413+0000 mon.c (mon.2) 683 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-04T16:02:28.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:28 smithi117 bash[28843]: audit 2024-03-04T16:02:27.273349+0000 mon.a (mon.0) 2763 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-04T16:02:28.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:28 smithi151 bash[23756]: audit 2024-03-04T16:02:26.929042+0000 mgr.y (mgr.24428) 499 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:02:28.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:28 smithi151 bash[23756]: cluster 2024-03-04T16:02:27.060325+0000 mgr.y (mgr.24428) 500 : cluster 0 pgmap v745: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:28.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:28 smithi151 bash[23756]: audit 2024-03-04T16:02:27.216343+0000 mon.a (mon.0) 2761 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-04T16:02:28.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:28 smithi151 bash[23756]: cluster 2024-03-04T16:02:27.225384+0000 mon.a (mon.0) 2762 : cluster 0 osdmap e461: 8 total, 8 up, 8 in 2024-03-04T16:02:28.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:28 smithi151 bash[23756]: audit 2024-03-04T16:02:27.272413+0000 mon.c (mon.2) 683 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-04T16:02:28.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:28 smithi151 bash[23756]: audit 2024-03-04T16:02:27.273349+0000 mon.a (mon.0) 2763 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-03-04T16:02:29.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:29 smithi117 bash[21480]: audit 2024-03-04T16:02:28.219194+0000 mon.a (mon.0) 2764 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-03-04T16:02:29.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:29 smithi117 bash[21480]: cluster 2024-03-04T16:02:28.226911+0000 mon.a (mon.0) 2765 : cluster 0 osdmap e462: 8 total, 8 up, 8 in 2024-03-04T16:02:29.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:29 smithi117 bash[21480]: audit 2024-03-04T16:02:28.287676+0000 mon.c (mon.2) 684 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-04T16:02:29.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:29 smithi117 bash[21480]: audit 2024-03-04T16:02:28.288738+0000 mon.a (mon.0) 2766 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-04T16:02:29.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:29 smithi117 bash[21480]: cluster 2024-03-04T16:02:28.808940+0000 mon.a (mon.0) 2767 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:02:29.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:29 smithi117 bash[28843]: audit 2024-03-04T16:02:28.219194+0000 mon.a (mon.0) 2764 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-03-04T16:02:29.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:29 smithi117 bash[28843]: cluster 2024-03-04T16:02:28.226911+0000 mon.a (mon.0) 2765 : cluster 0 osdmap e462: 8 total, 8 up, 8 in 2024-03-04T16:02:29.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:29 smithi117 bash[28843]: audit 2024-03-04T16:02:28.287676+0000 mon.c (mon.2) 684 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-04T16:02:29.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:29 smithi117 bash[28843]: audit 2024-03-04T16:02:28.288738+0000 mon.a (mon.0) 2766 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-04T16:02:29.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:29 smithi117 bash[28843]: cluster 2024-03-04T16:02:28.808940+0000 mon.a (mon.0) 2767 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:02:29.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:29 smithi151 bash[23756]: audit 2024-03-04T16:02:28.219194+0000 mon.a (mon.0) 2764 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-03-04T16:02:29.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:29 smithi151 bash[23756]: cluster 2024-03-04T16:02:28.226911+0000 mon.a (mon.0) 2765 : cluster 0 osdmap e462: 8 total, 8 up, 8 in 2024-03-04T16:02:29.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:29 smithi151 bash[23756]: audit 2024-03-04T16:02:28.287676+0000 mon.c (mon.2) 684 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-04T16:02:29.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:29 smithi151 bash[23756]: audit 2024-03-04T16:02:28.288738+0000 mon.a (mon.0) 2766 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-03-04T16:02:29.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:29 smithi151 bash[23756]: cluster 2024-03-04T16:02:28.808940+0000 mon.a (mon.0) 2767 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:02:30.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:30 smithi117 bash[21480]: cluster 2024-03-04T16:02:29.061184+0000 mgr.y (mgr.24428) 501 : cluster 0 pgmap v748: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:30.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:30 smithi117 bash[21480]: audit 2024-03-04T16:02:29.240521+0000 mon.a (mon.0) 2768 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-03-04T16:02:30.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:30 smithi117 bash[21480]: cluster 2024-03-04T16:02:29.250671+0000 mon.a (mon.0) 2769 : cluster 0 osdmap e463: 8 total, 8 up, 8 in 2024-03-04T16:02:30.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:30 smithi117 bash[21480]: audit 2024-03-04T16:02:29.283552+0000 mon.c (mon.2) 685 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:30.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:30 smithi117 bash[21480]: audit 2024-03-04T16:02:29.284511+0000 mon.a (mon.0) 2770 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:30.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:30 smithi117 bash[21480]: audit 2024-03-04T16:02:29.395251+0000 mon.c (mon.2) 686 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:02:30.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:30 smithi117 bash[28843]: cluster 2024-03-04T16:02:29.061184+0000 mgr.y (mgr.24428) 501 : cluster 0 pgmap v748: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:30.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:30 smithi117 bash[28843]: audit 2024-03-04T16:02:29.240521+0000 mon.a (mon.0) 2768 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-03-04T16:02:30.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:30 smithi117 bash[28843]: cluster 2024-03-04T16:02:29.250671+0000 mon.a (mon.0) 2769 : cluster 0 osdmap e463: 8 total, 8 up, 8 in 2024-03-04T16:02:30.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:30 smithi117 bash[28843]: audit 2024-03-04T16:02:29.283552+0000 mon.c (mon.2) 685 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:30.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:30 smithi117 bash[28843]: audit 2024-03-04T16:02:29.284511+0000 mon.a (mon.0) 2770 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:30.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:30 smithi117 bash[28843]: audit 2024-03-04T16:02:29.395251+0000 mon.c (mon.2) 686 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:02:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:30 smithi151 bash[23756]: cluster 2024-03-04T16:02:29.061184+0000 mgr.y (mgr.24428) 501 : cluster 0 pgmap v748: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:30 smithi151 bash[23756]: audit 2024-03-04T16:02:29.240521+0000 mon.a (mon.0) 2768 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-03-04T16:02:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:30 smithi151 bash[23756]: cluster 2024-03-04T16:02:29.250671+0000 mon.a (mon.0) 2769 : cluster 0 osdmap e463: 8 total, 8 up, 8 in 2024-03-04T16:02:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:30 smithi151 bash[23756]: audit 2024-03-04T16:02:29.283552+0000 mon.c (mon.2) 685 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:30.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:30 smithi151 bash[23756]: audit 2024-03-04T16:02:29.284511+0000 mon.a (mon.0) 2770 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:30.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:30 smithi151 bash[23756]: audit 2024-03-04T16:02:29.395251+0000 mon.c (mon.2) 686 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:02:31.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:31 smithi117 bash[21480]: audit 2024-03-04T16:02:30.251465+0000 mon.a (mon.0) 2771 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-04T16:02:31.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:31 smithi117 bash[21480]: cluster 2024-03-04T16:02:30.258606+0000 mon.a (mon.0) 2772 : cluster 0 osdmap e464: 8 total, 8 up, 8 in 2024-03-04T16:02:31.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:31 smithi117 bash[21480]: audit 2024-03-04T16:02:30.328714+0000 mon.c (mon.2) 687 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:31.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:31 smithi117 bash[21480]: audit 2024-03-04T16:02:30.329618+0000 mon.a (mon.0) 2773 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:31.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:31 smithi117 bash[21480]: audit 2024-03-04T16:02:30.331004+0000 mon.c (mon.2) 688 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-81"}]: dispatch 2024-03-04T16:02:31.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:31 smithi117 bash[21480]: audit 2024-03-04T16:02:30.331712+0000 mon.a (mon.0) 2774 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-81"}]: dispatch 2024-03-04T16:02:31.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:31 smithi117 bash[28843]: audit 2024-03-04T16:02:30.251465+0000 mon.a (mon.0) 2771 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-04T16:02:31.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:31 smithi117 bash[28843]: cluster 2024-03-04T16:02:30.258606+0000 mon.a (mon.0) 2772 : cluster 0 osdmap e464: 8 total, 8 up, 8 in 2024-03-04T16:02:31.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:31 smithi117 bash[28843]: audit 2024-03-04T16:02:30.328714+0000 mon.c (mon.2) 687 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:31.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:31 smithi117 bash[28843]: audit 2024-03-04T16:02:30.329618+0000 mon.a (mon.0) 2773 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:31.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:31 smithi117 bash[28843]: audit 2024-03-04T16:02:30.331004+0000 mon.c (mon.2) 688 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-81"}]: dispatch 2024-03-04T16:02:31.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:31 smithi117 bash[28843]: audit 2024-03-04T16:02:30.331712+0000 mon.a (mon.0) 2774 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-81"}]: dispatch 2024-03-04T16:02:31.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:31 smithi151 bash[23756]: audit 2024-03-04T16:02:30.251465+0000 mon.a (mon.0) 2771 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-04T16:02:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:31 smithi151 bash[23756]: cluster 2024-03-04T16:02:30.258606+0000 mon.a (mon.0) 2772 : cluster 0 osdmap e464: 8 total, 8 up, 8 in 2024-03-04T16:02:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:31 smithi151 bash[23756]: audit 2024-03-04T16:02:30.328714+0000 mon.c (mon.2) 687 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:31 smithi151 bash[23756]: audit 2024-03-04T16:02:30.329618+0000 mon.a (mon.0) 2773 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:31 smithi151 bash[23756]: audit 2024-03-04T16:02:30.331004+0000 mon.c (mon.2) 688 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-81"}]: dispatch 2024-03-04T16:02:31.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:31 smithi151 bash[23756]: audit 2024-03-04T16:02:30.331712+0000 mon.a (mon.0) 2774 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-81"}]: dispatch 2024-03-04T16:02:32.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:32 smithi117 bash[21480]: cluster 2024-03-04T16:02:31.062002+0000 mgr.y (mgr.24428) 502 : cluster 0 pgmap v751: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:02:32.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:32 smithi117 bash[21480]: cluster 2024-03-04T16:02:31.275018+0000 mon.a (mon.0) 2775 : cluster 0 osdmap e465: 8 total, 8 up, 8 in 2024-03-04T16:02:32.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:32 smithi117 bash[28843]: cluster 2024-03-04T16:02:31.062002+0000 mgr.y (mgr.24428) 502 : cluster 0 pgmap v751: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:02:32.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:32 smithi117 bash[28843]: cluster 2024-03-04T16:02:31.275018+0000 mon.a (mon.0) 2775 : cluster 0 osdmap e465: 8 total, 8 up, 8 in 2024-03-04T16:02:32.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:32 smithi151 bash[23756]: cluster 2024-03-04T16:02:31.062002+0000 mgr.y (mgr.24428) 502 : cluster 0 pgmap v751: 228 pgs: 228 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:02:32.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:32 smithi151 bash[23756]: cluster 2024-03-04T16:02:31.275018+0000 mon.a (mon.0) 2775 : cluster 0 osdmap e465: 8 total, 8 up, 8 in 2024-03-04T16:02:33.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:33 smithi117 bash[21480]: cluster 2024-03-04T16:02:32.285845+0000 mon.a (mon.0) 2776 : cluster 0 osdmap e466: 8 total, 8 up, 8 in 2024-03-04T16:02:33.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:33 smithi117 bash[21480]: audit 2024-03-04T16:02:32.289454+0000 mon.c (mon.2) 689 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:33.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:33 smithi117 bash[21480]: audit 2024-03-04T16:02:32.290652+0000 mon.a (mon.0) 2777 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:33.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:33 smithi117 bash[28843]: cluster 2024-03-04T16:02:32.285845+0000 mon.a (mon.0) 2776 : cluster 0 osdmap e466: 8 total, 8 up, 8 in 2024-03-04T16:02:33.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:33 smithi117 bash[28843]: audit 2024-03-04T16:02:32.289454+0000 mon.c (mon.2) 689 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:33.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:33 smithi117 bash[28843]: audit 2024-03-04T16:02:32.290652+0000 mon.a (mon.0) 2777 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:33.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:33 smithi151 bash[23756]: cluster 2024-03-04T16:02:32.285845+0000 mon.a (mon.0) 2776 : cluster 0 osdmap e466: 8 total, 8 up, 8 in 2024-03-04T16:02:33.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:33 smithi151 bash[23756]: audit 2024-03-04T16:02:32.289454+0000 mon.c (mon.2) 689 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:33.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:33 smithi151 bash[23756]: audit 2024-03-04T16:02:32.290652+0000 mon.a (mon.0) 2777 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:34.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:34 smithi117 bash[21480]: cluster 2024-03-04T16:02:33.062802+0000 mgr.y (mgr.24428) 503 : cluster 0 pgmap v754: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-03-04T16:02:34.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:34 smithi117 bash[21480]: audit 2024-03-04T16:02:33.285812+0000 mon.a (mon.0) 2778 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:02:34.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:34 smithi117 bash[21480]: cluster 2024-03-04T16:02:33.296880+0000 mon.a (mon.0) 2779 : cluster 0 osdmap e467: 8 total, 8 up, 8 in 2024-03-04T16:02:34.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:34 smithi117 bash[21480]: audit 2024-03-04T16:02:33.309095+0000 mon.c (mon.2) 690 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:02:34.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:34 smithi117 bash[21480]: audit 2024-03-04T16:02:33.322824+0000 mon.c (mon.2) 691 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:34.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:34 smithi117 bash[21480]: audit 2024-03-04T16:02:33.324540+0000 mon.a (mon.0) 2780 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:34.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:34 smithi117 bash[21480]: cluster 2024-03-04T16:02:33.810100+0000 mon.a (mon.0) 2781 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:02:34.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:34 smithi117 bash[28843]: cluster 2024-03-04T16:02:33.062802+0000 mgr.y (mgr.24428) 503 : cluster 0 pgmap v754: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-03-04T16:02:34.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:34 smithi117 bash[28843]: audit 2024-03-04T16:02:33.285812+0000 mon.a (mon.0) 2778 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:02:34.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:34 smithi117 bash[28843]: cluster 2024-03-04T16:02:33.296880+0000 mon.a (mon.0) 2779 : cluster 0 osdmap e467: 8 total, 8 up, 8 in 2024-03-04T16:02:34.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:34 smithi117 bash[28843]: audit 2024-03-04T16:02:33.309095+0000 mon.c (mon.2) 690 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:02:34.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:34 smithi117 bash[28843]: audit 2024-03-04T16:02:33.322824+0000 mon.c (mon.2) 691 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:34.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:34 smithi117 bash[28843]: audit 2024-03-04T16:02:33.324540+0000 mon.a (mon.0) 2780 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:34.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:34 smithi117 bash[28843]: cluster 2024-03-04T16:02:33.810100+0000 mon.a (mon.0) 2781 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:02:34.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:34 smithi151 bash[23756]: cluster 2024-03-04T16:02:33.062802+0000 mgr.y (mgr.24428) 503 : cluster 0 pgmap v754: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-03-04T16:02:34.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:34 smithi151 bash[23756]: audit 2024-03-04T16:02:33.285812+0000 mon.a (mon.0) 2778 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:02:34.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:34 smithi151 bash[23756]: cluster 2024-03-04T16:02:33.296880+0000 mon.a (mon.0) 2779 : cluster 0 osdmap e467: 8 total, 8 up, 8 in 2024-03-04T16:02:34.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:34 smithi151 bash[23756]: audit 2024-03-04T16:02:33.309095+0000 mon.c (mon.2) 690 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:02:34.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:34 smithi151 bash[23756]: audit 2024-03-04T16:02:33.322824+0000 mon.c (mon.2) 691 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:34.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:34 smithi151 bash[23756]: audit 2024-03-04T16:02:33.324540+0000 mon.a (mon.0) 2780 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:34.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:34 smithi151 bash[23756]: cluster 2024-03-04T16:02:33.810100+0000 mon.a (mon.0) 2781 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:02:35.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:35 smithi117 bash[21480]: audit 2024-03-04T16:02:34.288649+0000 mon.a (mon.0) 2782 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:02:35.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:35 smithi117 bash[21480]: cluster 2024-03-04T16:02:34.295122+0000 mon.a (mon.0) 2783 : cluster 0 osdmap e468: 8 total, 8 up, 8 in 2024-03-04T16:02:35.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:35 smithi117 bash[21480]: audit 2024-03-04T16:02:34.303373+0000 mon.c (mon.2) 692 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:35.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:35 smithi117 bash[21480]: audit 2024-03-04T16:02:34.315020+0000 mon.a (mon.0) 2784 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:35.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:35 smithi117 bash[28843]: audit 2024-03-04T16:02:34.288649+0000 mon.a (mon.0) 2782 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:02:35.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:35 smithi117 bash[28843]: cluster 2024-03-04T16:02:34.295122+0000 mon.a (mon.0) 2783 : cluster 0 osdmap e468: 8 total, 8 up, 8 in 2024-03-04T16:02:35.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:35 smithi117 bash[28843]: audit 2024-03-04T16:02:34.303373+0000 mon.c (mon.2) 692 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:35.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:35 smithi117 bash[28843]: audit 2024-03-04T16:02:34.315020+0000 mon.a (mon.0) 2784 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:35.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:35 smithi151 bash[23756]: audit 2024-03-04T16:02:34.288649+0000 mon.a (mon.0) 2782 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:02:35.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:35 smithi151 bash[23756]: cluster 2024-03-04T16:02:34.295122+0000 mon.a (mon.0) 2783 : cluster 0 osdmap e468: 8 total, 8 up, 8 in 2024-03-04T16:02:35.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:35 smithi151 bash[23756]: audit 2024-03-04T16:02:34.303373+0000 mon.c (mon.2) 692 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:35.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:35 smithi151 bash[23756]: audit 2024-03-04T16:02:34.315020+0000 mon.a (mon.0) 2784 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:36.565 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:02:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:02:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:02:36.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:36 smithi117 bash[21480]: cluster 2024-03-04T16:02:35.063661+0000 mgr.y (mgr.24428) 504 : cluster 0 pgmap v757: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-03-04T16:02:36.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:36 smithi117 bash[21480]: audit 2024-03-04T16:02:35.320316+0000 mon.a (mon.0) 2785 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T16:02:36.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:36 smithi117 bash[21480]: cluster 2024-03-04T16:02:35.331055+0000 mon.a (mon.0) 2786 : cluster 0 osdmap e469: 8 total, 8 up, 8 in 2024-03-04T16:02:36.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:36 smithi117 bash[21480]: audit 2024-03-04T16:02:35.332976+0000 mon.c (mon.2) 693 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:36.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:36 smithi117 bash[21480]: audit 2024-03-04T16:02:35.333915+0000 mon.a (mon.0) 2787 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:36.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:36 smithi117 bash[28843]: cluster 2024-03-04T16:02:35.063661+0000 mgr.y (mgr.24428) 504 : cluster 0 pgmap v757: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-03-04T16:02:36.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:36 smithi117 bash[28843]: audit 2024-03-04T16:02:35.320316+0000 mon.a (mon.0) 2785 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T16:02:36.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:36 smithi117 bash[28843]: cluster 2024-03-04T16:02:35.331055+0000 mon.a (mon.0) 2786 : cluster 0 osdmap e469: 8 total, 8 up, 8 in 2024-03-04T16:02:36.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:36 smithi117 bash[28843]: audit 2024-03-04T16:02:35.332976+0000 mon.c (mon.2) 693 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:36.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:36 smithi117 bash[28843]: audit 2024-03-04T16:02:35.333915+0000 mon.a (mon.0) 2787 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:36.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:36 smithi151 bash[23756]: cluster 2024-03-04T16:02:35.063661+0000 mgr.y (mgr.24428) 504 : cluster 0 pgmap v757: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 620 MiB used, 715 GiB / 715 GiB avail; 10 KiB/s rd, 12 KiB/s wr, 39 op/s 2024-03-04T16:02:36.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:36 smithi151 bash[23756]: audit 2024-03-04T16:02:35.320316+0000 mon.a (mon.0) 2785 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T16:02:36.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:36 smithi151 bash[23756]: cluster 2024-03-04T16:02:35.331055+0000 mon.a (mon.0) 2786 : cluster 0 osdmap e469: 8 total, 8 up, 8 in 2024-03-04T16:02:36.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:36 smithi151 bash[23756]: audit 2024-03-04T16:02:35.332976+0000 mon.c (mon.2) 693 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:36.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:36 smithi151 bash[23756]: audit 2024-03-04T16:02:35.333915+0000 mon.a (mon.0) 2787 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:37.344 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:02:36 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:02:37.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:37 smithi151 bash[23756]: audit 2024-03-04T16:02:36.323001+0000 mon.a (mon.0) 2788 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-04T16:02:37.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:37 smithi151 bash[23756]: cluster 2024-03-04T16:02:36.330135+0000 mon.a (mon.0) 2789 : cluster 0 osdmap e470: 8 total, 8 up, 8 in 2024-03-04T16:02:37.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:37 smithi151 bash[23756]: audit 2024-03-04T16:02:36.332437+0000 mon.c (mon.2) 694 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:37.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:37 smithi151 bash[23756]: audit 2024-03-04T16:02:36.344851+0000 mon.a (mon.0) 2790 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:37.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:37 smithi151 bash[23756]: audit 2024-03-04T16:02:36.937031+0000 mgr.y (mgr.24428) 505 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:02:37.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:37 smithi151 bash[23756]: cluster 2024-03-04T16:02:37.065899+0000 mgr.y (mgr.24428) 506 : cluster 0 pgmap v760: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:37.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:37 smithi117 bash[21480]: audit 2024-03-04T16:02:36.323001+0000 mon.a (mon.0) 2788 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-04T16:02:37.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:37 smithi117 bash[21480]: cluster 2024-03-04T16:02:36.330135+0000 mon.a (mon.0) 2789 : cluster 0 osdmap e470: 8 total, 8 up, 8 in 2024-03-04T16:02:37.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:37 smithi117 bash[21480]: audit 2024-03-04T16:02:36.332437+0000 mon.c (mon.2) 694 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:37.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:37 smithi117 bash[21480]: audit 2024-03-04T16:02:36.344851+0000 mon.a (mon.0) 2790 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:37.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:37 smithi117 bash[21480]: audit 2024-03-04T16:02:36.937031+0000 mgr.y (mgr.24428) 505 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:02:37.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:37 smithi117 bash[21480]: cluster 2024-03-04T16:02:37.065899+0000 mgr.y (mgr.24428) 506 : cluster 0 pgmap v760: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:37.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:37 smithi117 bash[28843]: audit 2024-03-04T16:02:36.323001+0000 mon.a (mon.0) 2788 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-04T16:02:37.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:37 smithi117 bash[28843]: cluster 2024-03-04T16:02:36.330135+0000 mon.a (mon.0) 2789 : cluster 0 osdmap e470: 8 total, 8 up, 8 in 2024-03-04T16:02:37.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:37 smithi117 bash[28843]: audit 2024-03-04T16:02:36.332437+0000 mon.c (mon.2) 694 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:37.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:37 smithi117 bash[28843]: audit 2024-03-04T16:02:36.344851+0000 mon.a (mon.0) 2790 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:37.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:37 smithi117 bash[28843]: audit 2024-03-04T16:02:36.937031+0000 mgr.y (mgr.24428) 505 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:02:37.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:37 smithi117 bash[28843]: cluster 2024-03-04T16:02:37.065899+0000 mgr.y (mgr.24428) 506 : cluster 0 pgmap v760: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:38.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:38 smithi151 bash[23756]: audit 2024-03-04T16:02:37.336696+0000 mon.a (mon.0) 2791 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-04T16:02:38.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:38 smithi151 bash[23756]: cluster 2024-03-04T16:02:37.343926+0000 mon.a (mon.0) 2792 : cluster 0 osdmap e471: 8 total, 8 up, 8 in 2024-03-04T16:02:38.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:38 smithi117 bash[21480]: audit 2024-03-04T16:02:37.336696+0000 mon.a (mon.0) 2791 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-04T16:02:38.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:38 smithi117 bash[21480]: cluster 2024-03-04T16:02:37.343926+0000 mon.a (mon.0) 2792 : cluster 0 osdmap e471: 8 total, 8 up, 8 in 2024-03-04T16:02:38.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:38 smithi117 bash[28843]: audit 2024-03-04T16:02:37.336696+0000 mon.a (mon.0) 2791 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-04T16:02:38.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:38 smithi117 bash[28843]: cluster 2024-03-04T16:02:37.343926+0000 mon.a (mon.0) 2792 : cluster 0 osdmap e471: 8 total, 8 up, 8 in 2024-03-04T16:02:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:39 smithi151 bash[23756]: cluster 2024-03-04T16:02:38.364804+0000 mon.a (mon.0) 2793 : cluster 0 osdmap e472: 8 total, 8 up, 8 in 2024-03-04T16:02:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:39 smithi151 bash[23756]: cluster 2024-03-04T16:02:38.811459+0000 mon.a (mon.0) 2794 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:02:39.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:39 smithi151 bash[23756]: cluster 2024-03-04T16:02:39.066772+0000 mgr.y (mgr.24428) 507 : cluster 0 pgmap v763: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:39.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:39 smithi117 bash[21480]: cluster 2024-03-04T16:02:38.364804+0000 mon.a (mon.0) 2793 : cluster 0 osdmap e472: 8 total, 8 up, 8 in 2024-03-04T16:02:39.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:39 smithi117 bash[21480]: cluster 2024-03-04T16:02:38.811459+0000 mon.a (mon.0) 2794 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:02:39.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:39 smithi117 bash[21480]: cluster 2024-03-04T16:02:39.066772+0000 mgr.y (mgr.24428) 507 : cluster 0 pgmap v763: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:39.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:39 smithi117 bash[28843]: cluster 2024-03-04T16:02:38.364804+0000 mon.a (mon.0) 2793 : cluster 0 osdmap e472: 8 total, 8 up, 8 in 2024-03-04T16:02:39.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:39 smithi117 bash[28843]: cluster 2024-03-04T16:02:38.811459+0000 mon.a (mon.0) 2794 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:02:39.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:39 smithi117 bash[28843]: cluster 2024-03-04T16:02:39.066772+0000 mgr.y (mgr.24428) 507 : cluster 0 pgmap v763: 228 pgs: 228 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:40.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:40 smithi151 bash[23756]: cluster 2024-03-04T16:02:39.376891+0000 mon.a (mon.0) 2795 : cluster 0 osdmap e473: 8 total, 8 up, 8 in 2024-03-04T16:02:40.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:40 smithi151 bash[23756]: audit 2024-03-04T16:02:39.457757+0000 mon.c (mon.2) 695 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:40.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:40 smithi151 bash[23756]: audit 2024-03-04T16:02:39.458541+0000 mon.a (mon.0) 2796 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:40.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:40 smithi151 bash[23756]: audit 2024-03-04T16:02:39.459706+0000 mon.c (mon.2) 696 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-83"}]: dispatch 2024-03-04T16:02:40.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:40 smithi151 bash[23756]: audit 2024-03-04T16:02:39.460352+0000 mon.a (mon.0) 2797 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-83"}]: dispatch 2024-03-04T16:02:40.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:40 smithi117 bash[28843]: cluster 2024-03-04T16:02:39.376891+0000 mon.a (mon.0) 2795 : cluster 0 osdmap e473: 8 total, 8 up, 8 in 2024-03-04T16:02:40.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:40 smithi117 bash[28843]: audit 2024-03-04T16:02:39.457757+0000 mon.c (mon.2) 695 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:40.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:40 smithi117 bash[28843]: audit 2024-03-04T16:02:39.458541+0000 mon.a (mon.0) 2796 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:40.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:40 smithi117 bash[28843]: audit 2024-03-04T16:02:39.459706+0000 mon.c (mon.2) 696 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-83"}]: dispatch 2024-03-04T16:02:40.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:40 smithi117 bash[28843]: audit 2024-03-04T16:02:39.460352+0000 mon.a (mon.0) 2797 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-83"}]: dispatch 2024-03-04T16:02:40.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:40 smithi117 bash[21480]: cluster 2024-03-04T16:02:39.376891+0000 mon.a (mon.0) 2795 : cluster 0 osdmap e473: 8 total, 8 up, 8 in 2024-03-04T16:02:40.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:40 smithi117 bash[21480]: audit 2024-03-04T16:02:39.457757+0000 mon.c (mon.2) 695 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:40.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:40 smithi117 bash[21480]: audit 2024-03-04T16:02:39.458541+0000 mon.a (mon.0) 2796 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:40.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:40 smithi117 bash[21480]: audit 2024-03-04T16:02:39.459706+0000 mon.c (mon.2) 696 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-83"}]: dispatch 2024-03-04T16:02:40.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:40 smithi117 bash[21480]: audit 2024-03-04T16:02:39.460352+0000 mon.a (mon.0) 2797 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-83"}]: dispatch 2024-03-04T16:02:41.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:41 smithi151 bash[23756]: cluster 2024-03-04T16:02:40.376282+0000 mon.a (mon.0) 2798 : cluster 0 osdmap e474: 8 total, 8 up, 8 in 2024-03-04T16:02:41.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:41 smithi151 bash[23756]: cluster 2024-03-04T16:02:41.067495+0000 mgr.y (mgr.24428) 508 : cluster 0 pgmap v766: 196 pgs: 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:02:41.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:41 smithi117 bash[21480]: cluster 2024-03-04T16:02:40.376282+0000 mon.a (mon.0) 2798 : cluster 0 osdmap e474: 8 total, 8 up, 8 in 2024-03-04T16:02:41.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:41 smithi117 bash[21480]: cluster 2024-03-04T16:02:41.067495+0000 mgr.y (mgr.24428) 508 : cluster 0 pgmap v766: 196 pgs: 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:02:41.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:41 smithi117 bash[28843]: cluster 2024-03-04T16:02:40.376282+0000 mon.a (mon.0) 2798 : cluster 0 osdmap e474: 8 total, 8 up, 8 in 2024-03-04T16:02:41.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:41 smithi117 bash[28843]: cluster 2024-03-04T16:02:41.067495+0000 mgr.y (mgr.24428) 508 : cluster 0 pgmap v766: 196 pgs: 196 active+clean; 8.4 MiB data, 625 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:02:42.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:42 smithi151 bash[23756]: cluster 2024-03-04T16:02:41.411963+0000 mon.a (mon.0) 2799 : cluster 0 osdmap e475: 8 total, 8 up, 8 in 2024-03-04T16:02:42.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:42 smithi151 bash[23756]: audit 2024-03-04T16:02:41.416150+0000 mon.c (mon.2) 697 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:42.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:42 smithi151 bash[23756]: audit 2024-03-04T16:02:41.420028+0000 mon.a (mon.0) 2800 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:42.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:42 smithi117 bash[21480]: cluster 2024-03-04T16:02:41.411963+0000 mon.a (mon.0) 2799 : cluster 0 osdmap e475: 8 total, 8 up, 8 in 2024-03-04T16:02:42.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:42 smithi117 bash[21480]: audit 2024-03-04T16:02:41.416150+0000 mon.c (mon.2) 697 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:42.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:42 smithi117 bash[21480]: audit 2024-03-04T16:02:41.420028+0000 mon.a (mon.0) 2800 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:42.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:42 smithi117 bash[28843]: cluster 2024-03-04T16:02:41.411963+0000 mon.a (mon.0) 2799 : cluster 0 osdmap e475: 8 total, 8 up, 8 in 2024-03-04T16:02:42.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:42 smithi117 bash[28843]: audit 2024-03-04T16:02:41.416150+0000 mon.c (mon.2) 697 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:42.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:42 smithi117 bash[28843]: audit 2024-03-04T16:02:41.420028+0000 mon.a (mon.0) 2800 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:43 smithi151 bash[23756]: audit 2024-03-04T16:02:42.412600+0000 mon.a (mon.0) 2801 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:02:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:43 smithi151 bash[23756]: cluster 2024-03-04T16:02:42.424336+0000 mon.a (mon.0) 2802 : cluster 0 osdmap e476: 8 total, 8 up, 8 in 2024-03-04T16:02:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:43 smithi151 bash[23756]: audit 2024-03-04T16:02:42.428268+0000 mon.c (mon.2) 698 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:02:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:43 smithi151 bash[23756]: audit 2024-03-04T16:02:42.446574+0000 mon.c (mon.2) 699 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:43 smithi151 bash[23756]: audit 2024-03-04T16:02:42.448284+0000 mon.a (mon.0) 2803 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:43.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:43 smithi151 bash[23756]: cluster 2024-03-04T16:02:43.068351+0000 mgr.y (mgr.24428) 509 : cluster 0 pgmap v769: 228 pgs: 29 creating+peering, 3 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-04T16:02:43.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:43 smithi117 bash[21480]: audit 2024-03-04T16:02:42.412600+0000 mon.a (mon.0) 2801 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:02:43.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:43 smithi117 bash[21480]: cluster 2024-03-04T16:02:42.424336+0000 mon.a (mon.0) 2802 : cluster 0 osdmap e476: 8 total, 8 up, 8 in 2024-03-04T16:02:43.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:43 smithi117 bash[21480]: audit 2024-03-04T16:02:42.428268+0000 mon.c (mon.2) 698 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:02:43.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:43 smithi117 bash[21480]: audit 2024-03-04T16:02:42.446574+0000 mon.c (mon.2) 699 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:43.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:43 smithi117 bash[21480]: audit 2024-03-04T16:02:42.448284+0000 mon.a (mon.0) 2803 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:43.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:43 smithi117 bash[21480]: cluster 2024-03-04T16:02:43.068351+0000 mgr.y (mgr.24428) 509 : cluster 0 pgmap v769: 228 pgs: 29 creating+peering, 3 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-04T16:02:43.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:43 smithi117 bash[28843]: audit 2024-03-04T16:02:42.412600+0000 mon.a (mon.0) 2801 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:02:43.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:43 smithi117 bash[28843]: cluster 2024-03-04T16:02:42.424336+0000 mon.a (mon.0) 2802 : cluster 0 osdmap e476: 8 total, 8 up, 8 in 2024-03-04T16:02:43.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:43 smithi117 bash[28843]: audit 2024-03-04T16:02:42.428268+0000 mon.c (mon.2) 698 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:02:43.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:43 smithi117 bash[28843]: audit 2024-03-04T16:02:42.446574+0000 mon.c (mon.2) 699 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:43.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:43 smithi117 bash[28843]: audit 2024-03-04T16:02:42.448284+0000 mon.a (mon.0) 2803 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:43.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:43 smithi117 bash[28843]: cluster 2024-03-04T16:02:43.068351+0000 mgr.y (mgr.24428) 509 : cluster 0 pgmap v769: 228 pgs: 29 creating+peering, 3 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-04T16:02:44.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:44 smithi151 bash[23756]: audit 2024-03-04T16:02:43.423372+0000 mon.a (mon.0) 2804 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:02:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:44 smithi151 bash[23756]: cluster 2024-03-04T16:02:43.436205+0000 mon.a (mon.0) 2805 : cluster 0 osdmap e477: 8 total, 8 up, 8 in 2024-03-04T16:02:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:44 smithi151 bash[23756]: audit 2024-03-04T16:02:43.441313+0000 mon.c (mon.2) 700 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:44 smithi151 bash[23756]: audit 2024-03-04T16:02:43.443544+0000 mon.a (mon.0) 2806 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:44 smithi151 bash[23756]: audit 2024-03-04T16:02:43.880672+0000 mon.c (mon.2) 701 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:02:44.815 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:44 smithi117 bash[28843]: audit 2024-03-04T16:02:43.423372+0000 mon.a (mon.0) 2804 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:02:44.815 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:44 smithi117 bash[28843]: cluster 2024-03-04T16:02:43.436205+0000 mon.a (mon.0) 2805 : cluster 0 osdmap e477: 8 total, 8 up, 8 in 2024-03-04T16:02:44.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:44 smithi117 bash[28843]: audit 2024-03-04T16:02:43.441313+0000 mon.c (mon.2) 700 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:44.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:44 smithi117 bash[28843]: audit 2024-03-04T16:02:43.443544+0000 mon.a (mon.0) 2806 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:44.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:44 smithi117 bash[28843]: audit 2024-03-04T16:02:43.880672+0000 mon.c (mon.2) 701 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:02:44.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:44 smithi117 bash[21480]: audit 2024-03-04T16:02:43.423372+0000 mon.a (mon.0) 2804 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:02:44.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:44 smithi117 bash[21480]: cluster 2024-03-04T16:02:43.436205+0000 mon.a (mon.0) 2805 : cluster 0 osdmap e477: 8 total, 8 up, 8 in 2024-03-04T16:02:44.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:44 smithi117 bash[21480]: audit 2024-03-04T16:02:43.441313+0000 mon.c (mon.2) 700 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:44.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:44 smithi117 bash[21480]: audit 2024-03-04T16:02:43.443544+0000 mon.a (mon.0) 2806 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:44.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:44 smithi117 bash[21480]: audit 2024-03-04T16:02:43.880672+0000 mon.c (mon.2) 701 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:02:45.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:45 smithi117 bash[21480]: audit 2024-03-04T16:02:44.434763+0000 mon.c (mon.2) 702 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:02:45.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:45 smithi117 bash[21480]: audit 2024-03-04T16:02:44.435391+0000 mon.a (mon.0) 2807 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T16:02:45.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:45 smithi117 bash[21480]: cluster 2024-03-04T16:02:44.442241+0000 mon.a (mon.0) 2808 : cluster 0 osdmap e478: 8 total, 8 up, 8 in 2024-03-04T16:02:45.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:45 smithi117 bash[21480]: audit 2024-03-04T16:02:44.445101+0000 mon.c (mon.2) 703 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:45.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:45 smithi117 bash[21480]: audit 2024-03-04T16:02:44.446967+0000 mon.a (mon.0) 2809 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:45.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:45 smithi117 bash[21480]: cluster 2024-03-04T16:02:45.069236+0000 mgr.y (mgr.24428) 510 : cluster 0 pgmap v772: 228 pgs: 29 creating+peering, 3 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-04T16:02:45.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:45 smithi117 bash[28843]: audit 2024-03-04T16:02:44.434763+0000 mon.c (mon.2) 702 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:02:45.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:45 smithi117 bash[28843]: audit 2024-03-04T16:02:44.435391+0000 mon.a (mon.0) 2807 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T16:02:45.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:45 smithi117 bash[28843]: cluster 2024-03-04T16:02:44.442241+0000 mon.a (mon.0) 2808 : cluster 0 osdmap e478: 8 total, 8 up, 8 in 2024-03-04T16:02:45.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:45 smithi117 bash[28843]: audit 2024-03-04T16:02:44.445101+0000 mon.c (mon.2) 703 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:45.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:45 smithi117 bash[28843]: audit 2024-03-04T16:02:44.446967+0000 mon.a (mon.0) 2809 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:45.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:45 smithi117 bash[28843]: cluster 2024-03-04T16:02:45.069236+0000 mgr.y (mgr.24428) 510 : cluster 0 pgmap v772: 228 pgs: 29 creating+peering, 3 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-04T16:02:45.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:45 smithi151 bash[23756]: audit 2024-03-04T16:02:44.434763+0000 mon.c (mon.2) 702 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:02:45.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:45 smithi151 bash[23756]: audit 2024-03-04T16:02:44.435391+0000 mon.a (mon.0) 2807 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_tier","val": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T16:02:45.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:45 smithi151 bash[23756]: cluster 2024-03-04T16:02:44.442241+0000 mon.a (mon.0) 2808 : cluster 0 osdmap e478: 8 total, 8 up, 8 in 2024-03-04T16:02:45.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:45 smithi151 bash[23756]: audit 2024-03-04T16:02:44.445101+0000 mon.c (mon.2) 703 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:45.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:45 smithi151 bash[23756]: audit 2024-03-04T16:02:44.446967+0000 mon.a (mon.0) 2809 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:45.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:45 smithi151 bash[23756]: cluster 2024-03-04T16:02:45.069236+0000 mgr.y (mgr.24428) 510 : cluster 0 pgmap v772: 228 pgs: 29 creating+peering, 3 unknown, 196 active+clean; 8.4 MiB data, 630 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-03-04T16:02:46.455 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:02:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:02:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:02:46.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:46 smithi117 bash[28843]: audit 2024-03-04T16:02:45.445224+0000 mon.a (mon.0) 2810 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-04T16:02:46.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:46 smithi117 bash[28843]: cluster 2024-03-04T16:02:45.454385+0000 mon.a (mon.0) 2811 : cluster 0 osdmap e479: 8 total, 8 up, 8 in 2024-03-04T16:02:46.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:46 smithi117 bash[28843]: audit 2024-03-04T16:02:45.457699+0000 mon.c (mon.2) 704 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:46.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:46 smithi117 bash[28843]: audit 2024-03-04T16:02:45.458922+0000 mon.a (mon.0) 2812 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:46.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:46 smithi117 bash[21480]: audit 2024-03-04T16:02:45.445224+0000 mon.a (mon.0) 2810 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-04T16:02:46.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:46 smithi117 bash[21480]: cluster 2024-03-04T16:02:45.454385+0000 mon.a (mon.0) 2811 : cluster 0 osdmap e479: 8 total, 8 up, 8 in 2024-03-04T16:02:46.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:46 smithi117 bash[21480]: audit 2024-03-04T16:02:45.457699+0000 mon.c (mon.2) 704 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:46.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:46 smithi117 bash[21480]: audit 2024-03-04T16:02:45.458922+0000 mon.a (mon.0) 2812 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:46.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:46 smithi151 bash[23756]: audit 2024-03-04T16:02:45.445224+0000 mon.a (mon.0) 2810 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-04T16:02:46.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:46 smithi151 bash[23756]: cluster 2024-03-04T16:02:45.454385+0000 mon.a (mon.0) 2811 : cluster 0 osdmap e479: 8 total, 8 up, 8 in 2024-03-04T16:02:46.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:46 smithi151 bash[23756]: audit 2024-03-04T16:02:45.457699+0000 mon.c (mon.2) 704 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:46.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:46 smithi151 bash[23756]: audit 2024-03-04T16:02:45.458922+0000 mon.a (mon.0) 2812 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:47.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:02:46 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:02:47.815 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:47 smithi117 bash[28843]: audit 2024-03-04T16:02:46.447860+0000 mon.a (mon.0) 2813 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-04T16:02:47.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:47 smithi117 bash[28843]: cluster 2024-03-04T16:02:46.455006+0000 mon.a (mon.0) 2814 : cluster 0 osdmap e480: 8 total, 8 up, 8 in 2024-03-04T16:02:47.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:47 smithi117 bash[28843]: audit 2024-03-04T16:02:46.945135+0000 mgr.y (mgr.24428) 511 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:02:47.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:47 smithi117 bash[28843]: cluster 2024-03-04T16:02:47.070007+0000 mgr.y (mgr.24428) 512 : cluster 0 pgmap v775: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:47.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:47 smithi117 bash[21480]: audit 2024-03-04T16:02:46.447860+0000 mon.a (mon.0) 2813 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-04T16:02:47.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:47 smithi117 bash[21480]: cluster 2024-03-04T16:02:46.455006+0000 mon.a (mon.0) 2814 : cluster 0 osdmap e480: 8 total, 8 up, 8 in 2024-03-04T16:02:47.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:47 smithi117 bash[21480]: audit 2024-03-04T16:02:46.945135+0000 mgr.y (mgr.24428) 511 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:02:47.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:47 smithi117 bash[21480]: cluster 2024-03-04T16:02:47.070007+0000 mgr.y (mgr.24428) 512 : cluster 0 pgmap v775: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:47.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:47 smithi151 bash[23756]: audit 2024-03-04T16:02:46.447860+0000 mon.a (mon.0) 2813 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-04T16:02:47.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:47 smithi151 bash[23756]: cluster 2024-03-04T16:02:46.455006+0000 mon.a (mon.0) 2814 : cluster 0 osdmap e480: 8 total, 8 up, 8 in 2024-03-04T16:02:47.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:47 smithi151 bash[23756]: audit 2024-03-04T16:02:46.945135+0000 mgr.y (mgr.24428) 511 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:02:47.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:47 smithi151 bash[23756]: cluster 2024-03-04T16:02:47.070007+0000 mgr.y (mgr.24428) 512 : cluster 0 pgmap v775: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:48.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:48 smithi117 bash[21480]: cluster 2024-03-04T16:02:47.478524+0000 mon.a (mon.0) 2815 : cluster 0 osdmap e481: 8 total, 8 up, 8 in 2024-03-04T16:02:48.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:48 smithi117 bash[28843]: cluster 2024-03-04T16:02:47.478524+0000 mon.a (mon.0) 2815 : cluster 0 osdmap e481: 8 total, 8 up, 8 in 2024-03-04T16:02:48.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:48 smithi151 bash[23756]: cluster 2024-03-04T16:02:47.478524+0000 mon.a (mon.0) 2815 : cluster 0 osdmap e481: 8 total, 8 up, 8 in 2024-03-04T16:02:49.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:49 smithi117 bash[21480]: cluster 2024-03-04T16:02:48.481390+0000 mon.a (mon.0) 2816 : cluster 0 osdmap e482: 8 total, 8 up, 8 in 2024-03-04T16:02:49.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:49 smithi117 bash[21480]: audit 2024-03-04T16:02:48.564974+0000 mon.c (mon.2) 705 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:49.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:49 smithi117 bash[21480]: audit 2024-03-04T16:02:48.565711+0000 mon.a (mon.0) 2817 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:49.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:49 smithi117 bash[21480]: audit 2024-03-04T16:02:48.566830+0000 mon.c (mon.2) 706 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-85"}]: dispatch 2024-03-04T16:02:49.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:49 smithi117 bash[21480]: audit 2024-03-04T16:02:48.567437+0000 mon.a (mon.0) 2818 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-85"}]: dispatch 2024-03-04T16:02:49.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:49 smithi117 bash[21480]: cluster 2024-03-04T16:02:49.070590+0000 mgr.y (mgr.24428) 513 : cluster 0 pgmap v778: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:49.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:49 smithi117 bash[28843]: cluster 2024-03-04T16:02:48.481390+0000 mon.a (mon.0) 2816 : cluster 0 osdmap e482: 8 total, 8 up, 8 in 2024-03-04T16:02:49.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:49 smithi117 bash[28843]: audit 2024-03-04T16:02:48.564974+0000 mon.c (mon.2) 705 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:49.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:49 smithi117 bash[28843]: audit 2024-03-04T16:02:48.565711+0000 mon.a (mon.0) 2817 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:49.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:49 smithi117 bash[28843]: audit 2024-03-04T16:02:48.566830+0000 mon.c (mon.2) 706 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-85"}]: dispatch 2024-03-04T16:02:49.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:49 smithi117 bash[28843]: audit 2024-03-04T16:02:48.567437+0000 mon.a (mon.0) 2818 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-85"}]: dispatch 2024-03-04T16:02:49.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:49 smithi117 bash[28843]: cluster 2024-03-04T16:02:49.070590+0000 mgr.y (mgr.24428) 513 : cluster 0 pgmap v778: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:49.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:49 smithi151 bash[23756]: cluster 2024-03-04T16:02:48.481390+0000 mon.a (mon.0) 2816 : cluster 0 osdmap e482: 8 total, 8 up, 8 in 2024-03-04T16:02:49.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:49 smithi151 bash[23756]: audit 2024-03-04T16:02:48.564974+0000 mon.c (mon.2) 705 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:49.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:49 smithi151 bash[23756]: audit 2024-03-04T16:02:48.565711+0000 mon.a (mon.0) 2817 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:49.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:49 smithi151 bash[23756]: audit 2024-03-04T16:02:48.566830+0000 mon.c (mon.2) 706 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-85"}]: dispatch 2024-03-04T16:02:49.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:49 smithi151 bash[23756]: audit 2024-03-04T16:02:48.567437+0000 mon.a (mon.0) 2818 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-85"}]: dispatch 2024-03-04T16:02:49.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:49 smithi151 bash[23756]: cluster 2024-03-04T16:02:49.070590+0000 mgr.y (mgr.24428) 513 : cluster 0 pgmap v778: 228 pgs: 228 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:50.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:50 smithi117 bash[21480]: cluster 2024-03-04T16:02:49.493476+0000 mon.a (mon.0) 2819 : cluster 0 osdmap e483: 8 total, 8 up, 8 in 2024-03-04T16:02:50.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:50 smithi117 bash[21480]: audit 2024-03-04T16:02:49.925719+0000 mon.a (mon.0) 2820 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:02:50.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:50 smithi117 bash[21480]: audit 2024-03-04T16:02:49.938750+0000 mon.a (mon.0) 2821 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:02:50.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:50 smithi117 bash[21480]: audit 2024-03-04T16:02:49.984543+0000 mon.a (mon.0) 2822 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:02:50.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:50 smithi117 bash[21480]: audit 2024-03-04T16:02:49.994766+0000 mon.a (mon.0) 2823 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:02:50.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:50 smithi117 bash[28843]: cluster 2024-03-04T16:02:49.493476+0000 mon.a (mon.0) 2819 : cluster 0 osdmap e483: 8 total, 8 up, 8 in 2024-03-04T16:02:50.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:50 smithi117 bash[28843]: audit 2024-03-04T16:02:49.925719+0000 mon.a (mon.0) 2820 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:02:50.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:50 smithi117 bash[28843]: audit 2024-03-04T16:02:49.938750+0000 mon.a (mon.0) 2821 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:02:50.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:50 smithi117 bash[28843]: audit 2024-03-04T16:02:49.984543+0000 mon.a (mon.0) 2822 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:02:50.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:50 smithi117 bash[28843]: audit 2024-03-04T16:02:49.994766+0000 mon.a (mon.0) 2823 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:02:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:50 smithi151 bash[23756]: cluster 2024-03-04T16:02:49.493476+0000 mon.a (mon.0) 2819 : cluster 0 osdmap e483: 8 total, 8 up, 8 in 2024-03-04T16:02:51.399 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:50 smithi151 bash[23756]: audit 2024-03-04T16:02:49.925719+0000 mon.a (mon.0) 2820 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:02:51.400 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:50 smithi151 bash[23756]: audit 2024-03-04T16:02:49.938750+0000 mon.a (mon.0) 2821 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:02:51.400 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:50 smithi151 bash[23756]: audit 2024-03-04T16:02:49.984543+0000 mon.a (mon.0) 2822 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:02:51.400 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:50 smithi151 bash[23756]: audit 2024-03-04T16:02:49.994766+0000 mon.a (mon.0) 2823 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:02:51.549 INFO:tasks.workunit.client.0.smithi117.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (18240 ms) 2024-03-04T16:02:51.550 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-03-04T16:02:51.550 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (23158 ms) 2024-03-04T16:02:51.550 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-03-04T16:02:51.550 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3025 ms) 2024-03-04T16:02:51.550 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-03-04T16:02:51.550 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:02:51.550 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3060 ms) 2024-03-04T16:02:51.550 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-03-04T16:02:51.550 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3034 ms) 2024-03-04T16:02:51.550 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-03-04T16:02:51.550 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:02:51.550 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3043 ms) 2024-03-04T16:02:51.551 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-03-04T16:02:51.551 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:02:51.551 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4082 ms) 2024-03-04T16:02:51.551 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-03-04T16:02:51.551 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:02:51.551 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (39747 ms) 2024-03-04T16:02:51.551 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-03-04T16:02:51.551 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:02:51.551 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (17154 ms) 2024-03-04T16:02:51.551 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-03-04T16:02:51.551 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:02:51.551 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4059 ms) 2024-03-04T16:02:51.551 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-03-04T16:02:51.551 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:02:51.552 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3040 ms) 2024-03-04T16:02:51.552 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-03-04T16:02:51.552 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:02:51.552 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (24818 ms) 2024-03-04T16:02:51.552 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-03-04T16:02:51.552 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:02:51.552 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (15091 ms) 2024-03-04T16:02:51.552 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-03-04T16:02:51.552 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:02:51.552 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5073 ms) 2024-03-04T16:02:51.552 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-03-04T16:02:51.552 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:02:51.552 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4073 ms) 2024-03-04T16:02:51.552 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-03-04T16:02:51.553 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:02:51.553 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: waiting for scrubs... 2024-03-04T16:02:51.553 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: done waiting 2024-03-04T16:02:51.553 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (24791 ms) 2024-03-04T16:02:51.553 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-03-04T16:02:51.553 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:02:51.553 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10108 ms) 2024-03-04T16:02:51.553 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-03-04T16:02:51.554 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:02:51.554 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9103 ms) 2024-03-04T16:02:51.554 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-03-04T16:02:51.555 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:02:51.555 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9119 ms) 2024-03-04T16:02:51.555 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-03-04T16:02:51.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:51 smithi117 bash[21480]: cluster 2024-03-04T16:02:50.503962+0000 mon.a (mon.0) 2824 : cluster 0 osdmap e484: 8 total, 8 up, 8 in 2024-03-04T16:02:51.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:51 smithi117 bash[21480]: audit 2024-03-04T16:02:50.505576+0000 mon.c (mon.2) 707 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:51.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:51 smithi117 bash[21480]: audit 2024-03-04T16:02:50.506440+0000 mon.a (mon.0) 2825 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:51.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:51 smithi117 bash[21480]: audit 2024-03-04T16:02:50.679990+0000 mon.c (mon.2) 708 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:02:51.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:51 smithi117 bash[21480]: audit 2024-03-04T16:02:50.681767+0000 mon.c (mon.2) 709 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:02:51.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:51 smithi117 bash[21480]: audit 2024-03-04T16:02:50.692195+0000 mon.a (mon.0) 2826 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:02:51.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:51 smithi117 bash[21480]: cluster 2024-03-04T16:02:51.071375+0000 mgr.y (mgr.24428) 514 : cluster 0 pgmap v781: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:02:51.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:51 smithi117 bash[28843]: cluster 2024-03-04T16:02:50.503962+0000 mon.a (mon.0) 2824 : cluster 0 osdmap e484: 8 total, 8 up, 8 in 2024-03-04T16:02:51.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:51 smithi117 bash[28843]: audit 2024-03-04T16:02:50.505576+0000 mon.c (mon.2) 707 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:51.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:51 smithi117 bash[28843]: audit 2024-03-04T16:02:50.506440+0000 mon.a (mon.0) 2825 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:51.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:51 smithi117 bash[28843]: audit 2024-03-04T16:02:50.679990+0000 mon.c (mon.2) 708 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:02:51.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:51 smithi117 bash[28843]: audit 2024-03-04T16:02:50.681767+0000 mon.c (mon.2) 709 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:02:51.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:51 smithi117 bash[28843]: audit 2024-03-04T16:02:50.692195+0000 mon.a (mon.0) 2826 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:02:51.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:51 smithi117 bash[28843]: cluster 2024-03-04T16:02:51.071375+0000 mgr.y (mgr.24428) 514 : cluster 0 pgmap v781: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:02:51.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:51 smithi151 bash[23756]: cluster 2024-03-04T16:02:50.503962+0000 mon.a (mon.0) 2824 : cluster 0 osdmap e484: 8 total, 8 up, 8 in 2024-03-04T16:02:51.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:51 smithi151 bash[23756]: audit 2024-03-04T16:02:50.505576+0000 mon.c (mon.2) 707 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:51.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:51 smithi151 bash[23756]: audit 2024-03-04T16:02:50.506440+0000 mon.a (mon.0) 2825 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:02:51.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:51 smithi151 bash[23756]: audit 2024-03-04T16:02:50.679990+0000 mon.c (mon.2) 708 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:02:51.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:51 smithi151 bash[23756]: audit 2024-03-04T16:02:50.681767+0000 mon.c (mon.2) 709 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:02:51.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:51 smithi151 bash[23756]: audit 2024-03-04T16:02:50.692195+0000 mon.a (mon.0) 2826 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:02:51.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:51 smithi151 bash[23756]: cluster 2024-03-04T16:02:51.071375+0000 mgr.y (mgr.24428) 514 : cluster 0 pgmap v781: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:02:52.815 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:52 smithi117 bash[28843]: audit 2024-03-04T16:02:51.502634+0000 mon.a (mon.0) 2827 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:02:52.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:52 smithi117 bash[28843]: cluster 2024-03-04T16:02:51.513702+0000 mon.a (mon.0) 2828 : cluster 0 osdmap e485: 8 total, 8 up, 8 in 2024-03-04T16:02:52.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:52 smithi117 bash[28843]: audit 2024-03-04T16:02:51.539188+0000 mon.c (mon.2) 710 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:02:52.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:52 smithi117 bash[28843]: cluster 2024-03-04T16:02:52.004165+0000 mon.a (mon.0) 2829 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:02:52.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:52 smithi117 bash[21480]: audit 2024-03-04T16:02:51.502634+0000 mon.a (mon.0) 2827 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:02:52.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:52 smithi117 bash[21480]: cluster 2024-03-04T16:02:51.513702+0000 mon.a (mon.0) 2828 : cluster 0 osdmap e485: 8 total, 8 up, 8 in 2024-03-04T16:02:52.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:52 smithi117 bash[21480]: audit 2024-03-04T16:02:51.539188+0000 mon.c (mon.2) 710 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:02:52.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:52 smithi117 bash[21480]: cluster 2024-03-04T16:02:52.004165+0000 mon.a (mon.0) 2829 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:02:52.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:52 smithi151 bash[23756]: audit 2024-03-04T16:02:51.502634+0000 mon.a (mon.0) 2827 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:02:52.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:52 smithi151 bash[23756]: cluster 2024-03-04T16:02:51.513702+0000 mon.a (mon.0) 2828 : cluster 0 osdmap e485: 8 total, 8 up, 8 in 2024-03-04T16:02:52.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:52 smithi151 bash[23756]: audit 2024-03-04T16:02:51.539188+0000 mon.c (mon.2) 710 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:02:52.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:52 smithi151 bash[23756]: cluster 2024-03-04T16:02:52.004165+0000 mon.a (mon.0) 2829 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:02:53.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:53 smithi117 bash[21480]: cluster 2024-03-04T16:02:52.543951+0000 mon.a (mon.0) 2830 : cluster 0 osdmap e486: 8 total, 8 up, 8 in 2024-03-04T16:02:53.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:53 smithi117 bash[21480]: audit 2024-03-04T16:02:52.547948+0000 mon.c (mon.2) 711 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:53.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:53 smithi117 bash[21480]: audit 2024-03-04T16:02:52.551965+0000 mon.a (mon.0) 2831 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:53.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:53 smithi117 bash[21480]: cluster 2024-03-04T16:02:53.072248+0000 mgr.y (mgr.24428) 515 : cluster 0 pgmap v784: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-04T16:02:53.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:53 smithi117 bash[28843]: cluster 2024-03-04T16:02:52.543951+0000 mon.a (mon.0) 2830 : cluster 0 osdmap e486: 8 total, 8 up, 8 in 2024-03-04T16:02:53.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:53 smithi117 bash[28843]: audit 2024-03-04T16:02:52.547948+0000 mon.c (mon.2) 711 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:53.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:53 smithi117 bash[28843]: audit 2024-03-04T16:02:52.551965+0000 mon.a (mon.0) 2831 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:53.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:53 smithi117 bash[28843]: cluster 2024-03-04T16:02:53.072248+0000 mgr.y (mgr.24428) 515 : cluster 0 pgmap v784: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-04T16:02:53.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:53 smithi151 bash[23756]: cluster 2024-03-04T16:02:52.543951+0000 mon.a (mon.0) 2830 : cluster 0 osdmap e486: 8 total, 8 up, 8 in 2024-03-04T16:02:53.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:53 smithi151 bash[23756]: audit 2024-03-04T16:02:52.547948+0000 mon.c (mon.2) 711 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:53.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:53 smithi151 bash[23756]: audit 2024-03-04T16:02:52.551965+0000 mon.a (mon.0) 2831 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:02:53.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:53 smithi151 bash[23756]: cluster 2024-03-04T16:02:53.072248+0000 mgr.y (mgr.24428) 515 : cluster 0 pgmap v784: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-04T16:02:54.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:54 smithi117 bash[28843]: audit 2024-03-04T16:02:53.547658+0000 mon.a (mon.0) 2832 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:02:54.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:54 smithi117 bash[28843]: cluster 2024-03-04T16:02:53.558618+0000 mon.a (mon.0) 2833 : cluster 0 osdmap e487: 8 total, 8 up, 8 in 2024-03-04T16:02:54.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:54 smithi117 bash[28843]: audit 2024-03-04T16:02:53.559907+0000 mon.c (mon.2) 712 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_tier","val": "test-rados-api-smithi117-64061-89-test-flush"}]: dispatch 2024-03-04T16:02:54.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:54 smithi117 bash[28843]: audit 2024-03-04T16:02:53.561949+0000 mon.a (mon.0) 2834 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_tier","val": "test-rados-api-smithi117-64061-89-test-flush"}]: dispatch 2024-03-04T16:02:54.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:54 smithi117 bash[21480]: audit 2024-03-04T16:02:53.547658+0000 mon.a (mon.0) 2832 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:02:54.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:54 smithi117 bash[21480]: cluster 2024-03-04T16:02:53.558618+0000 mon.a (mon.0) 2833 : cluster 0 osdmap e487: 8 total, 8 up, 8 in 2024-03-04T16:02:54.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:54 smithi117 bash[21480]: audit 2024-03-04T16:02:53.559907+0000 mon.c (mon.2) 712 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_tier","val": "test-rados-api-smithi117-64061-89-test-flush"}]: dispatch 2024-03-04T16:02:54.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:54 smithi117 bash[21480]: audit 2024-03-04T16:02:53.561949+0000 mon.a (mon.0) 2834 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_tier","val": "test-rados-api-smithi117-64061-89-test-flush"}]: dispatch 2024-03-04T16:02:54.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:54 smithi151 bash[23756]: audit 2024-03-04T16:02:53.547658+0000 mon.a (mon.0) 2832 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:02:54.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:54 smithi151 bash[23756]: cluster 2024-03-04T16:02:53.558618+0000 mon.a (mon.0) 2833 : cluster 0 osdmap e487: 8 total, 8 up, 8 in 2024-03-04T16:02:54.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:54 smithi151 bash[23756]: audit 2024-03-04T16:02:53.559907+0000 mon.c (mon.2) 712 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_tier","val": "test-rados-api-smithi117-64061-89-test-flush"}]: dispatch 2024-03-04T16:02:54.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:54 smithi151 bash[23756]: audit 2024-03-04T16:02:53.561949+0000 mon.a (mon.0) 2834 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_tier","val": "test-rados-api-smithi117-64061-89-test-flush"}]: dispatch 2024-03-04T16:02:55.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:55 smithi151 bash[23756]: audit 2024-03-04T16:02:54.550633+0000 mon.a (mon.0) 2835 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_tier","val": "test-rados-api-smithi117-64061-89-test-flush"}]': finished 2024-03-04T16:02:55.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:55 smithi151 bash[23756]: cluster 2024-03-04T16:02:54.559994+0000 mon.a (mon.0) 2836 : cluster 0 osdmap e488: 8 total, 8 up, 8 in 2024-03-04T16:02:55.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:55 smithi151 bash[23756]: audit 2024-03-04T16:02:54.562716+0000 mon.c (mon.2) 713 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:55.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:55 smithi151 bash[23756]: audit 2024-03-04T16:02:54.564736+0000 mon.a (mon.0) 2837 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:55.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:55 smithi151 bash[23756]: cluster 2024-03-04T16:02:55.073205+0000 mgr.y (mgr.24428) 516 : cluster 0 pgmap v787: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-04T16:02:55.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:55 smithi151 bash[23756]: audit 2024-03-04T16:02:55.553420+0000 mon.a (mon.0) 2838 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-04T16:02:55.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:55 smithi151 bash[23756]: cluster 2024-03-04T16:02:55.560769+0000 mon.a (mon.0) 2839 : cluster 0 osdmap e489: 8 total, 8 up, 8 in 2024-03-04T16:02:55.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:55 smithi151 bash[23756]: audit 2024-03-04T16:02:55.563326+0000 mon.c (mon.2) 714 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:55.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:55 smithi151 bash[23756]: audit 2024-03-04T16:02:55.564221+0000 mon.a (mon.0) 2840 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:56.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:55 smithi117 bash[28843]: audit 2024-03-04T16:02:54.550633+0000 mon.a (mon.0) 2835 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_tier","val": "test-rados-api-smithi117-64061-89-test-flush"}]': finished 2024-03-04T16:02:56.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:55 smithi117 bash[28843]: cluster 2024-03-04T16:02:54.559994+0000 mon.a (mon.0) 2836 : cluster 0 osdmap e488: 8 total, 8 up, 8 in 2024-03-04T16:02:56.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:55 smithi117 bash[28843]: audit 2024-03-04T16:02:54.562716+0000 mon.c (mon.2) 713 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:56.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:55 smithi117 bash[28843]: audit 2024-03-04T16:02:54.564736+0000 mon.a (mon.0) 2837 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:56.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:55 smithi117 bash[28843]: cluster 2024-03-04T16:02:55.073205+0000 mgr.y (mgr.24428) 516 : cluster 0 pgmap v787: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-04T16:02:56.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:55 smithi117 bash[28843]: audit 2024-03-04T16:02:55.553420+0000 mon.a (mon.0) 2838 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-04T16:02:56.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:55 smithi117 bash[28843]: cluster 2024-03-04T16:02:55.560769+0000 mon.a (mon.0) 2839 : cluster 0 osdmap e489: 8 total, 8 up, 8 in 2024-03-04T16:02:56.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:55 smithi117 bash[28843]: audit 2024-03-04T16:02:55.563326+0000 mon.c (mon.2) 714 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:56.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:55 smithi117 bash[28843]: audit 2024-03-04T16:02:55.564221+0000 mon.a (mon.0) 2840 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:56.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:55 smithi117 bash[21480]: audit 2024-03-04T16:02:54.550633+0000 mon.a (mon.0) 2835 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_tier","val": "test-rados-api-smithi117-64061-89-test-flush"}]': finished 2024-03-04T16:02:56.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:55 smithi117 bash[21480]: cluster 2024-03-04T16:02:54.559994+0000 mon.a (mon.0) 2836 : cluster 0 osdmap e488: 8 total, 8 up, 8 in 2024-03-04T16:02:56.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:55 smithi117 bash[21480]: audit 2024-03-04T16:02:54.562716+0000 mon.c (mon.2) 713 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:56.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:55 smithi117 bash[21480]: audit 2024-03-04T16:02:54.564736+0000 mon.a (mon.0) 2837 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:02:56.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:55 smithi117 bash[21480]: cluster 2024-03-04T16:02:55.073205+0000 mgr.y (mgr.24428) 516 : cluster 0 pgmap v787: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 639 MiB used, 715 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.7 KiB/s wr, 11 op/s 2024-03-04T16:02:56.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:55 smithi117 bash[21480]: audit 2024-03-04T16:02:55.553420+0000 mon.a (mon.0) 2838 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-04T16:02:56.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:55 smithi117 bash[21480]: cluster 2024-03-04T16:02:55.560769+0000 mon.a (mon.0) 2839 : cluster 0 osdmap e489: 8 total, 8 up, 8 in 2024-03-04T16:02:56.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:55 smithi117 bash[21480]: audit 2024-03-04T16:02:55.563326+0000 mon.c (mon.2) 714 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:56.069 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:55 smithi117 bash[21480]: audit 2024-03-04T16:02:55.564221+0000 mon.a (mon.0) 2840 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:02:56.565 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:02:56 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:02:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:02:57.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:02:56 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:02:57.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:57 smithi117 bash[21480]: audit 2024-03-04T16:02:56.556263+0000 mon.a (mon.0) 2841 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-04T16:02:57.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:57 smithi117 bash[21480]: cluster 2024-03-04T16:02:56.563898+0000 mon.a (mon.0) 2842 : cluster 0 osdmap e490: 8 total, 8 up, 8 in 2024-03-04T16:02:57.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:57 smithi117 bash[21480]: audit 2024-03-04T16:02:56.956638+0000 mgr.y (mgr.24428) 517 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:02:57.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:57 smithi117 bash[21480]: cluster 2024-03-04T16:02:57.074018+0000 mgr.y (mgr.24428) 518 : cluster 0 pgmap v790: 260 pgs: 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:57.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:57 smithi117 bash[28843]: audit 2024-03-04T16:02:56.556263+0000 mon.a (mon.0) 2841 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-04T16:02:57.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:57 smithi117 bash[28843]: cluster 2024-03-04T16:02:56.563898+0000 mon.a (mon.0) 2842 : cluster 0 osdmap e490: 8 total, 8 up, 8 in 2024-03-04T16:02:57.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:57 smithi117 bash[28843]: audit 2024-03-04T16:02:56.956638+0000 mgr.y (mgr.24428) 517 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:02:57.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:57 smithi117 bash[28843]: cluster 2024-03-04T16:02:57.074018+0000 mgr.y (mgr.24428) 518 : cluster 0 pgmap v790: 260 pgs: 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:57.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:57 smithi151 bash[23756]: audit 2024-03-04T16:02:56.556263+0000 mon.a (mon.0) 2841 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-04T16:02:57.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:57 smithi151 bash[23756]: cluster 2024-03-04T16:02:56.563898+0000 mon.a (mon.0) 2842 : cluster 0 osdmap e490: 8 total, 8 up, 8 in 2024-03-04T16:02:57.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:57 smithi151 bash[23756]: audit 2024-03-04T16:02:56.956638+0000 mgr.y (mgr.24428) 517 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:02:57.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:57 smithi151 bash[23756]: cluster 2024-03-04T16:02:57.074018+0000 mgr.y (mgr.24428) 518 : cluster 0 pgmap v790: 260 pgs: 260 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:58.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:58 smithi117 bash[21480]: cluster 2024-03-04T16:02:57.573613+0000 mon.a (mon.0) 2843 : cluster 0 osdmap e491: 8 total, 8 up, 8 in 2024-03-04T16:02:58.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:58 smithi117 bash[21480]: audit 2024-03-04T16:02:57.649136+0000 mon.c (mon.2) 715 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:58.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:58 smithi117 bash[21480]: audit 2024-03-04T16:02:57.649665+0000 mon.a (mon.0) 2844 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:58.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:58 smithi117 bash[21480]: audit 2024-03-04T16:02:57.650590+0000 mon.c (mon.2) 716 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-87"}]: dispatch 2024-03-04T16:02:58.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:58 smithi117 bash[21480]: audit 2024-03-04T16:02:57.651011+0000 mon.a (mon.0) 2845 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-87"}]: dispatch 2024-03-04T16:02:58.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:58 smithi117 bash[28843]: cluster 2024-03-04T16:02:57.573613+0000 mon.a (mon.0) 2843 : cluster 0 osdmap e491: 8 total, 8 up, 8 in 2024-03-04T16:02:58.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:58 smithi117 bash[28843]: audit 2024-03-04T16:02:57.649136+0000 mon.c (mon.2) 715 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:58.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:58 smithi117 bash[28843]: audit 2024-03-04T16:02:57.649665+0000 mon.a (mon.0) 2844 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:58.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:58 smithi117 bash[28843]: audit 2024-03-04T16:02:57.650590+0000 mon.c (mon.2) 716 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-87"}]: dispatch 2024-03-04T16:02:58.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:58 smithi117 bash[28843]: audit 2024-03-04T16:02:57.651011+0000 mon.a (mon.0) 2845 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-87"}]: dispatch 2024-03-04T16:02:58.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:58 smithi151 bash[23756]: cluster 2024-03-04T16:02:57.573613+0000 mon.a (mon.0) 2843 : cluster 0 osdmap e491: 8 total, 8 up, 8 in 2024-03-04T16:02:58.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:58 smithi151 bash[23756]: audit 2024-03-04T16:02:57.649136+0000 mon.c (mon.2) 715 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:58.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:58 smithi151 bash[23756]: audit 2024-03-04T16:02:57.649665+0000 mon.a (mon.0) 2844 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:02:58.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:58 smithi151 bash[23756]: audit 2024-03-04T16:02:57.650590+0000 mon.c (mon.2) 716 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-87"}]: dispatch 2024-03-04T16:02:58.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:58 smithi151 bash[23756]: audit 2024-03-04T16:02:57.651011+0000 mon.a (mon.0) 2845 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-87"}]: dispatch 2024-03-04T16:02:59.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:59 smithi151 bash[23756]: cluster 2024-03-04T16:02:58.582282+0000 mon.a (mon.0) 2846 : cluster 0 osdmap e492: 8 total, 8 up, 8 in 2024-03-04T16:02:59.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:59 smithi151 bash[23756]: cluster 2024-03-04T16:02:59.074778+0000 mgr.y (mgr.24428) 519 : cluster 0 pgmap v793: 196 pgs: 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:02:59.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:02:59 smithi151 bash[23756]: audit 2024-03-04T16:02:59.484712+0000 mon.c (mon.2) 717 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:03:00.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:59 smithi117 bash[21480]: cluster 2024-03-04T16:02:58.582282+0000 mon.a (mon.0) 2846 : cluster 0 osdmap e492: 8 total, 8 up, 8 in 2024-03-04T16:03:00.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:59 smithi117 bash[21480]: cluster 2024-03-04T16:02:59.074778+0000 mgr.y (mgr.24428) 519 : cluster 0 pgmap v793: 196 pgs: 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:03:00.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:02:59 smithi117 bash[21480]: audit 2024-03-04T16:02:59.484712+0000 mon.c (mon.2) 717 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:03:00.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:59 smithi117 bash[28843]: cluster 2024-03-04T16:02:58.582282+0000 mon.a (mon.0) 2846 : cluster 0 osdmap e492: 8 total, 8 up, 8 in 2024-03-04T16:03:00.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:59 smithi117 bash[28843]: cluster 2024-03-04T16:02:59.074778+0000 mgr.y (mgr.24428) 519 : cluster 0 pgmap v793: 196 pgs: 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:03:00.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:02:59 smithi117 bash[28843]: audit 2024-03-04T16:02:59.484712+0000 mon.c (mon.2) 717 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:03:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:00 smithi151 bash[23756]: cluster 2024-03-04T16:02:59.575417+0000 mon.a (mon.0) 2847 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:00 smithi151 bash[23756]: cluster 2024-03-04T16:02:59.600197+0000 mon.a (mon.0) 2848 : cluster 0 osdmap e493: 8 total, 8 up, 8 in 2024-03-04T16:03:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:00 smithi151 bash[23756]: audit 2024-03-04T16:02:59.606195+0000 mon.c (mon.2) 718 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:00 smithi151 bash[23756]: audit 2024-03-04T16:02:59.607299+0000 mon.a (mon.0) 2849 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:01.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:00 smithi117 bash[21480]: cluster 2024-03-04T16:02:59.575417+0000 mon.a (mon.0) 2847 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:01.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:00 smithi117 bash[21480]: cluster 2024-03-04T16:02:59.600197+0000 mon.a (mon.0) 2848 : cluster 0 osdmap e493: 8 total, 8 up, 8 in 2024-03-04T16:03:01.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:00 smithi117 bash[21480]: audit 2024-03-04T16:02:59.606195+0000 mon.c (mon.2) 718 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:01.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:00 smithi117 bash[21480]: audit 2024-03-04T16:02:59.607299+0000 mon.a (mon.0) 2849 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:01.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:00 smithi117 bash[28843]: cluster 2024-03-04T16:02:59.575417+0000 mon.a (mon.0) 2847 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:01.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:00 smithi117 bash[28843]: cluster 2024-03-04T16:02:59.600197+0000 mon.a (mon.0) 2848 : cluster 0 osdmap e493: 8 total, 8 up, 8 in 2024-03-04T16:03:01.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:00 smithi117 bash[28843]: audit 2024-03-04T16:02:59.606195+0000 mon.c (mon.2) 718 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:01.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:00 smithi117 bash[28843]: audit 2024-03-04T16:02:59.607299+0000 mon.a (mon.0) 2849 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:01.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:01 smithi151 bash[23756]: audit 2024-03-04T16:03:00.591063+0000 mon.a (mon.0) 2850 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:03:01.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:01 smithi151 bash[23756]: cluster 2024-03-04T16:03:00.603420+0000 mon.a (mon.0) 2851 : cluster 0 osdmap e494: 8 total, 8 up, 8 in 2024-03-04T16:03:01.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:01 smithi151 bash[23756]: audit 2024-03-04T16:03:00.606814+0000 mon.c (mon.2) 719 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:03:01.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:01 smithi151 bash[23756]: audit 2024-03-04T16:03:00.653741+0000 mon.c (mon.2) 720 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:03:01.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:01 smithi151 bash[23756]: audit 2024-03-04T16:03:00.657800+0000 mon.a (mon.0) 2852 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:03:01.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:01 smithi151 bash[23756]: cluster 2024-03-04T16:03:01.075596+0000 mgr.y (mgr.24428) 520 : cluster 0 pgmap v796: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:03:02.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:01 smithi117 bash[21480]: audit 2024-03-04T16:03:00.591063+0000 mon.a (mon.0) 2850 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:03:02.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:01 smithi117 bash[21480]: cluster 2024-03-04T16:03:00.603420+0000 mon.a (mon.0) 2851 : cluster 0 osdmap e494: 8 total, 8 up, 8 in 2024-03-04T16:03:02.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:01 smithi117 bash[21480]: audit 2024-03-04T16:03:00.606814+0000 mon.c (mon.2) 719 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:03:02.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:01 smithi117 bash[21480]: audit 2024-03-04T16:03:00.653741+0000 mon.c (mon.2) 720 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:03:02.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:01 smithi117 bash[21480]: audit 2024-03-04T16:03:00.657800+0000 mon.a (mon.0) 2852 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:03:02.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:01 smithi117 bash[21480]: cluster 2024-03-04T16:03:01.075596+0000 mgr.y (mgr.24428) 520 : cluster 0 pgmap v796: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:03:02.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:01 smithi117 bash[28843]: audit 2024-03-04T16:03:00.591063+0000 mon.a (mon.0) 2850 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:03:02.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:01 smithi117 bash[28843]: cluster 2024-03-04T16:03:00.603420+0000 mon.a (mon.0) 2851 : cluster 0 osdmap e494: 8 total, 8 up, 8 in 2024-03-04T16:03:02.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:01 smithi117 bash[28843]: audit 2024-03-04T16:03:00.606814+0000 mon.c (mon.2) 719 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:03:02.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:01 smithi117 bash[28843]: audit 2024-03-04T16:03:00.653741+0000 mon.c (mon.2) 720 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:03:02.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:01 smithi117 bash[28843]: audit 2024-03-04T16:03:00.657800+0000 mon.a (mon.0) 2852 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:03:02.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:01 smithi117 bash[28843]: cluster 2024-03-04T16:03:01.075596+0000 mgr.y (mgr.24428) 520 : cluster 0 pgmap v796: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:03:02.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:02 smithi151 bash[23756]: audit 2024-03-04T16:03:01.618547+0000 mon.a (mon.0) 2853 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:03:02.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:02 smithi151 bash[23756]: cluster 2024-03-04T16:03:01.631193+0000 mon.a (mon.0) 2854 : cluster 0 osdmap e495: 8 total, 8 up, 8 in 2024-03-04T16:03:03.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:02 smithi117 bash[21480]: audit 2024-03-04T16:03:01.618547+0000 mon.a (mon.0) 2853 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:03:03.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:02 smithi117 bash[21480]: cluster 2024-03-04T16:03:01.631193+0000 mon.a (mon.0) 2854 : cluster 0 osdmap e495: 8 total, 8 up, 8 in 2024-03-04T16:03:03.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:02 smithi117 bash[28843]: audit 2024-03-04T16:03:01.618547+0000 mon.a (mon.0) 2853 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:03:03.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:02 smithi117 bash[28843]: cluster 2024-03-04T16:03:01.631193+0000 mon.a (mon.0) 2854 : cluster 0 osdmap e495: 8 total, 8 up, 8 in 2024-03-04T16:03:03.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:03 smithi151 bash[23756]: cluster 2024-03-04T16:03:02.635576+0000 mon.a (mon.0) 2855 : cluster 0 osdmap e496: 8 total, 8 up, 8 in 2024-03-04T16:03:03.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:03 smithi151 bash[23756]: cluster 2024-03-04T16:03:03.076446+0000 mgr.y (mgr.24428) 521 : cluster 0 pgmap v799: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T16:03:04.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:03 smithi117 bash[21480]: cluster 2024-03-04T16:03:02.635576+0000 mon.a (mon.0) 2855 : cluster 0 osdmap e496: 8 total, 8 up, 8 in 2024-03-04T16:03:04.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:03 smithi117 bash[21480]: cluster 2024-03-04T16:03:03.076446+0000 mgr.y (mgr.24428) 521 : cluster 0 pgmap v799: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T16:03:04.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:03 smithi117 bash[28843]: cluster 2024-03-04T16:03:02.635576+0000 mon.a (mon.0) 2855 : cluster 0 osdmap e496: 8 total, 8 up, 8 in 2024-03-04T16:03:04.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:03 smithi117 bash[28843]: cluster 2024-03-04T16:03:03.076446+0000 mgr.y (mgr.24428) 521 : cluster 0 pgmap v799: 228 pgs: 228 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T16:03:04.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:04 smithi151 bash[23756]: cluster 2024-03-04T16:03:03.651318+0000 mon.a (mon.0) 2856 : cluster 0 osdmap e497: 8 total, 8 up, 8 in 2024-03-04T16:03:04.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:04 smithi151 bash[23756]: audit 2024-03-04T16:03:03.738775+0000 mon.c (mon.2) 721 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:04.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:04 smithi151 bash[23756]: audit 2024-03-04T16:03:03.739502+0000 mon.a (mon.0) 2857 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:04.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:04 smithi151 bash[23756]: audit 2024-03-04T16:03:03.740781+0000 mon.c (mon.2) 722 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-90"}]: dispatch 2024-03-04T16:03:04.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:04 smithi151 bash[23756]: audit 2024-03-04T16:03:03.741418+0000 mon.a (mon.0) 2858 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-90"}]: dispatch 2024-03-04T16:03:05.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:04 smithi117 bash[21480]: cluster 2024-03-04T16:03:03.651318+0000 mon.a (mon.0) 2856 : cluster 0 osdmap e497: 8 total, 8 up, 8 in 2024-03-04T16:03:05.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:04 smithi117 bash[21480]: audit 2024-03-04T16:03:03.738775+0000 mon.c (mon.2) 721 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:05.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:04 smithi117 bash[21480]: audit 2024-03-04T16:03:03.739502+0000 mon.a (mon.0) 2857 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:05.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:04 smithi117 bash[21480]: audit 2024-03-04T16:03:03.740781+0000 mon.c (mon.2) 722 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-90"}]: dispatch 2024-03-04T16:03:05.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:04 smithi117 bash[21480]: audit 2024-03-04T16:03:03.741418+0000 mon.a (mon.0) 2858 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-90"}]: dispatch 2024-03-04T16:03:05.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:04 smithi117 bash[28843]: cluster 2024-03-04T16:03:03.651318+0000 mon.a (mon.0) 2856 : cluster 0 osdmap e497: 8 total, 8 up, 8 in 2024-03-04T16:03:05.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:04 smithi117 bash[28843]: audit 2024-03-04T16:03:03.738775+0000 mon.c (mon.2) 721 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:05.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:04 smithi117 bash[28843]: audit 2024-03-04T16:03:03.739502+0000 mon.a (mon.0) 2857 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:05.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:04 smithi117 bash[28843]: audit 2024-03-04T16:03:03.740781+0000 mon.c (mon.2) 722 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-90"}]: dispatch 2024-03-04T16:03:05.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:04 smithi117 bash[28843]: audit 2024-03-04T16:03:03.741418+0000 mon.a (mon.0) 2858 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-90"}]: dispatch 2024-03-04T16:03:05.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:05 smithi151 bash[23756]: cluster 2024-03-04T16:03:04.646519+0000 mon.a (mon.0) 2859 : cluster 0 osdmap e498: 8 total, 8 up, 8 in 2024-03-04T16:03:05.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:05 smithi151 bash[23756]: cluster 2024-03-04T16:03:05.077221+0000 mgr.y (mgr.24428) 522 : cluster 0 pgmap v802: 196 pgs: 196 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T16:03:06.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:05 smithi117 bash[21480]: cluster 2024-03-04T16:03:04.646519+0000 mon.a (mon.0) 2859 : cluster 0 osdmap e498: 8 total, 8 up, 8 in 2024-03-04T16:03:06.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:05 smithi117 bash[21480]: cluster 2024-03-04T16:03:05.077221+0000 mgr.y (mgr.24428) 522 : cluster 0 pgmap v802: 196 pgs: 196 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T16:03:06.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:05 smithi117 bash[28843]: cluster 2024-03-04T16:03:04.646519+0000 mon.a (mon.0) 2859 : cluster 0 osdmap e498: 8 total, 8 up, 8 in 2024-03-04T16:03:06.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:05 smithi117 bash[28843]: cluster 2024-03-04T16:03:05.077221+0000 mgr.y (mgr.24428) 522 : cluster 0 pgmap v802: 196 pgs: 196 active+clean; 8.4 MiB data, 649 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T16:03:06.565 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:03:06 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:03:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:03:06.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:06 smithi151 bash[23756]: cluster 2024-03-04T16:03:05.676143+0000 mon.a (mon.0) 2860 : cluster 0 osdmap e499: 8 total, 8 up, 8 in 2024-03-04T16:03:06.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:06 smithi151 bash[23756]: audit 2024-03-04T16:03:05.689279+0000 mon.c (mon.2) 723 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:06.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:06 smithi151 bash[23756]: audit 2024-03-04T16:03:05.690380+0000 mon.a (mon.0) 2861 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:07.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:06 smithi117 bash[21480]: cluster 2024-03-04T16:03:05.676143+0000 mon.a (mon.0) 2860 : cluster 0 osdmap e499: 8 total, 8 up, 8 in 2024-03-04T16:03:07.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:06 smithi117 bash[21480]: audit 2024-03-04T16:03:05.689279+0000 mon.c (mon.2) 723 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:07.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:06 smithi117 bash[21480]: audit 2024-03-04T16:03:05.690380+0000 mon.a (mon.0) 2861 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:07.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:06 smithi117 bash[28843]: cluster 2024-03-04T16:03:05.676143+0000 mon.a (mon.0) 2860 : cluster 0 osdmap e499: 8 total, 8 up, 8 in 2024-03-04T16:03:07.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:06 smithi117 bash[28843]: audit 2024-03-04T16:03:05.689279+0000 mon.c (mon.2) 723 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:07.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:06 smithi117 bash[28843]: audit 2024-03-04T16:03:05.690380+0000 mon.a (mon.0) 2861 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:07.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:03:06 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:03:08.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:07 smithi117 bash[21480]: audit 2024-03-04T16:03:06.677931+0000 mon.a (mon.0) 2862 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:03:08.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:07 smithi117 bash[21480]: cluster 2024-03-04T16:03:06.686435+0000 mon.a (mon.0) 2863 : cluster 0 osdmap e500: 8 total, 8 up, 8 in 2024-03-04T16:03:08.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:07 smithi117 bash[21480]: audit 2024-03-04T16:03:06.695665+0000 mon.c (mon.2) 724 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:03:08.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:07 smithi117 bash[21480]: audit 2024-03-04T16:03:06.965265+0000 mgr.y (mgr.24428) 523 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:03:08.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:07 smithi117 bash[21480]: cluster 2024-03-04T16:03:07.077971+0000 mgr.y (mgr.24428) 524 : cluster 0 pgmap v805: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-04T16:03:08.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:07 smithi117 bash[28843]: audit 2024-03-04T16:03:06.677931+0000 mon.a (mon.0) 2862 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:03:08.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:07 smithi117 bash[28843]: cluster 2024-03-04T16:03:06.686435+0000 mon.a (mon.0) 2863 : cluster 0 osdmap e500: 8 total, 8 up, 8 in 2024-03-04T16:03:08.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:07 smithi117 bash[28843]: audit 2024-03-04T16:03:06.695665+0000 mon.c (mon.2) 724 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:03:08.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:07 smithi117 bash[28843]: audit 2024-03-04T16:03:06.965265+0000 mgr.y (mgr.24428) 523 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:03:08.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:07 smithi117 bash[28843]: cluster 2024-03-04T16:03:07.077971+0000 mgr.y (mgr.24428) 524 : cluster 0 pgmap v805: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-04T16:03:08.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:07 smithi151 bash[23756]: audit 2024-03-04T16:03:06.677931+0000 mon.a (mon.0) 2862 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:03:08.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:07 smithi151 bash[23756]: cluster 2024-03-04T16:03:06.686435+0000 mon.a (mon.0) 2863 : cluster 0 osdmap e500: 8 total, 8 up, 8 in 2024-03-04T16:03:08.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:07 smithi151 bash[23756]: audit 2024-03-04T16:03:06.695665+0000 mon.c (mon.2) 724 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:03:08.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:07 smithi151 bash[23756]: audit 2024-03-04T16:03:06.965265+0000 mgr.y (mgr.24428) 523 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:03:08.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:07 smithi151 bash[23756]: cluster 2024-03-04T16:03:07.077971+0000 mgr.y (mgr.24428) 524 : cluster 0 pgmap v805: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-04T16:03:09.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:08 smithi117 bash[21480]: cluster 2024-03-04T16:03:07.701053+0000 mon.a (mon.0) 2864 : cluster 0 osdmap e501: 8 total, 8 up, 8 in 2024-03-04T16:03:09.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:08 smithi117 bash[28843]: cluster 2024-03-04T16:03:07.701053+0000 mon.a (mon.0) 2864 : cluster 0 osdmap e501: 8 total, 8 up, 8 in 2024-03-04T16:03:09.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:08 smithi151 bash[23756]: cluster 2024-03-04T16:03:07.701053+0000 mon.a (mon.0) 2864 : cluster 0 osdmap e501: 8 total, 8 up, 8 in 2024-03-04T16:03:10.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:09 smithi117 bash[21480]: cluster 2024-03-04T16:03:08.707825+0000 mon.a (mon.0) 2865 : cluster 0 osdmap e502: 8 total, 8 up, 8 in 2024-03-04T16:03:10.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:09 smithi117 bash[21480]: audit 2024-03-04T16:03:08.799996+0000 mon.c (mon.2) 725 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:10.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:09 smithi117 bash[21480]: audit 2024-03-04T16:03:08.800742+0000 mon.a (mon.0) 2866 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:10.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:09 smithi117 bash[21480]: audit 2024-03-04T16:03:08.802035+0000 mon.c (mon.2) 726 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-92"}]: dispatch 2024-03-04T16:03:10.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:09 smithi117 bash[21480]: audit 2024-03-04T16:03:08.802728+0000 mon.a (mon.0) 2867 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-92"}]: dispatch 2024-03-04T16:03:10.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:09 smithi117 bash[21480]: cluster 2024-03-04T16:03:09.078821+0000 mgr.y (mgr.24428) 525 : cluster 0 pgmap v808: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-04T16:03:10.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:09 smithi117 bash[28843]: cluster 2024-03-04T16:03:08.707825+0000 mon.a (mon.0) 2865 : cluster 0 osdmap e502: 8 total, 8 up, 8 in 2024-03-04T16:03:10.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:09 smithi117 bash[28843]: audit 2024-03-04T16:03:08.799996+0000 mon.c (mon.2) 725 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:10.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:09 smithi117 bash[28843]: audit 2024-03-04T16:03:08.800742+0000 mon.a (mon.0) 2866 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:10.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:09 smithi117 bash[28843]: audit 2024-03-04T16:03:08.802035+0000 mon.c (mon.2) 726 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-92"}]: dispatch 2024-03-04T16:03:10.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:09 smithi117 bash[28843]: audit 2024-03-04T16:03:08.802728+0000 mon.a (mon.0) 2867 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-92"}]: dispatch 2024-03-04T16:03:10.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:09 smithi117 bash[28843]: cluster 2024-03-04T16:03:09.078821+0000 mgr.y (mgr.24428) 525 : cluster 0 pgmap v808: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-04T16:03:10.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:09 smithi151 bash[23756]: cluster 2024-03-04T16:03:08.707825+0000 mon.a (mon.0) 2865 : cluster 0 osdmap e502: 8 total, 8 up, 8 in 2024-03-04T16:03:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:09 smithi151 bash[23756]: audit 2024-03-04T16:03:08.799996+0000 mon.c (mon.2) 725 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:09 smithi151 bash[23756]: audit 2024-03-04T16:03:08.800742+0000 mon.a (mon.0) 2866 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:09 smithi151 bash[23756]: audit 2024-03-04T16:03:08.802035+0000 mon.c (mon.2) 726 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-92"}]: dispatch 2024-03-04T16:03:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:09 smithi151 bash[23756]: audit 2024-03-04T16:03:08.802728+0000 mon.a (mon.0) 2867 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-92"}]: dispatch 2024-03-04T16:03:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:09 smithi151 bash[23756]: cluster 2024-03-04T16:03:09.078821+0000 mgr.y (mgr.24428) 525 : cluster 0 pgmap v808: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-03-04T16:03:11.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:10 smithi117 bash[21480]: cluster 2024-03-04T16:03:09.718629+0000 mon.a (mon.0) 2868 : cluster 0 osdmap e503: 8 total, 8 up, 8 in 2024-03-04T16:03:11.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:10 smithi117 bash[28843]: cluster 2024-03-04T16:03:09.718629+0000 mon.a (mon.0) 2868 : cluster 0 osdmap e503: 8 total, 8 up, 8 in 2024-03-04T16:03:11.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:10 smithi151 bash[23756]: cluster 2024-03-04T16:03:09.718629+0000 mon.a (mon.0) 2868 : cluster 0 osdmap e503: 8 total, 8 up, 8 in 2024-03-04T16:03:12.065 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:11 smithi117 bash[28843]: cluster 2024-03-04T16:03:10.730264+0000 mon.a (mon.0) 2869 : cluster 0 osdmap e504: 8 total, 8 up, 8 in 2024-03-04T16:03:12.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:11 smithi117 bash[28843]: audit 2024-03-04T16:03:10.737329+0000 mon.c (mon.2) 727 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:12.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:11 smithi117 bash[28843]: audit 2024-03-04T16:03:10.741546+0000 mon.a (mon.0) 2870 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:12.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:11 smithi117 bash[28843]: cluster 2024-03-04T16:03:11.079633+0000 mgr.y (mgr.24428) 526 : cluster 0 pgmap v811: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:03:12.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:11 smithi117 bash[21480]: cluster 2024-03-04T16:03:10.730264+0000 mon.a (mon.0) 2869 : cluster 0 osdmap e504: 8 total, 8 up, 8 in 2024-03-04T16:03:12.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:11 smithi117 bash[21480]: audit 2024-03-04T16:03:10.737329+0000 mon.c (mon.2) 727 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:12.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:11 smithi117 bash[21480]: audit 2024-03-04T16:03:10.741546+0000 mon.a (mon.0) 2870 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:12.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:11 smithi117 bash[21480]: cluster 2024-03-04T16:03:11.079633+0000 mgr.y (mgr.24428) 526 : cluster 0 pgmap v811: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:03:12.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:11 smithi151 bash[23756]: cluster 2024-03-04T16:03:10.730264+0000 mon.a (mon.0) 2869 : cluster 0 osdmap e504: 8 total, 8 up, 8 in 2024-03-04T16:03:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:11 smithi151 bash[23756]: audit 2024-03-04T16:03:10.737329+0000 mon.c (mon.2) 727 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:11 smithi151 bash[23756]: audit 2024-03-04T16:03:10.741546+0000 mon.a (mon.0) 2870 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:11 smithi151 bash[23756]: cluster 2024-03-04T16:03:11.079633+0000 mgr.y (mgr.24428) 526 : cluster 0 pgmap v811: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:03:13.065 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:12 smithi117 bash[28843]: cluster 2024-03-04T16:03:11.722389+0000 mon.a (mon.0) 2871 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:13.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:12 smithi117 bash[28843]: audit 2024-03-04T16:03:11.734295+0000 mon.a (mon.0) 2872 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:03:13.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:12 smithi117 bash[28843]: cluster 2024-03-04T16:03:11.745789+0000 mon.a (mon.0) 2873 : cluster 0 osdmap e505: 8 total, 8 up, 8 in 2024-03-04T16:03:13.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:12 smithi117 bash[28843]: audit 2024-03-04T16:03:11.757716+0000 mon.c (mon.2) 728 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:03:13.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:12 smithi117 bash[21480]: cluster 2024-03-04T16:03:11.722389+0000 mon.a (mon.0) 2871 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:13.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:12 smithi117 bash[21480]: audit 2024-03-04T16:03:11.734295+0000 mon.a (mon.0) 2872 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:03:13.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:12 smithi117 bash[21480]: cluster 2024-03-04T16:03:11.745789+0000 mon.a (mon.0) 2873 : cluster 0 osdmap e505: 8 total, 8 up, 8 in 2024-03-04T16:03:13.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:12 smithi117 bash[21480]: audit 2024-03-04T16:03:11.757716+0000 mon.c (mon.2) 728 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:03:13.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:12 smithi151 bash[23756]: cluster 2024-03-04T16:03:11.722389+0000 mon.a (mon.0) 2871 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:13.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:12 smithi151 bash[23756]: audit 2024-03-04T16:03:11.734295+0000 mon.a (mon.0) 2872 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:03:13.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:12 smithi151 bash[23756]: cluster 2024-03-04T16:03:11.745789+0000 mon.a (mon.0) 2873 : cluster 0 osdmap e505: 8 total, 8 up, 8 in 2024-03-04T16:03:13.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:12 smithi151 bash[23756]: audit 2024-03-04T16:03:11.757716+0000 mon.c (mon.2) 728 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:03:14.065 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:13 smithi117 bash[28843]: cluster 2024-03-04T16:03:12.744989+0000 mon.a (mon.0) 2874 : cluster 0 osdmap e506: 8 total, 8 up, 8 in 2024-03-04T16:03:14.065 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:13 smithi117 bash[28843]: cluster 2024-03-04T16:03:13.080559+0000 mgr.y (mgr.24428) 527 : cluster 0 pgmap v814: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T16:03:14.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:13 smithi117 bash[21480]: cluster 2024-03-04T16:03:12.744989+0000 mon.a (mon.0) 2874 : cluster 0 osdmap e506: 8 total, 8 up, 8 in 2024-03-04T16:03:14.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:13 smithi117 bash[21480]: cluster 2024-03-04T16:03:13.080559+0000 mgr.y (mgr.24428) 527 : cluster 0 pgmap v814: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T16:03:14.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:13 smithi151 bash[23756]: cluster 2024-03-04T16:03:12.744989+0000 mon.a (mon.0) 2874 : cluster 0 osdmap e506: 8 total, 8 up, 8 in 2024-03-04T16:03:14.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:13 smithi151 bash[23756]: cluster 2024-03-04T16:03:13.080559+0000 mgr.y (mgr.24428) 527 : cluster 0 pgmap v814: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-03-04T16:03:15.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:14 smithi117 bash[21480]: cluster 2024-03-04T16:03:13.771897+0000 mon.a (mon.0) 2875 : cluster 0 osdmap e507: 8 total, 8 up, 8 in 2024-03-04T16:03:15.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:14 smithi117 bash[21480]: audit 2024-03-04T16:03:14.518872+0000 mon.c (mon.2) 729 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:03:15.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:14 smithi117 bash[28843]: cluster 2024-03-04T16:03:13.771897+0000 mon.a (mon.0) 2875 : cluster 0 osdmap e507: 8 total, 8 up, 8 in 2024-03-04T16:03:15.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:14 smithi117 bash[28843]: audit 2024-03-04T16:03:14.518872+0000 mon.c (mon.2) 729 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:03:15.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:14 smithi151 bash[23756]: cluster 2024-03-04T16:03:13.771897+0000 mon.a (mon.0) 2875 : cluster 0 osdmap e507: 8 total, 8 up, 8 in 2024-03-04T16:03:15.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:14 smithi151 bash[23756]: audit 2024-03-04T16:03:14.518872+0000 mon.c (mon.2) 729 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:03:16.065 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:15 smithi117 bash[21480]: cluster 2024-03-04T16:03:15.081537+0000 mgr.y (mgr.24428) 528 : cluster 0 pgmap v816: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 705 B/s wr, 2 op/s 2024-03-04T16:03:16.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:15 smithi117 bash[28843]: cluster 2024-03-04T16:03:15.081537+0000 mgr.y (mgr.24428) 528 : cluster 0 pgmap v816: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 705 B/s wr, 2 op/s 2024-03-04T16:03:16.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:15 smithi151 bash[23756]: cluster 2024-03-04T16:03:15.081537+0000 mgr.y (mgr.24428) 528 : cluster 0 pgmap v816: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 650 MiB used, 715 GiB / 715 GiB avail; 1.6 KiB/s rd, 705 B/s wr, 2 op/s 2024-03-04T16:03:16.565 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:03:16 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:03:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:03:17.432 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:03:16 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:03:18.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:18 smithi151 bash[23756]: audit 2024-03-04T16:03:16.976724+0000 mgr.y (mgr.24428) 529 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:03:18.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:18 smithi151 bash[23756]: cluster 2024-03-04T16:03:17.083707+0000 mgr.y (mgr.24428) 530 : cluster 0 pgmap v817: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-03-04T16:03:18.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:18 smithi117 bash[21480]: audit 2024-03-04T16:03:16.976724+0000 mgr.y (mgr.24428) 529 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:03:18.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:18 smithi117 bash[21480]: cluster 2024-03-04T16:03:17.083707+0000 mgr.y (mgr.24428) 530 : cluster 0 pgmap v817: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-03-04T16:03:18.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:18 smithi117 bash[28843]: audit 2024-03-04T16:03:16.976724+0000 mgr.y (mgr.24428) 529 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:03:18.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:18 smithi117 bash[28843]: cluster 2024-03-04T16:03:17.083707+0000 mgr.y (mgr.24428) 530 : cluster 0 pgmap v817: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.8 KiB/s wr, 6 op/s 2024-03-04T16:03:19.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:19 smithi151 bash[23756]: cluster 2024-03-04T16:03:19.045775+0000 mon.a (mon.0) 2876 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:19.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:19 smithi117 bash[21480]: cluster 2024-03-04T16:03:19.045775+0000 mon.a (mon.0) 2876 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:19.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:19 smithi117 bash[28843]: cluster 2024-03-04T16:03:19.045775+0000 mon.a (mon.0) 2876 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:20.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:20 smithi151 bash[23756]: cluster 2024-03-04T16:03:19.084508+0000 mgr.y (mgr.24428) 531 : cluster 0 pgmap v818: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-03-04T16:03:20.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:20 smithi117 bash[21480]: cluster 2024-03-04T16:03:19.084508+0000 mgr.y (mgr.24428) 531 : cluster 0 pgmap v818: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-03-04T16:03:20.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:20 smithi117 bash[28843]: cluster 2024-03-04T16:03:19.084508+0000 mgr.y (mgr.24428) 531 : cluster 0 pgmap v818: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.1 KiB/s wr, 4 op/s 2024-03-04T16:03:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:22 smithi151 bash[23756]: cluster 2024-03-04T16:03:21.085356+0000 mgr.y (mgr.24428) 532 : cluster 0 pgmap v819: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 981 B/s wr, 3 op/s 2024-03-04T16:03:22.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:22 smithi117 bash[21480]: cluster 2024-03-04T16:03:21.085356+0000 mgr.y (mgr.24428) 532 : cluster 0 pgmap v819: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 981 B/s wr, 3 op/s 2024-03-04T16:03:22.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:22 smithi117 bash[28843]: cluster 2024-03-04T16:03:21.085356+0000 mgr.y (mgr.24428) 532 : cluster 0 pgmap v819: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 981 B/s wr, 3 op/s 2024-03-04T16:03:24.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:24 smithi151 bash[23756]: cluster 2024-03-04T16:03:23.087612+0000 mgr.y (mgr.24428) 533 : cluster 0 pgmap v820: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-03-04T16:03:24.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:24 smithi117 bash[21480]: cluster 2024-03-04T16:03:23.087612+0000 mgr.y (mgr.24428) 533 : cluster 0 pgmap v820: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-03-04T16:03:24.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:24 smithi117 bash[28843]: cluster 2024-03-04T16:03:23.087612+0000 mgr.y (mgr.24428) 533 : cluster 0 pgmap v820: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-03-04T16:03:25.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:25 smithi151 bash[23756]: cluster 2024-03-04T16:03:24.169406+0000 mon.a (mon.0) 2877 : cluster 0 osdmap e508: 8 total, 8 up, 8 in 2024-03-04T16:03:25.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:25 smithi117 bash[21480]: cluster 2024-03-04T16:03:24.169406+0000 mon.a (mon.0) 2877 : cluster 0 osdmap e508: 8 total, 8 up, 8 in 2024-03-04T16:03:25.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:25 smithi117 bash[28843]: cluster 2024-03-04T16:03:24.169406+0000 mon.a (mon.0) 2877 : cluster 0 osdmap e508: 8 total, 8 up, 8 in 2024-03-04T16:03:26.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:26 smithi151 bash[23756]: cluster 2024-03-04T16:03:25.088427+0000 mgr.y (mgr.24428) 534 : cluster 0 pgmap v822: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-03-04T16:03:26.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:26 smithi117 bash[21480]: cluster 2024-03-04T16:03:25.088427+0000 mgr.y (mgr.24428) 534 : cluster 0 pgmap v822: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-03-04T16:03:26.565 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:03:26 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:03:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:03:26.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:26 smithi117 bash[28843]: cluster 2024-03-04T16:03:25.088427+0000 mgr.y (mgr.24428) 534 : cluster 0 pgmap v822: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 818 B/s wr, 3 op/s 2024-03-04T16:03:27.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:03:26 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:03:28.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:28 smithi151 bash[23756]: audit 2024-03-04T16:03:26.988278+0000 mgr.y (mgr.24428) 535 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:03:28.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:28 smithi151 bash[23756]: cluster 2024-03-04T16:03:27.090612+0000 mgr.y (mgr.24428) 536 : cluster 0 pgmap v823: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:03:28.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:28 smithi117 bash[21480]: audit 2024-03-04T16:03:26.988278+0000 mgr.y (mgr.24428) 535 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:03:28.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:28 smithi117 bash[21480]: cluster 2024-03-04T16:03:27.090612+0000 mgr.y (mgr.24428) 536 : cluster 0 pgmap v823: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:03:28.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:28 smithi117 bash[28843]: audit 2024-03-04T16:03:26.988278+0000 mgr.y (mgr.24428) 535 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:03:28.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:28 smithi117 bash[28843]: cluster 2024-03-04T16:03:27.090612+0000 mgr.y (mgr.24428) 536 : cluster 0 pgmap v823: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:03:30.315 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:30 smithi117 bash[21480]: cluster 2024-03-04T16:03:29.060135+0000 mon.a (mon.0) 2878 : cluster 0 osdmap e509: 8 total, 8 up, 8 in 2024-03-04T16:03:30.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:30 smithi117 bash[21480]: cluster 2024-03-04T16:03:29.091431+0000 mgr.y (mgr.24428) 537 : cluster 0 pgmap v825: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:03:30.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:30 smithi117 bash[21480]: audit 2024-03-04T16:03:29.552240+0000 mon.c (mon.2) 730 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:03:30.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:30 smithi117 bash[28843]: cluster 2024-03-04T16:03:29.060135+0000 mon.a (mon.0) 2878 : cluster 0 osdmap e509: 8 total, 8 up, 8 in 2024-03-04T16:03:30.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:30 smithi117 bash[28843]: cluster 2024-03-04T16:03:29.091431+0000 mgr.y (mgr.24428) 537 : cluster 0 pgmap v825: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:03:30.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:30 smithi117 bash[28843]: audit 2024-03-04T16:03:29.552240+0000 mon.c (mon.2) 730 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:03:30.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:30 smithi151 bash[23756]: cluster 2024-03-04T16:03:29.060135+0000 mon.a (mon.0) 2878 : cluster 0 osdmap e509: 8 total, 8 up, 8 in 2024-03-04T16:03:30.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:30 smithi151 bash[23756]: cluster 2024-03-04T16:03:29.091431+0000 mgr.y (mgr.24428) 537 : cluster 0 pgmap v825: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:03:30.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:30 smithi151 bash[23756]: audit 2024-03-04T16:03:29.552240+0000 mon.c (mon.2) 730 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:03:32.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:32 smithi151 bash[23756]: cluster 2024-03-04T16:03:31.092276+0000 mgr.y (mgr.24428) 538 : cluster 0 pgmap v826: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:03:32.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:32 smithi117 bash[21480]: cluster 2024-03-04T16:03:31.092276+0000 mgr.y (mgr.24428) 538 : cluster 0 pgmap v826: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:03:32.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:32 smithi117 bash[28843]: cluster 2024-03-04T16:03:31.092276+0000 mgr.y (mgr.24428) 538 : cluster 0 pgmap v826: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:03:34.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:34 smithi151 bash[23756]: cluster 2024-03-04T16:03:33.094573+0000 mgr.y (mgr.24428) 539 : cluster 0 pgmap v827: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:03:34.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:34 smithi117 bash[21480]: cluster 2024-03-04T16:03:33.094573+0000 mgr.y (mgr.24428) 539 : cluster 0 pgmap v827: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:03:34.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:34 smithi117 bash[28843]: cluster 2024-03-04T16:03:33.094573+0000 mgr.y (mgr.24428) 539 : cluster 0 pgmap v827: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:03:35.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:35 smithi151 bash[23756]: audit 2024-03-04T16:03:34.221424+0000 mon.c (mon.2) 731 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:35.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:35 smithi151 bash[23756]: audit 2024-03-04T16:03:34.222207+0000 mon.a (mon.0) 2879 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:35.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:35 smithi151 bash[23756]: audit 2024-03-04T16:03:34.223629+0000 mon.c (mon.2) 732 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-94"}]: dispatch 2024-03-04T16:03:35.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:35 smithi151 bash[23756]: audit 2024-03-04T16:03:34.224389+0000 mon.a (mon.0) 2880 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-94"}]: dispatch 2024-03-04T16:03:35.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:35 smithi117 bash[28843]: audit 2024-03-04T16:03:34.221424+0000 mon.c (mon.2) 731 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:35.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:35 smithi117 bash[28843]: audit 2024-03-04T16:03:34.222207+0000 mon.a (mon.0) 2879 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:35.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:35 smithi117 bash[28843]: audit 2024-03-04T16:03:34.223629+0000 mon.c (mon.2) 732 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-94"}]: dispatch 2024-03-04T16:03:35.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:35 smithi117 bash[28843]: audit 2024-03-04T16:03:34.224389+0000 mon.a (mon.0) 2880 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-94"}]: dispatch 2024-03-04T16:03:35.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:35 smithi117 bash[21480]: audit 2024-03-04T16:03:34.221424+0000 mon.c (mon.2) 731 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:35.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:35 smithi117 bash[21480]: audit 2024-03-04T16:03:34.222207+0000 mon.a (mon.0) 2879 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:35.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:35 smithi117 bash[21480]: audit 2024-03-04T16:03:34.223629+0000 mon.c (mon.2) 732 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-94"}]: dispatch 2024-03-04T16:03:35.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:35 smithi117 bash[21480]: audit 2024-03-04T16:03:34.224389+0000 mon.a (mon.0) 2880 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-94"}]: dispatch 2024-03-04T16:03:36.565 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:03:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:03:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:03:36.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:36 smithi117 bash[21480]: cluster 2024-03-04T16:03:35.095389+0000 mgr.y (mgr.24428) 540 : cluster 0 pgmap v828: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:03:36.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:36 smithi117 bash[21480]: cluster 2024-03-04T16:03:35.178572+0000 mon.a (mon.0) 2881 : cluster 0 osdmap e510: 8 total, 8 up, 8 in 2024-03-04T16:03:36.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:36 smithi117 bash[28843]: cluster 2024-03-04T16:03:35.095389+0000 mgr.y (mgr.24428) 540 : cluster 0 pgmap v828: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:03:36.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:36 smithi117 bash[28843]: cluster 2024-03-04T16:03:35.178572+0000 mon.a (mon.0) 2881 : cluster 0 osdmap e510: 8 total, 8 up, 8 in 2024-03-04T16:03:36.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:36 smithi151 bash[23756]: cluster 2024-03-04T16:03:35.095389+0000 mgr.y (mgr.24428) 540 : cluster 0 pgmap v828: 228 pgs: 228 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:03:36.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:36 smithi151 bash[23756]: cluster 2024-03-04T16:03:35.178572+0000 mon.a (mon.0) 2881 : cluster 0 osdmap e510: 8 total, 8 up, 8 in 2024-03-04T16:03:37.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:03:36 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:03:37.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:37 smithi151 bash[23756]: cluster 2024-03-04T16:03:36.179155+0000 mon.a (mon.0) 2882 : cluster 0 osdmap e511: 8 total, 8 up, 8 in 2024-03-04T16:03:37.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:37 smithi151 bash[23756]: audit 2024-03-04T16:03:36.183237+0000 mon.c (mon.2) 733 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:37.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:37 smithi151 bash[23756]: audit 2024-03-04T16:03:36.188157+0000 mon.a (mon.0) 2883 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:37.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:37 smithi151 bash[23756]: audit 2024-03-04T16:03:37.171247+0000 mon.a (mon.0) 2884 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:03:37.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:37 smithi151 bash[23756]: cluster 2024-03-04T16:03:37.184511+0000 mon.a (mon.0) 2885 : cluster 0 osdmap e512: 8 total, 8 up, 8 in 2024-03-04T16:03:37.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:37 smithi117 bash[21480]: cluster 2024-03-04T16:03:36.179155+0000 mon.a (mon.0) 2882 : cluster 0 osdmap e511: 8 total, 8 up, 8 in 2024-03-04T16:03:37.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:37 smithi117 bash[21480]: audit 2024-03-04T16:03:36.183237+0000 mon.c (mon.2) 733 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:37.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:37 smithi117 bash[21480]: audit 2024-03-04T16:03:36.188157+0000 mon.a (mon.0) 2883 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:37.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:37 smithi117 bash[21480]: audit 2024-03-04T16:03:37.171247+0000 mon.a (mon.0) 2884 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:03:37.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:37 smithi117 bash[21480]: cluster 2024-03-04T16:03:37.184511+0000 mon.a (mon.0) 2885 : cluster 0 osdmap e512: 8 total, 8 up, 8 in 2024-03-04T16:03:37.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:37 smithi117 bash[28843]: cluster 2024-03-04T16:03:36.179155+0000 mon.a (mon.0) 2882 : cluster 0 osdmap e511: 8 total, 8 up, 8 in 2024-03-04T16:03:37.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:37 smithi117 bash[28843]: audit 2024-03-04T16:03:36.183237+0000 mon.c (mon.2) 733 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:37.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:37 smithi117 bash[28843]: audit 2024-03-04T16:03:36.188157+0000 mon.a (mon.0) 2883 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:37.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:37 smithi117 bash[28843]: audit 2024-03-04T16:03:37.171247+0000 mon.a (mon.0) 2884 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:03:37.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:37 smithi117 bash[28843]: cluster 2024-03-04T16:03:37.184511+0000 mon.a (mon.0) 2885 : cluster 0 osdmap e512: 8 total, 8 up, 8 in 2024-03-04T16:03:38.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:38 smithi117 bash[21480]: audit 2024-03-04T16:03:36.999800+0000 mgr.y (mgr.24428) 541 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:03:38.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:38 smithi117 bash[21480]: cluster 2024-03-04T16:03:37.097664+0000 mgr.y (mgr.24428) 542 : cluster 0 pgmap v831: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:03:38.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:38 smithi117 bash[21480]: audit 2024-03-04T16:03:37.186016+0000 mon.c (mon.2) 734 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:03:38.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:38 smithi117 bash[21480]: cluster 2024-03-04T16:03:37.190910+0000 mon.a (mon.0) 2886 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:38.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:38 smithi117 bash[21480]: cluster 2024-03-04T16:03:38.214742+0000 mon.a (mon.0) 2887 : cluster 0 osdmap e513: 8 total, 8 up, 8 in 2024-03-04T16:03:38.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:38 smithi117 bash[28843]: audit 2024-03-04T16:03:36.999800+0000 mgr.y (mgr.24428) 541 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:03:38.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:38 smithi117 bash[28843]: cluster 2024-03-04T16:03:37.097664+0000 mgr.y (mgr.24428) 542 : cluster 0 pgmap v831: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:03:38.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:38 smithi117 bash[28843]: audit 2024-03-04T16:03:37.186016+0000 mon.c (mon.2) 734 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:03:38.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:38 smithi117 bash[28843]: cluster 2024-03-04T16:03:37.190910+0000 mon.a (mon.0) 2886 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:38.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:38 smithi117 bash[28843]: cluster 2024-03-04T16:03:38.214742+0000 mon.a (mon.0) 2887 : cluster 0 osdmap e513: 8 total, 8 up, 8 in 2024-03-04T16:03:38.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:38 smithi151 bash[23756]: audit 2024-03-04T16:03:36.999800+0000 mgr.y (mgr.24428) 541 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:03:38.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:38 smithi151 bash[23756]: cluster 2024-03-04T16:03:37.097664+0000 mgr.y (mgr.24428) 542 : cluster 0 pgmap v831: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:03:38.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:38 smithi151 bash[23756]: audit 2024-03-04T16:03:37.186016+0000 mon.c (mon.2) 734 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:03:38.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:38 smithi151 bash[23756]: cluster 2024-03-04T16:03:37.190910+0000 mon.a (mon.0) 2886 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:38.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:38 smithi151 bash[23756]: cluster 2024-03-04T16:03:38.214742+0000 mon.a (mon.0) 2887 : cluster 0 osdmap e513: 8 total, 8 up, 8 in 2024-03-04T16:03:40.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:40 smithi117 bash[28843]: cluster 2024-03-04T16:03:39.098561+0000 mgr.y (mgr.24428) 543 : cluster 0 pgmap v834: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:03:40.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:40 smithi117 bash[21480]: cluster 2024-03-04T16:03:39.098561+0000 mgr.y (mgr.24428) 543 : cluster 0 pgmap v834: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:03:40.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:40 smithi151 bash[23756]: cluster 2024-03-04T16:03:39.098561+0000 mgr.y (mgr.24428) 543 : cluster 0 pgmap v834: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:03:42.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:42 smithi117 bash[21480]: cluster 2024-03-04T16:03:41.099382+0000 mgr.y (mgr.24428) 544 : cluster 0 pgmap v835: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 863 B/s rd, 172 B/s wr, 1 op/s 2024-03-04T16:03:42.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:42 smithi117 bash[28843]: cluster 2024-03-04T16:03:41.099382+0000 mgr.y (mgr.24428) 544 : cluster 0 pgmap v835: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 863 B/s rd, 172 B/s wr, 1 op/s 2024-03-04T16:03:42.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:42 smithi151 bash[23756]: cluster 2024-03-04T16:03:41.099382+0000 mgr.y (mgr.24428) 544 : cluster 0 pgmap v835: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 651 MiB used, 715 GiB / 715 GiB avail; 863 B/s rd, 172 B/s wr, 1 op/s 2024-03-04T16:03:44.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:44 smithi117 bash[28843]: cluster 2024-03-04T16:03:43.101691+0000 mgr.y (mgr.24428) 545 : cluster 0 pgmap v836: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 886 B/s wr, 3 op/s 2024-03-04T16:03:44.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:44 smithi117 bash[28843]: cluster 2024-03-04T16:03:44.054384+0000 mon.a (mon.0) 2888 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:44.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:44 smithi117 bash[21480]: cluster 2024-03-04T16:03:43.101691+0000 mgr.y (mgr.24428) 545 : cluster 0 pgmap v836: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 886 B/s wr, 3 op/s 2024-03-04T16:03:44.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:44 smithi117 bash[21480]: cluster 2024-03-04T16:03:44.054384+0000 mon.a (mon.0) 2888 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:44.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:44 smithi151 bash[23756]: cluster 2024-03-04T16:03:43.101691+0000 mgr.y (mgr.24428) 545 : cluster 0 pgmap v836: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 886 B/s wr, 3 op/s 2024-03-04T16:03:44.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:44 smithi151 bash[23756]: cluster 2024-03-04T16:03:44.054384+0000 mon.a (mon.0) 2888 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:45.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:45 smithi117 bash[21480]: audit 2024-03-04T16:03:44.585981+0000 mon.c (mon.2) 735 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:03:45.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:45 smithi117 bash[28843]: audit 2024-03-04T16:03:44.585981+0000 mon.c (mon.2) 735 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:03:45.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:45 smithi151 bash[23756]: audit 2024-03-04T16:03:44.585981+0000 mon.c (mon.2) 735 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:03:46.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:46 smithi117 bash[21480]: cluster 2024-03-04T16:03:45.102452+0000 mgr.y (mgr.24428) 546 : cluster 0 pgmap v837: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-03-04T16:03:46.566 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:03:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:03:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:03:46.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:46 smithi117 bash[28843]: cluster 2024-03-04T16:03:45.102452+0000 mgr.y (mgr.24428) 546 : cluster 0 pgmap v837: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-03-04T16:03:46.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:46 smithi151 bash[23756]: cluster 2024-03-04T16:03:45.102452+0000 mgr.y (mgr.24428) 546 : cluster 0 pgmap v837: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 639 B/s wr, 1 op/s 2024-03-04T16:03:47.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:03:47 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:03:48.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:48 smithi117 bash[28843]: audit 2024-03-04T16:03:47.007572+0000 mgr.y (mgr.24428) 547 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:03:48.565 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:48 smithi117 bash[28843]: cluster 2024-03-04T16:03:47.104662+0000 mgr.y (mgr.24428) 548 : cluster 0 pgmap v838: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 515 B/s wr, 2 op/s 2024-03-04T16:03:48.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:48 smithi117 bash[21480]: audit 2024-03-04T16:03:47.007572+0000 mgr.y (mgr.24428) 547 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:03:48.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:48 smithi117 bash[21480]: cluster 2024-03-04T16:03:47.104662+0000 mgr.y (mgr.24428) 548 : cluster 0 pgmap v838: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 515 B/s wr, 2 op/s 2024-03-04T16:03:48.566 INFO:journalctl@ceph.osd.1.smithi117.stdout:Mar 04 16:03:48 smithi117 bash[37600]: debug 2024-03-04T16:03:48.276+0000 7f357511b700 -1 snap_mapper.add_oid found existing snaps mapped on 100:ac821c9c:test-rados-api-smithi117-64061-97::foo:21, removing 2024-03-04T16:03:48.566 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 04 16:03:48 smithi117 bash[48517]: debug 2024-03-04T16:03:48.276+0000 7f720a24d700 -1 snap_mapper.add_oid found existing snaps mapped on 100:ac821c9c:test-rados-api-smithi117-64061-97::foo:21, removing 2024-03-04T16:03:48.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:48 smithi151 bash[23756]: audit 2024-03-04T16:03:47.007572+0000 mgr.y (mgr.24428) 547 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:03:48.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:48 smithi151 bash[23756]: cluster 2024-03-04T16:03:47.104662+0000 mgr.y (mgr.24428) 548 : cluster 0 pgmap v838: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.0 KiB/s rd, 515 B/s wr, 2 op/s 2024-03-04T16:03:48.685 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:03:48 smithi151 bash[43933]: debug 2024-03-04T16:03:48.270+0000 7f00b2472700 -1 snap_mapper.add_oid found existing snaps mapped on 100:ac821c9c:test-rados-api-smithi117-64061-97::foo:21, removing 2024-03-04T16:03:49.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:49 smithi117 bash[21480]: audit 2024-03-04T16:03:48.314796+0000 mon.c (mon.2) 736 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:49.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:49 smithi117 bash[21480]: audit 2024-03-04T16:03:48.315559+0000 mon.a (mon.0) 2889 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:49.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:49 smithi117 bash[21480]: audit 2024-03-04T16:03:48.317287+0000 mon.c (mon.2) 737 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-96"}]: dispatch 2024-03-04T16:03:49.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:49 smithi117 bash[21480]: audit 2024-03-04T16:03:48.317900+0000 mon.a (mon.0) 2890 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-96"}]: dispatch 2024-03-04T16:03:49.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:49 smithi117 bash[28843]: audit 2024-03-04T16:03:48.314796+0000 mon.c (mon.2) 736 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:49.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:49 smithi117 bash[28843]: audit 2024-03-04T16:03:48.315559+0000 mon.a (mon.0) 2889 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:49.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:49 smithi117 bash[28843]: audit 2024-03-04T16:03:48.317287+0000 mon.c (mon.2) 737 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-96"}]: dispatch 2024-03-04T16:03:49.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:49 smithi117 bash[28843]: audit 2024-03-04T16:03:48.317900+0000 mon.a (mon.0) 2890 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-96"}]: dispatch 2024-03-04T16:03:49.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:49 smithi151 bash[23756]: audit 2024-03-04T16:03:48.314796+0000 mon.c (mon.2) 736 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:49 smithi151 bash[23756]: audit 2024-03-04T16:03:48.315559+0000 mon.a (mon.0) 2889 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:49 smithi151 bash[23756]: audit 2024-03-04T16:03:48.317287+0000 mon.c (mon.2) 737 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-96"}]: dispatch 2024-03-04T16:03:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:49 smithi151 bash[23756]: audit 2024-03-04T16:03:48.317900+0000 mon.a (mon.0) 2890 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-96"}]: dispatch 2024-03-04T16:03:50.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:50 smithi117 bash[21480]: cluster 2024-03-04T16:03:49.105521+0000 mgr.y (mgr.24428) 549 : cluster 0 pgmap v839: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 939 B/s rd, 469 B/s wr, 1 op/s 2024-03-04T16:03:50.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:50 smithi117 bash[21480]: cluster 2024-03-04T16:03:49.280691+0000 mon.a (mon.0) 2891 : cluster 0 osdmap e514: 8 total, 8 up, 8 in 2024-03-04T16:03:50.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:50 smithi117 bash[28843]: cluster 2024-03-04T16:03:49.105521+0000 mgr.y (mgr.24428) 549 : cluster 0 pgmap v839: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 939 B/s rd, 469 B/s wr, 1 op/s 2024-03-04T16:03:50.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:50 smithi117 bash[28843]: cluster 2024-03-04T16:03:49.280691+0000 mon.a (mon.0) 2891 : cluster 0 osdmap e514: 8 total, 8 up, 8 in 2024-03-04T16:03:50.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:50 smithi151 bash[23756]: cluster 2024-03-04T16:03:49.105521+0000 mgr.y (mgr.24428) 549 : cluster 0 pgmap v839: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 939 B/s rd, 469 B/s wr, 1 op/s 2024-03-04T16:03:50.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:50 smithi151 bash[23756]: cluster 2024-03-04T16:03:49.280691+0000 mon.a (mon.0) 2891 : cluster 0 osdmap e514: 8 total, 8 up, 8 in 2024-03-04T16:03:51.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:51 smithi117 bash[21480]: cluster 2024-03-04T16:03:50.292954+0000 mon.a (mon.0) 2892 : cluster 0 osdmap e515: 8 total, 8 up, 8 in 2024-03-04T16:03:51.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:51 smithi117 bash[21480]: audit 2024-03-04T16:03:50.306653+0000 mon.c (mon.2) 738 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:51.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:51 smithi117 bash[21480]: audit 2024-03-04T16:03:50.307963+0000 mon.a (mon.0) 2893 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:51.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:51 smithi117 bash[21480]: audit 2024-03-04T16:03:50.745346+0000 mon.c (mon.2) 739 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:03:51.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:51 smithi117 bash[28843]: cluster 2024-03-04T16:03:50.292954+0000 mon.a (mon.0) 2892 : cluster 0 osdmap e515: 8 total, 8 up, 8 in 2024-03-04T16:03:51.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:51 smithi117 bash[28843]: audit 2024-03-04T16:03:50.306653+0000 mon.c (mon.2) 738 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:51.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:51 smithi117 bash[28843]: audit 2024-03-04T16:03:50.307963+0000 mon.a (mon.0) 2893 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:51.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:51 smithi117 bash[28843]: audit 2024-03-04T16:03:50.745346+0000 mon.c (mon.2) 739 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:03:51.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:51 smithi151 bash[23756]: cluster 2024-03-04T16:03:50.292954+0000 mon.a (mon.0) 2892 : cluster 0 osdmap e515: 8 total, 8 up, 8 in 2024-03-04T16:03:51.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:51 smithi151 bash[23756]: audit 2024-03-04T16:03:50.306653+0000 mon.c (mon.2) 738 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:51.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:51 smithi151 bash[23756]: audit 2024-03-04T16:03:50.307963+0000 mon.a (mon.0) 2893 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:03:51.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:51 smithi151 bash[23756]: audit 2024-03-04T16:03:50.745346+0000 mon.c (mon.2) 739 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:03:52.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:52 smithi117 bash[21480]: cluster 2024-03-04T16:03:51.106261+0000 mgr.y (mgr.24428) 550 : cluster 0 pgmap v842: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-04T16:03:52.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:52 smithi117 bash[21480]: cluster 2024-03-04T16:03:51.282806+0000 mon.a (mon.0) 2894 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:52.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:52 smithi117 bash[21480]: audit 2024-03-04T16:03:51.290853+0000 mon.a (mon.0) 2895 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:03:52.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:52 smithi117 bash[21480]: cluster 2024-03-04T16:03:51.297844+0000 mon.a (mon.0) 2896 : cluster 0 osdmap e516: 8 total, 8 up, 8 in 2024-03-04T16:03:52.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:52 smithi117 bash[21480]: audit 2024-03-04T16:03:51.327189+0000 mon.c (mon.2) 740 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:03:52.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:52 smithi117 bash[21480]: audit 2024-03-04T16:03:51.327734+0000 mon.a (mon.0) 2897 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:03:52.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:52 smithi117 bash[21480]: audit 2024-03-04T16:03:51.488906+0000 mon.c (mon.2) 741 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:03:52.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:52 smithi117 bash[21480]: audit 2024-03-04T16:03:51.490579+0000 mon.c (mon.2) 742 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:03:52.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:52 smithi117 bash[21480]: audit 2024-03-04T16:03:51.501053+0000 mon.a (mon.0) 2898 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:03:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:52 smithi117 bash[28843]: cluster 2024-03-04T16:03:51.106261+0000 mgr.y (mgr.24428) 550 : cluster 0 pgmap v842: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-04T16:03:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:52 smithi117 bash[28843]: cluster 2024-03-04T16:03:51.282806+0000 mon.a (mon.0) 2894 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:52 smithi117 bash[28843]: audit 2024-03-04T16:03:51.290853+0000 mon.a (mon.0) 2895 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:03:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:52 smithi117 bash[28843]: cluster 2024-03-04T16:03:51.297844+0000 mon.a (mon.0) 2896 : cluster 0 osdmap e516: 8 total, 8 up, 8 in 2024-03-04T16:03:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:52 smithi117 bash[28843]: audit 2024-03-04T16:03:51.327189+0000 mon.c (mon.2) 740 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:03:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:52 smithi117 bash[28843]: audit 2024-03-04T16:03:51.327734+0000 mon.a (mon.0) 2897 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:03:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:52 smithi117 bash[28843]: audit 2024-03-04T16:03:51.488906+0000 mon.c (mon.2) 741 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:03:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:52 smithi117 bash[28843]: audit 2024-03-04T16:03:51.490579+0000 mon.c (mon.2) 742 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:03:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:52 smithi117 bash[28843]: audit 2024-03-04T16:03:51.501053+0000 mon.a (mon.0) 2898 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:03:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:52 smithi151 bash[23756]: cluster 2024-03-04T16:03:51.106261+0000 mgr.y (mgr.24428) 550 : cluster 0 pgmap v842: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-04T16:03:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:52 smithi151 bash[23756]: cluster 2024-03-04T16:03:51.282806+0000 mon.a (mon.0) 2894 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:52 smithi151 bash[23756]: audit 2024-03-04T16:03:51.290853+0000 mon.a (mon.0) 2895 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:03:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:52 smithi151 bash[23756]: cluster 2024-03-04T16:03:51.297844+0000 mon.a (mon.0) 2896 : cluster 0 osdmap e516: 8 total, 8 up, 8 in 2024-03-04T16:03:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:52 smithi151 bash[23756]: audit 2024-03-04T16:03:51.327189+0000 mon.c (mon.2) 740 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:03:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:52 smithi151 bash[23756]: audit 2024-03-04T16:03:51.327734+0000 mon.a (mon.0) 2897 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:03:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:52 smithi151 bash[23756]: audit 2024-03-04T16:03:51.488906+0000 mon.c (mon.2) 741 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:03:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:52 smithi151 bash[23756]: audit 2024-03-04T16:03:51.490579+0000 mon.c (mon.2) 742 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:03:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:52 smithi151 bash[23756]: audit 2024-03-04T16:03:51.501053+0000 mon.a (mon.0) 2898 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:03:53.565 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:53 smithi117 bash[21480]: audit 2024-03-04T16:03:52.301863+0000 mon.a (mon.0) 2899 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-98", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:03:53.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:53 smithi117 bash[21480]: cluster 2024-03-04T16:03:52.309884+0000 mon.a (mon.0) 2900 : cluster 0 osdmap e517: 8 total, 8 up, 8 in 2024-03-04T16:03:53.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:53 smithi117 bash[21480]: audit 2024-03-04T16:03:52.312491+0000 mon.c (mon.2) 743 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-98", "mode": "writeback"}]: dispatch 2024-03-04T16:03:53.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:53 smithi117 bash[21480]: audit 2024-03-04T16:03:52.313218+0000 mon.a (mon.0) 2901 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-98", "mode": "writeback"}]: dispatch 2024-03-04T16:03:53.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:53 smithi117 bash[28843]: audit 2024-03-04T16:03:52.301863+0000 mon.a (mon.0) 2899 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-98", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:03:53.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:53 smithi117 bash[28843]: cluster 2024-03-04T16:03:52.309884+0000 mon.a (mon.0) 2900 : cluster 0 osdmap e517: 8 total, 8 up, 8 in 2024-03-04T16:03:53.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:53 smithi117 bash[28843]: audit 2024-03-04T16:03:52.312491+0000 mon.c (mon.2) 743 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-98", "mode": "writeback"}]: dispatch 2024-03-04T16:03:53.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:53 smithi117 bash[28843]: audit 2024-03-04T16:03:52.313218+0000 mon.a (mon.0) 2901 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-98", "mode": "writeback"}]: dispatch 2024-03-04T16:03:53.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:53 smithi151 bash[23756]: audit 2024-03-04T16:03:52.301863+0000 mon.a (mon.0) 2899 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-98", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:03:53.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:53 smithi151 bash[23756]: cluster 2024-03-04T16:03:52.309884+0000 mon.a (mon.0) 2900 : cluster 0 osdmap e517: 8 total, 8 up, 8 in 2024-03-04T16:03:53.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:53 smithi151 bash[23756]: audit 2024-03-04T16:03:52.312491+0000 mon.c (mon.2) 743 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-98", "mode": "writeback"}]: dispatch 2024-03-04T16:03:53.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:53 smithi151 bash[23756]: audit 2024-03-04T16:03:52.313218+0000 mon.a (mon.0) 2901 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-98", "mode": "writeback"}]: dispatch 2024-03-04T16:03:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:54 smithi151 bash[23756]: cluster 2024-03-04T16:03:53.107064+0000 mgr.y (mgr.24428) 551 : cluster 0 pgmap v845: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-03-04T16:03:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:54 smithi151 bash[23756]: cluster 2024-03-04T16:03:53.302775+0000 mon.a (mon.0) 2902 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:03:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:54 smithi151 bash[23756]: audit 2024-03-04T16:03:53.312884+0000 mon.a (mon.0) 2903 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-98", "mode": "writeback"}]': finished 2024-03-04T16:03:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:54 smithi151 bash[23756]: cluster 2024-03-04T16:03:53.321820+0000 mon.a (mon.0) 2904 : cluster 0 osdmap e518: 8 total, 8 up, 8 in 2024-03-04T16:03:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:54 smithi151 bash[23756]: audit 2024-03-04T16:03:53.324592+0000 mon.c (mon.2) 744 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-98"}]: dispatch 2024-03-04T16:03:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:54 smithi151 bash[23756]: audit 2024-03-04T16:03:53.325706+0000 mon.a (mon.0) 2905 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-98"}]: dispatch 2024-03-04T16:03:54.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:54 smithi117 bash[28843]: cluster 2024-03-04T16:03:53.107064+0000 mgr.y (mgr.24428) 551 : cluster 0 pgmap v845: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-03-04T16:03:54.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:54 smithi117 bash[28843]: cluster 2024-03-04T16:03:53.302775+0000 mon.a (mon.0) 2902 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:03:54.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:54 smithi117 bash[28843]: audit 2024-03-04T16:03:53.312884+0000 mon.a (mon.0) 2903 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-98", "mode": "writeback"}]': finished 2024-03-04T16:03:54.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:54 smithi117 bash[28843]: cluster 2024-03-04T16:03:53.321820+0000 mon.a (mon.0) 2904 : cluster 0 osdmap e518: 8 total, 8 up, 8 in 2024-03-04T16:03:54.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:54 smithi117 bash[28843]: audit 2024-03-04T16:03:53.324592+0000 mon.c (mon.2) 744 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-98"}]: dispatch 2024-03-04T16:03:54.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:54 smithi117 bash[28843]: audit 2024-03-04T16:03:53.325706+0000 mon.a (mon.0) 2905 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-98"}]: dispatch 2024-03-04T16:03:54.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:54 smithi117 bash[21480]: cluster 2024-03-04T16:03:53.107064+0000 mgr.y (mgr.24428) 551 : cluster 0 pgmap v845: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-03-04T16:03:54.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:54 smithi117 bash[21480]: cluster 2024-03-04T16:03:53.302775+0000 mon.a (mon.0) 2902 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:03:54.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:54 smithi117 bash[21480]: audit 2024-03-04T16:03:53.312884+0000 mon.a (mon.0) 2903 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-98", "mode": "writeback"}]': finished 2024-03-04T16:03:54.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:54 smithi117 bash[21480]: cluster 2024-03-04T16:03:53.321820+0000 mon.a (mon.0) 2904 : cluster 0 osdmap e518: 8 total, 8 up, 8 in 2024-03-04T16:03:54.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:54 smithi117 bash[21480]: audit 2024-03-04T16:03:53.324592+0000 mon.c (mon.2) 744 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-98"}]: dispatch 2024-03-04T16:03:54.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:54 smithi117 bash[21480]: audit 2024-03-04T16:03:53.325706+0000 mon.a (mon.0) 2905 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-98"}]: dispatch 2024-03-04T16:03:55.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:55 smithi151 bash[23756]: audit 2024-03-04T16:03:54.315690+0000 mon.a (mon.0) 2906 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-98"}]': finished 2024-03-04T16:03:55.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:55 smithi151 bash[23756]: cluster 2024-03-04T16:03:54.322846+0000 mon.a (mon.0) 2907 : cluster 0 osdmap e519: 8 total, 8 up, 8 in 2024-03-04T16:03:55.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:55 smithi151 bash[23756]: audit 2024-03-04T16:03:54.326324+0000 mon.c (mon.2) 745 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T16:03:55.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:55 smithi151 bash[23756]: audit 2024-03-04T16:03:54.340726+0000 mon.a (mon.0) 2908 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T16:03:55.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:55 smithi117 bash[21480]: audit 2024-03-04T16:03:54.315690+0000 mon.a (mon.0) 2906 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-98"}]': finished 2024-03-04T16:03:55.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:55 smithi117 bash[21480]: cluster 2024-03-04T16:03:54.322846+0000 mon.a (mon.0) 2907 : cluster 0 osdmap e519: 8 total, 8 up, 8 in 2024-03-04T16:03:55.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:55 smithi117 bash[21480]: audit 2024-03-04T16:03:54.326324+0000 mon.c (mon.2) 745 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T16:03:55.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:55 smithi117 bash[21480]: audit 2024-03-04T16:03:54.340726+0000 mon.a (mon.0) 2908 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T16:03:55.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:55 smithi117 bash[28843]: audit 2024-03-04T16:03:54.315690+0000 mon.a (mon.0) 2906 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-98"}]': finished 2024-03-04T16:03:55.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:55 smithi117 bash[28843]: cluster 2024-03-04T16:03:54.322846+0000 mon.a (mon.0) 2907 : cluster 0 osdmap e519: 8 total, 8 up, 8 in 2024-03-04T16:03:55.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:55 smithi117 bash[28843]: audit 2024-03-04T16:03:54.326324+0000 mon.c (mon.2) 745 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T16:03:55.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:55 smithi117 bash[28843]: audit 2024-03-04T16:03:54.340726+0000 mon.a (mon.0) 2908 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T16:03:56.565 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:03:56 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:03:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:03:56.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:56 smithi117 bash[21480]: cluster 2024-03-04T16:03:55.107927+0000 mgr.y (mgr.24428) 552 : cluster 0 pgmap v848: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T16:03:56.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:56 smithi117 bash[21480]: cluster 2024-03-04T16:03:55.328381+0000 mon.a (mon.0) 2909 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:03:56.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:56 smithi117 bash[21480]: audit 2024-03-04T16:03:55.338711+0000 mon.a (mon.0) 2910 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T16:03:56.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:56 smithi117 bash[21480]: cluster 2024-03-04T16:03:55.352272+0000 mon.a (mon.0) 2911 : cluster 0 osdmap e520: 8 total, 8 up, 8 in 2024-03-04T16:03:56.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:56 smithi117 bash[21480]: audit 2024-03-04T16:03:55.354289+0000 mon.c (mon.2) 746 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-04T16:03:56.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:56 smithi117 bash[21480]: audit 2024-03-04T16:03:55.356034+0000 mon.a (mon.0) 2912 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-04T16:03:56.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:56 smithi117 bash[28843]: cluster 2024-03-04T16:03:55.107927+0000 mgr.y (mgr.24428) 552 : cluster 0 pgmap v848: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T16:03:56.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:56 smithi117 bash[28843]: cluster 2024-03-04T16:03:55.328381+0000 mon.a (mon.0) 2909 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:03:56.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:56 smithi117 bash[28843]: audit 2024-03-04T16:03:55.338711+0000 mon.a (mon.0) 2910 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T16:03:56.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:56 smithi117 bash[28843]: cluster 2024-03-04T16:03:55.352272+0000 mon.a (mon.0) 2911 : cluster 0 osdmap e520: 8 total, 8 up, 8 in 2024-03-04T16:03:56.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:56 smithi117 bash[28843]: audit 2024-03-04T16:03:55.354289+0000 mon.c (mon.2) 746 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-04T16:03:56.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:56 smithi117 bash[28843]: audit 2024-03-04T16:03:55.356034+0000 mon.a (mon.0) 2912 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-04T16:03:56.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:56 smithi151 bash[23756]: cluster 2024-03-04T16:03:55.107927+0000 mgr.y (mgr.24428) 552 : cluster 0 pgmap v848: 228 pgs: 228 active+clean; 8.4 MiB data, 656 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 2 op/s 2024-03-04T16:03:56.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:56 smithi151 bash[23756]: cluster 2024-03-04T16:03:55.328381+0000 mon.a (mon.0) 2909 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:03:56.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:56 smithi151 bash[23756]: audit 2024-03-04T16:03:55.338711+0000 mon.a (mon.0) 2910 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T16:03:56.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:56 smithi151 bash[23756]: cluster 2024-03-04T16:03:55.352272+0000 mon.a (mon.0) 2911 : cluster 0 osdmap e520: 8 total, 8 up, 8 in 2024-03-04T16:03:56.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:56 smithi151 bash[23756]: audit 2024-03-04T16:03:55.354289+0000 mon.c (mon.2) 746 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-04T16:03:56.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:56 smithi151 bash[23756]: audit 2024-03-04T16:03:55.356034+0000 mon.a (mon.0) 2912 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_count","val": "1"}]: dispatch 2024-03-04T16:03:57.363 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:03:57 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:03:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:57 smithi151 bash[23756]: audit 2024-03-04T16:03:56.349771+0000 mon.a (mon.0) 2913 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_count","val": "1"}]': finished 2024-03-04T16:03:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:57 smithi151 bash[23756]: cluster 2024-03-04T16:03:56.356133+0000 mon.a (mon.0) 2914 : cluster 0 osdmap e521: 8 total, 8 up, 8 in 2024-03-04T16:03:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:57 smithi151 bash[23756]: audit 2024-03-04T16:03:56.363588+0000 mon.c (mon.2) 747 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:03:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:57 smithi151 bash[23756]: audit 2024-03-04T16:03:56.364603+0000 mon.a (mon.0) 2915 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:03:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:57 smithi151 bash[23756]: audit 2024-03-04T16:03:57.017237+0000 mgr.y (mgr.24428) 553 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:03:57.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:57 smithi151 bash[23756]: cluster 2024-03-04T16:03:57.110181+0000 mgr.y (mgr.24428) 554 : cluster 0 pgmap v851: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:03:57.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:57 smithi117 bash[21480]: audit 2024-03-04T16:03:56.349771+0000 mon.a (mon.0) 2913 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_count","val": "1"}]': finished 2024-03-04T16:03:57.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:57 smithi117 bash[21480]: cluster 2024-03-04T16:03:56.356133+0000 mon.a (mon.0) 2914 : cluster 0 osdmap e521: 8 total, 8 up, 8 in 2024-03-04T16:03:57.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:57 smithi117 bash[21480]: audit 2024-03-04T16:03:56.363588+0000 mon.c (mon.2) 747 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:03:57.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:57 smithi117 bash[21480]: audit 2024-03-04T16:03:56.364603+0000 mon.a (mon.0) 2915 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:03:57.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:57 smithi117 bash[21480]: audit 2024-03-04T16:03:57.017237+0000 mgr.y (mgr.24428) 553 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:03:57.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:57 smithi117 bash[21480]: cluster 2024-03-04T16:03:57.110181+0000 mgr.y (mgr.24428) 554 : cluster 0 pgmap v851: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:03:57.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:57 smithi117 bash[28843]: audit 2024-03-04T16:03:56.349771+0000 mon.a (mon.0) 2913 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_count","val": "1"}]': finished 2024-03-04T16:03:57.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:57 smithi117 bash[28843]: cluster 2024-03-04T16:03:56.356133+0000 mon.a (mon.0) 2914 : cluster 0 osdmap e521: 8 total, 8 up, 8 in 2024-03-04T16:03:57.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:57 smithi117 bash[28843]: audit 2024-03-04T16:03:56.363588+0000 mon.c (mon.2) 747 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:03:57.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:57 smithi117 bash[28843]: audit 2024-03-04T16:03:56.364603+0000 mon.a (mon.0) 2915 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:03:57.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:57 smithi117 bash[28843]: audit 2024-03-04T16:03:57.017237+0000 mgr.y (mgr.24428) 553 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:03:57.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:57 smithi117 bash[28843]: cluster 2024-03-04T16:03:57.110181+0000 mgr.y (mgr.24428) 554 : cluster 0 pgmap v851: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:03:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:58 smithi151 bash[23756]: audit 2024-03-04T16:03:57.352410+0000 mon.a (mon.0) 2916 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_period","val": "600"}]': finished 2024-03-04T16:03:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:58 smithi151 bash[23756]: cluster 2024-03-04T16:03:57.361125+0000 mon.a (mon.0) 2917 : cluster 0 osdmap e522: 8 total, 8 up, 8 in 2024-03-04T16:03:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:58 smithi151 bash[23756]: audit 2024-03-04T16:03:57.363830+0000 mon.c (mon.2) 748 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-04T16:03:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:58 smithi151 bash[23756]: audit 2024-03-04T16:03:57.365494+0000 mon.a (mon.0) 2918 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-04T16:03:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:58 smithi151 bash[23756]: audit 2024-03-04T16:03:58.355275+0000 mon.a (mon.0) 2919 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "target_max_objects","val": "250"}]': finished 2024-03-04T16:03:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:58 smithi151 bash[23756]: cluster 2024-03-04T16:03:58.363126+0000 mon.a (mon.0) 2920 : cluster 0 osdmap e523: 8 total, 8 up, 8 in 2024-03-04T16:03:58.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:58 smithi117 bash[21480]: audit 2024-03-04T16:03:57.352410+0000 mon.a (mon.0) 2916 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_period","val": "600"}]': finished 2024-03-04T16:03:58.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:58 smithi117 bash[21480]: cluster 2024-03-04T16:03:57.361125+0000 mon.a (mon.0) 2917 : cluster 0 osdmap e522: 8 total, 8 up, 8 in 2024-03-04T16:03:58.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:58 smithi117 bash[21480]: audit 2024-03-04T16:03:57.363830+0000 mon.c (mon.2) 748 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-04T16:03:58.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:58 smithi117 bash[21480]: audit 2024-03-04T16:03:57.365494+0000 mon.a (mon.0) 2918 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-04T16:03:58.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:58 smithi117 bash[21480]: audit 2024-03-04T16:03:58.355275+0000 mon.a (mon.0) 2919 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "target_max_objects","val": "250"}]': finished 2024-03-04T16:03:58.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:58 smithi117 bash[21480]: cluster 2024-03-04T16:03:58.363126+0000 mon.a (mon.0) 2920 : cluster 0 osdmap e523: 8 total, 8 up, 8 in 2024-03-04T16:03:58.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:58 smithi117 bash[28843]: audit 2024-03-04T16:03:57.352410+0000 mon.a (mon.0) 2916 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "hit_set_period","val": "600"}]': finished 2024-03-04T16:03:58.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:58 smithi117 bash[28843]: cluster 2024-03-04T16:03:57.361125+0000 mon.a (mon.0) 2917 : cluster 0 osdmap e522: 8 total, 8 up, 8 in 2024-03-04T16:03:58.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:58 smithi117 bash[28843]: audit 2024-03-04T16:03:57.363830+0000 mon.c (mon.2) 748 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-04T16:03:58.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:58 smithi117 bash[28843]: audit 2024-03-04T16:03:57.365494+0000 mon.a (mon.0) 2918 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "target_max_objects","val": "250"}]: dispatch 2024-03-04T16:03:58.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:58 smithi117 bash[28843]: audit 2024-03-04T16:03:58.355275+0000 mon.a (mon.0) 2919 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-98","var": "target_max_objects","val": "250"}]': finished 2024-03-04T16:03:58.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:58 smithi117 bash[28843]: cluster 2024-03-04T16:03:58.363126+0000 mon.a (mon.0) 2920 : cluster 0 osdmap e523: 8 total, 8 up, 8 in 2024-03-04T16:03:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:59 smithi151 bash[23756]: audit 2024-03-04T16:03:58.439459+0000 mon.c (mon.2) 749 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:59 smithi151 bash[23756]: audit 2024-03-04T16:03:58.440350+0000 mon.a (mon.0) 2921 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:59 smithi151 bash[23756]: cluster 2024-03-04T16:03:59.057999+0000 mon.a (mon.0) 2922 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:03:59 smithi151 bash[23756]: cluster 2024-03-04T16:03:59.111027+0000 mgr.y (mgr.24428) 555 : cluster 0 pgmap v854: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:03:59.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:59 smithi117 bash[21480]: audit 2024-03-04T16:03:58.439459+0000 mon.c (mon.2) 749 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:59.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:59 smithi117 bash[21480]: audit 2024-03-04T16:03:58.440350+0000 mon.a (mon.0) 2921 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:59.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:59 smithi117 bash[21480]: cluster 2024-03-04T16:03:59.057999+0000 mon.a (mon.0) 2922 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:59.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:03:59 smithi117 bash[21480]: cluster 2024-03-04T16:03:59.111027+0000 mgr.y (mgr.24428) 555 : cluster 0 pgmap v854: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:03:59.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:59 smithi117 bash[28843]: audit 2024-03-04T16:03:58.439459+0000 mon.c (mon.2) 749 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:59.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:59 smithi117 bash[28843]: audit 2024-03-04T16:03:58.440350+0000 mon.a (mon.0) 2921 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:03:59.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:59 smithi117 bash[28843]: cluster 2024-03-04T16:03:59.057999+0000 mon.a (mon.0) 2922 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:03:59.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:03:59 smithi117 bash[28843]: cluster 2024-03-04T16:03:59.111027+0000 mgr.y (mgr.24428) 555 : cluster 0 pgmap v854: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:00 smithi151 bash[23756]: audit 2024-03-04T16:03:59.385256+0000 mon.a (mon.0) 2923 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T16:04:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:00 smithi151 bash[23756]: cluster 2024-03-04T16:03:59.391922+0000 mon.a (mon.0) 2924 : cluster 0 osdmap e524: 8 total, 8 up, 8 in 2024-03-04T16:04:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:00 smithi151 bash[23756]: audit 2024-03-04T16:03:59.395372+0000 mon.c (mon.2) 750 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-98"}]: dispatch 2024-03-04T16:04:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:00 smithi151 bash[23756]: audit 2024-03-04T16:03:59.397082+0000 mon.a (mon.0) 2925 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-98"}]: dispatch 2024-03-04T16:04:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:00 smithi151 bash[23756]: audit 2024-03-04T16:03:59.620666+0000 mon.c (mon.2) 751 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:04:00.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:00 smithi117 bash[21480]: audit 2024-03-04T16:03:59.385256+0000 mon.a (mon.0) 2923 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T16:04:00.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:00 smithi117 bash[21480]: cluster 2024-03-04T16:03:59.391922+0000 mon.a (mon.0) 2924 : cluster 0 osdmap e524: 8 total, 8 up, 8 in 2024-03-04T16:04:00.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:00 smithi117 bash[21480]: audit 2024-03-04T16:03:59.395372+0000 mon.c (mon.2) 750 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-98"}]: dispatch 2024-03-04T16:04:00.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:00 smithi117 bash[21480]: audit 2024-03-04T16:03:59.397082+0000 mon.a (mon.0) 2925 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-98"}]: dispatch 2024-03-04T16:04:00.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:00 smithi117 bash[21480]: audit 2024-03-04T16:03:59.620666+0000 mon.c (mon.2) 751 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:04:00.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:00 smithi117 bash[28843]: audit 2024-03-04T16:03:59.385256+0000 mon.a (mon.0) 2923 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T16:04:00.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:00 smithi117 bash[28843]: cluster 2024-03-04T16:03:59.391922+0000 mon.a (mon.0) 2924 : cluster 0 osdmap e524: 8 total, 8 up, 8 in 2024-03-04T16:04:00.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:00 smithi117 bash[28843]: audit 2024-03-04T16:03:59.395372+0000 mon.c (mon.2) 750 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-98"}]: dispatch 2024-03-04T16:04:00.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:00 smithi117 bash[28843]: audit 2024-03-04T16:03:59.397082+0000 mon.a (mon.0) 2925 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-98"}]: dispatch 2024-03-04T16:04:00.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:00 smithi117 bash[28843]: audit 2024-03-04T16:03:59.620666+0000 mon.c (mon.2) 751 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:04:01.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:01 smithi151 bash[23756]: audit 2024-03-04T16:04:00.388070+0000 mon.a (mon.0) 2926 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-98"}]': finished 2024-03-04T16:04:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:01 smithi151 bash[23756]: cluster 2024-03-04T16:04:00.396502+0000 mon.a (mon.0) 2927 : cluster 0 osdmap e525: 8 total, 8 up, 8 in 2024-03-04T16:04:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:01 smithi151 bash[23756]: cluster 2024-03-04T16:04:01.111831+0000 mgr.y (mgr.24428) 556 : cluster 0 pgmap v857: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:01.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:01 smithi117 bash[21480]: audit 2024-03-04T16:04:00.388070+0000 mon.a (mon.0) 2926 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-98"}]': finished 2024-03-04T16:04:01.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:01 smithi117 bash[21480]: cluster 2024-03-04T16:04:00.396502+0000 mon.a (mon.0) 2927 : cluster 0 osdmap e525: 8 total, 8 up, 8 in 2024-03-04T16:04:01.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:01 smithi117 bash[21480]: cluster 2024-03-04T16:04:01.111831+0000 mgr.y (mgr.24428) 556 : cluster 0 pgmap v857: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:01.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:01 smithi117 bash[28843]: audit 2024-03-04T16:04:00.388070+0000 mon.a (mon.0) 2926 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-98"}]': finished 2024-03-04T16:04:01.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:01 smithi117 bash[28843]: cluster 2024-03-04T16:04:00.396502+0000 mon.a (mon.0) 2927 : cluster 0 osdmap e525: 8 total, 8 up, 8 in 2024-03-04T16:04:01.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:01 smithi117 bash[28843]: cluster 2024-03-04T16:04:01.111831+0000 mgr.y (mgr.24428) 556 : cluster 0 pgmap v857: 228 pgs: 228 active+clean; 8.4 MiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:02.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:02 smithi117 bash[21480]: cluster 2024-03-04T16:04:01.422129+0000 mon.a (mon.0) 2928 : cluster 0 osdmap e526: 8 total, 8 up, 8 in 2024-03-04T16:04:02.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:02 smithi117 bash[28843]: cluster 2024-03-04T16:04:01.422129+0000 mon.a (mon.0) 2928 : cluster 0 osdmap e526: 8 total, 8 up, 8 in 2024-03-04T16:04:02.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:02 smithi151 bash[23756]: cluster 2024-03-04T16:04:01.422129+0000 mon.a (mon.0) 2928 : cluster 0 osdmap e526: 8 total, 8 up, 8 in 2024-03-04T16:04:03.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:03 smithi117 bash[21480]: cluster 2024-03-04T16:04:02.430784+0000 mon.a (mon.0) 2929 : cluster 0 osdmap e527: 8 total, 8 up, 8 in 2024-03-04T16:04:03.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:03 smithi117 bash[21480]: audit 2024-03-04T16:04:02.451338+0000 mon.c (mon.2) 752 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:03.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:03 smithi117 bash[21480]: audit 2024-03-04T16:04:02.461883+0000 mon.a (mon.0) 2930 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:03.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:03 smithi117 bash[21480]: cluster 2024-03-04T16:04:03.112789+0000 mgr.y (mgr.24428) 557 : cluster 0 pgmap v860: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:03.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:03 smithi117 bash[21480]: audit 2024-03-04T16:04:03.421527+0000 mon.a (mon.0) 2931 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:04:03.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:03 smithi117 bash[21480]: cluster 2024-03-04T16:04:03.430152+0000 mon.a (mon.0) 2932 : cluster 0 osdmap e528: 8 total, 8 up, 8 in 2024-03-04T16:04:03.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:03 smithi117 bash[21480]: audit 2024-03-04T16:04:03.441908+0000 mon.a (mon.0) 2933 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:04:03.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:03 smithi117 bash[28843]: cluster 2024-03-04T16:04:02.430784+0000 mon.a (mon.0) 2929 : cluster 0 osdmap e527: 8 total, 8 up, 8 in 2024-03-04T16:04:03.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:03 smithi117 bash[28843]: audit 2024-03-04T16:04:02.451338+0000 mon.c (mon.2) 752 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:03.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:03 smithi117 bash[28843]: audit 2024-03-04T16:04:02.461883+0000 mon.a (mon.0) 2930 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:03.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:03 smithi117 bash[28843]: cluster 2024-03-04T16:04:03.112789+0000 mgr.y (mgr.24428) 557 : cluster 0 pgmap v860: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:03.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:03 smithi117 bash[28843]: audit 2024-03-04T16:04:03.421527+0000 mon.a (mon.0) 2931 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:04:03.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:03 smithi117 bash[28843]: cluster 2024-03-04T16:04:03.430152+0000 mon.a (mon.0) 2932 : cluster 0 osdmap e528: 8 total, 8 up, 8 in 2024-03-04T16:04:03.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:03 smithi117 bash[28843]: audit 2024-03-04T16:04:03.441908+0000 mon.a (mon.0) 2933 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:04:03.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:03 smithi151 bash[23756]: cluster 2024-03-04T16:04:02.430784+0000 mon.a (mon.0) 2929 : cluster 0 osdmap e527: 8 total, 8 up, 8 in 2024-03-04T16:04:03.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:03 smithi151 bash[23756]: audit 2024-03-04T16:04:02.451338+0000 mon.c (mon.2) 752 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:03.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:03 smithi151 bash[23756]: audit 2024-03-04T16:04:02.461883+0000 mon.a (mon.0) 2930 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:03.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:03 smithi151 bash[23756]: cluster 2024-03-04T16:04:03.112789+0000 mgr.y (mgr.24428) 557 : cluster 0 pgmap v860: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:03.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:03 smithi151 bash[23756]: audit 2024-03-04T16:04:03.421527+0000 mon.a (mon.0) 2931 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:04:03.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:03 smithi151 bash[23756]: cluster 2024-03-04T16:04:03.430152+0000 mon.a (mon.0) 2932 : cluster 0 osdmap e528: 8 total, 8 up, 8 in 2024-03-04T16:04:03.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:03 smithi151 bash[23756]: audit 2024-03-04T16:04:03.441908+0000 mon.a (mon.0) 2933 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:04:04.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:04 smithi117 bash[21480]: audit 2024-03-04T16:04:03.440692+0000 mon.c (mon.2) 753 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:04:04.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:04 smithi117 bash[21480]: cluster 2024-03-04T16:04:04.059185+0000 mon.a (mon.0) 2934 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:04.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:04 smithi117 bash[21480]: audit 2024-03-04T16:04:04.424235+0000 mon.a (mon.0) 2935 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-100", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:04:04.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:04 smithi117 bash[21480]: cluster 2024-03-04T16:04:04.432243+0000 mon.a (mon.0) 2936 : cluster 0 osdmap e529: 8 total, 8 up, 8 in 2024-03-04T16:04:04.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:04 smithi117 bash[21480]: audit 2024-03-04T16:04:04.435170+0000 mon.c (mon.2) 754 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-100"}]: dispatch 2024-03-04T16:04:04.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:04 smithi117 bash[21480]: audit 2024-03-04T16:04:04.436310+0000 mon.a (mon.0) 2937 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-100"}]: dispatch 2024-03-04T16:04:04.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:04 smithi117 bash[28843]: audit 2024-03-04T16:04:03.440692+0000 mon.c (mon.2) 753 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:04:04.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:04 smithi117 bash[28843]: cluster 2024-03-04T16:04:04.059185+0000 mon.a (mon.0) 2934 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:04.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:04 smithi117 bash[28843]: audit 2024-03-04T16:04:04.424235+0000 mon.a (mon.0) 2935 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-100", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:04:04.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:04 smithi117 bash[28843]: cluster 2024-03-04T16:04:04.432243+0000 mon.a (mon.0) 2936 : cluster 0 osdmap e529: 8 total, 8 up, 8 in 2024-03-04T16:04:04.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:04 smithi117 bash[28843]: audit 2024-03-04T16:04:04.435170+0000 mon.c (mon.2) 754 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-100"}]: dispatch 2024-03-04T16:04:04.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:04 smithi117 bash[28843]: audit 2024-03-04T16:04:04.436310+0000 mon.a (mon.0) 2937 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-100"}]: dispatch 2024-03-04T16:04:04.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:04 smithi151 bash[23756]: audit 2024-03-04T16:04:03.440692+0000 mon.c (mon.2) 753 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:04:04.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:04 smithi151 bash[23756]: cluster 2024-03-04T16:04:04.059185+0000 mon.a (mon.0) 2934 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:04.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:04 smithi151 bash[23756]: audit 2024-03-04T16:04:04.424235+0000 mon.a (mon.0) 2935 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-100", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:04:04.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:04 smithi151 bash[23756]: cluster 2024-03-04T16:04:04.432243+0000 mon.a (mon.0) 2936 : cluster 0 osdmap e529: 8 total, 8 up, 8 in 2024-03-04T16:04:04.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:04 smithi151 bash[23756]: audit 2024-03-04T16:04:04.435170+0000 mon.c (mon.2) 754 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-100"}]: dispatch 2024-03-04T16:04:04.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:04 smithi151 bash[23756]: audit 2024-03-04T16:04:04.436310+0000 mon.a (mon.0) 2937 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-100"}]: dispatch 2024-03-04T16:04:05.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:05 smithi117 bash[28843]: cluster 2024-03-04T16:04:05.113670+0000 mgr.y (mgr.24428) 558 : cluster 0 pgmap v863: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:05.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:05 smithi117 bash[28843]: audit 2024-03-04T16:04:05.427026+0000 mon.a (mon.0) 2938 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-100"}]': finished 2024-03-04T16:04:05.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:05 smithi117 bash[28843]: cluster 2024-03-04T16:04:05.434291+0000 mon.a (mon.0) 2939 : cluster 0 osdmap e530: 8 total, 8 up, 8 in 2024-03-04T16:04:05.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:05 smithi117 bash[28843]: audit 2024-03-04T16:04:05.435884+0000 mon.c (mon.2) 755 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-100", "mode": "writeback"}]: dispatch 2024-03-04T16:04:05.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:05 smithi117 bash[28843]: audit 2024-03-04T16:04:05.438745+0000 mon.a (mon.0) 2940 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-100", "mode": "writeback"}]: dispatch 2024-03-04T16:04:05.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:05 smithi117 bash[21480]: cluster 2024-03-04T16:04:05.113670+0000 mgr.y (mgr.24428) 558 : cluster 0 pgmap v863: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:05.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:05 smithi117 bash[21480]: audit 2024-03-04T16:04:05.427026+0000 mon.a (mon.0) 2938 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-100"}]': finished 2024-03-04T16:04:05.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:05 smithi117 bash[21480]: cluster 2024-03-04T16:04:05.434291+0000 mon.a (mon.0) 2939 : cluster 0 osdmap e530: 8 total, 8 up, 8 in 2024-03-04T16:04:05.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:05 smithi117 bash[21480]: audit 2024-03-04T16:04:05.435884+0000 mon.c (mon.2) 755 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-100", "mode": "writeback"}]: dispatch 2024-03-04T16:04:05.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:05 smithi117 bash[21480]: audit 2024-03-04T16:04:05.438745+0000 mon.a (mon.0) 2940 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-100", "mode": "writeback"}]: dispatch 2024-03-04T16:04:05.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:05 smithi151 bash[23756]: cluster 2024-03-04T16:04:05.113670+0000 mgr.y (mgr.24428) 558 : cluster 0 pgmap v863: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 661 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:05.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:05 smithi151 bash[23756]: audit 2024-03-04T16:04:05.427026+0000 mon.a (mon.0) 2938 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-6", "overlaypool": "test-rados-api-smithi117-64061-100"}]': finished 2024-03-04T16:04:05.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:05 smithi151 bash[23756]: cluster 2024-03-04T16:04:05.434291+0000 mon.a (mon.0) 2939 : cluster 0 osdmap e530: 8 total, 8 up, 8 in 2024-03-04T16:04:05.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:05 smithi151 bash[23756]: audit 2024-03-04T16:04:05.435884+0000 mon.c (mon.2) 755 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-100", "mode": "writeback"}]: dispatch 2024-03-04T16:04:05.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:05 smithi151 bash[23756]: audit 2024-03-04T16:04:05.438745+0000 mon.a (mon.0) 2940 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-100", "mode": "writeback"}]: dispatch 2024-03-04T16:04:06.457 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:04:06 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:04:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:04:06.815 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:06 smithi117 bash[28843]: cluster 2024-03-04T16:04:06.427264+0000 mon.a (mon.0) 2941 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:04:06.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:06 smithi117 bash[28843]: audit 2024-03-04T16:04:06.429649+0000 mon.a (mon.0) 2942 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-100", "mode": "writeback"}]': finished 2024-03-04T16:04:06.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:06 smithi117 bash[28843]: cluster 2024-03-04T16:04:06.437010+0000 mon.a (mon.0) 2943 : cluster 0 osdmap e531: 8 total, 8 up, 8 in 2024-03-04T16:04:06.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:06 smithi117 bash[28843]: audit 2024-03-04T16:04:06.442176+0000 mon.c (mon.2) 756 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:04:06.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:06 smithi117 bash[28843]: audit 2024-03-04T16:04:06.443369+0000 mon.a (mon.0) 2944 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:04:06.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:06 smithi117 bash[21480]: cluster 2024-03-04T16:04:06.427264+0000 mon.a (mon.0) 2941 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:04:06.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:06 smithi117 bash[21480]: audit 2024-03-04T16:04:06.429649+0000 mon.a (mon.0) 2942 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-100", "mode": "writeback"}]': finished 2024-03-04T16:04:06.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:06 smithi117 bash[21480]: cluster 2024-03-04T16:04:06.437010+0000 mon.a (mon.0) 2943 : cluster 0 osdmap e531: 8 total, 8 up, 8 in 2024-03-04T16:04:06.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:06 smithi117 bash[21480]: audit 2024-03-04T16:04:06.442176+0000 mon.c (mon.2) 756 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:04:06.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:06 smithi117 bash[21480]: audit 2024-03-04T16:04:06.443369+0000 mon.a (mon.0) 2944 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:04:06.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:06 smithi151 bash[23756]: cluster 2024-03-04T16:04:06.427264+0000 mon.a (mon.0) 2941 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:04:06.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:06 smithi151 bash[23756]: audit 2024-03-04T16:04:06.429649+0000 mon.a (mon.0) 2942 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-100", "mode": "writeback"}]': finished 2024-03-04T16:04:06.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:06 smithi151 bash[23756]: cluster 2024-03-04T16:04:06.437010+0000 mon.a (mon.0) 2943 : cluster 0 osdmap e531: 8 total, 8 up, 8 in 2024-03-04T16:04:06.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:06 smithi151 bash[23756]: audit 2024-03-04T16:04:06.442176+0000 mon.c (mon.2) 756 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:04:06.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:06 smithi151 bash[23756]: audit 2024-03-04T16:04:06.443369+0000 mon.a (mon.0) 2944 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:04:07.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:04:07 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:04:07.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:07 smithi117 bash[28843]: audit 2024-03-04T16:04:07.028697+0000 mgr.y (mgr.24428) 559 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:04:07.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:07 smithi117 bash[28843]: cluster 2024-03-04T16:04:07.115795+0000 mgr.y (mgr.24428) 560 : cluster 0 pgmap v866: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:07.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:07 smithi117 bash[28843]: audit 2024-03-04T16:04:07.432306+0000 mon.a (mon.0) 2945 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_count","val": "2"}]': finished 2024-03-04T16:04:07.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:07 smithi117 bash[28843]: cluster 2024-03-04T16:04:07.439917+0000 mon.a (mon.0) 2946 : cluster 0 osdmap e532: 8 total, 8 up, 8 in 2024-03-04T16:04:07.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:07 smithi117 bash[28843]: audit 2024-03-04T16:04:07.444047+0000 mon.c (mon.2) 757 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:04:07.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:07 smithi117 bash[28843]: audit 2024-03-04T16:04:07.444896+0000 mon.a (mon.0) 2947 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:04:07.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:07 smithi117 bash[21480]: audit 2024-03-04T16:04:07.028697+0000 mgr.y (mgr.24428) 559 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:04:07.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:07 smithi117 bash[21480]: cluster 2024-03-04T16:04:07.115795+0000 mgr.y (mgr.24428) 560 : cluster 0 pgmap v866: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:07.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:07 smithi117 bash[21480]: audit 2024-03-04T16:04:07.432306+0000 mon.a (mon.0) 2945 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_count","val": "2"}]': finished 2024-03-04T16:04:07.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:07 smithi117 bash[21480]: cluster 2024-03-04T16:04:07.439917+0000 mon.a (mon.0) 2946 : cluster 0 osdmap e532: 8 total, 8 up, 8 in 2024-03-04T16:04:07.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:07 smithi117 bash[21480]: audit 2024-03-04T16:04:07.444047+0000 mon.c (mon.2) 757 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:04:07.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:07 smithi117 bash[21480]: audit 2024-03-04T16:04:07.444896+0000 mon.a (mon.0) 2947 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:04:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:07 smithi151 bash[23756]: audit 2024-03-04T16:04:07.028697+0000 mgr.y (mgr.24428) 559 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:04:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:07 smithi151 bash[23756]: cluster 2024-03-04T16:04:07.115795+0000 mgr.y (mgr.24428) 560 : cluster 0 pgmap v866: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:07 smithi151 bash[23756]: audit 2024-03-04T16:04:07.432306+0000 mon.a (mon.0) 2945 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_count","val": "2"}]': finished 2024-03-04T16:04:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:07 smithi151 bash[23756]: cluster 2024-03-04T16:04:07.439917+0000 mon.a (mon.0) 2946 : cluster 0 osdmap e532: 8 total, 8 up, 8 in 2024-03-04T16:04:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:07 smithi151 bash[23756]: audit 2024-03-04T16:04:07.444047+0000 mon.c (mon.2) 757 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:04:07.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:07 smithi151 bash[23756]: audit 2024-03-04T16:04:07.444896+0000 mon.a (mon.0) 2947 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:04:09.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:09 smithi117 bash[21480]: audit 2024-03-04T16:04:08.434817+0000 mon.a (mon.0) 2948 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_period","val": "600"}]': finished 2024-03-04T16:04:09.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:09 smithi117 bash[21480]: cluster 2024-03-04T16:04:08.441883+0000 mon.a (mon.0) 2949 : cluster 0 osdmap e533: 8 total, 8 up, 8 in 2024-03-04T16:04:09.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:09 smithi117 bash[21480]: audit 2024-03-04T16:04:08.445499+0000 mon.c (mon.2) 758 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T16:04:09.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:09 smithi117 bash[21480]: audit 2024-03-04T16:04:08.448404+0000 mon.a (mon.0) 2950 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T16:04:09.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:09 smithi117 bash[21480]: cluster 2024-03-04T16:04:09.060546+0000 mon.a (mon.0) 2951 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:09.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:09 smithi117 bash[21480]: cluster 2024-03-04T16:04:09.116573+0000 mgr.y (mgr.24428) 561 : cluster 0 pgmap v869: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:09.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:09 smithi117 bash[28843]: audit 2024-03-04T16:04:08.434817+0000 mon.a (mon.0) 2948 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_period","val": "600"}]': finished 2024-03-04T16:04:09.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:09 smithi117 bash[28843]: cluster 2024-03-04T16:04:08.441883+0000 mon.a (mon.0) 2949 : cluster 0 osdmap e533: 8 total, 8 up, 8 in 2024-03-04T16:04:09.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:09 smithi117 bash[28843]: audit 2024-03-04T16:04:08.445499+0000 mon.c (mon.2) 758 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T16:04:09.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:09 smithi117 bash[28843]: audit 2024-03-04T16:04:08.448404+0000 mon.a (mon.0) 2950 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T16:04:09.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:09 smithi117 bash[28843]: cluster 2024-03-04T16:04:09.060546+0000 mon.a (mon.0) 2951 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:09.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:09 smithi117 bash[28843]: cluster 2024-03-04T16:04:09.116573+0000 mgr.y (mgr.24428) 561 : cluster 0 pgmap v869: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:09.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:09 smithi151 bash[23756]: audit 2024-03-04T16:04:08.434817+0000 mon.a (mon.0) 2948 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_period","val": "600"}]': finished 2024-03-04T16:04:09.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:09 smithi151 bash[23756]: cluster 2024-03-04T16:04:08.441883+0000 mon.a (mon.0) 2949 : cluster 0 osdmap e533: 8 total, 8 up, 8 in 2024-03-04T16:04:09.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:09 smithi151 bash[23756]: audit 2024-03-04T16:04:08.445499+0000 mon.c (mon.2) 758 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T16:04:09.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:09 smithi151 bash[23756]: audit 2024-03-04T16:04:08.448404+0000 mon.a (mon.0) 2950 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T16:04:09.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:09 smithi151 bash[23756]: cluster 2024-03-04T16:04:09.060546+0000 mon.a (mon.0) 2951 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:09.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:09 smithi151 bash[23756]: cluster 2024-03-04T16:04:09.116573+0000 mgr.y (mgr.24428) 561 : cluster 0 pgmap v869: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:10.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:10 smithi117 bash[28843]: cluster 2024-03-04T16:04:09.435735+0000 mon.a (mon.0) 2952 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:04:10.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:10 smithi117 bash[28843]: audit 2024-03-04T16:04:09.443694+0000 mon.a (mon.0) 2953 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-04T16:04:10.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:10 smithi117 bash[28843]: cluster 2024-03-04T16:04:09.453598+0000 mon.a (mon.0) 2954 : cluster 0 osdmap e534: 8 total, 8 up, 8 in 2024-03-04T16:04:10.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:10 smithi117 bash[28843]: audit 2024-03-04T16:04:09.456241+0000 mon.c (mon.2) 759 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-04T16:04:10.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:10 smithi117 bash[28843]: audit 2024-03-04T16:04:09.457169+0000 mon.a (mon.0) 2955 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-04T16:04:10.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:10 smithi117 bash[21480]: cluster 2024-03-04T16:04:09.435735+0000 mon.a (mon.0) 2952 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:04:10.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:10 smithi117 bash[21480]: audit 2024-03-04T16:04:09.443694+0000 mon.a (mon.0) 2953 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-04T16:04:10.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:10 smithi117 bash[21480]: cluster 2024-03-04T16:04:09.453598+0000 mon.a (mon.0) 2954 : cluster 0 osdmap e534: 8 total, 8 up, 8 in 2024-03-04T16:04:10.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:10 smithi117 bash[21480]: audit 2024-03-04T16:04:09.456241+0000 mon.c (mon.2) 759 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-04T16:04:10.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:10 smithi117 bash[21480]: audit 2024-03-04T16:04:09.457169+0000 mon.a (mon.0) 2955 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-04T16:04:10.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:10 smithi151 bash[23756]: cluster 2024-03-04T16:04:09.435735+0000 mon.a (mon.0) 2952 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:04:10.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:10 smithi151 bash[23756]: audit 2024-03-04T16:04:09.443694+0000 mon.a (mon.0) 2953 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-04T16:04:10.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:10 smithi151 bash[23756]: cluster 2024-03-04T16:04:09.453598+0000 mon.a (mon.0) 2954 : cluster 0 osdmap e534: 8 total, 8 up, 8 in 2024-03-04T16:04:10.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:10 smithi151 bash[23756]: audit 2024-03-04T16:04:09.456241+0000 mon.c (mon.2) 759 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-04T16:04:10.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:10 smithi151 bash[23756]: audit 2024-03-04T16:04:09.457169+0000 mon.a (mon.0) 2955 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-03-04T16:04:11.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:11 smithi117 bash[28843]: audit 2024-03-04T16:04:10.454053+0000 mon.a (mon.0) 2956 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-03-04T16:04:11.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:11 smithi117 bash[28843]: cluster 2024-03-04T16:04:10.460852+0000 mon.a (mon.0) 2957 : cluster 0 osdmap e535: 8 total, 8 up, 8 in 2024-03-04T16:04:11.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:11 smithi117 bash[28843]: audit 2024-03-04T16:04:10.529481+0000 mon.c (mon.2) 760 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:04:11.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:11 smithi117 bash[28843]: audit 2024-03-04T16:04:10.530219+0000 mon.a (mon.0) 2958 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:04:11.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:11 smithi117 bash[28843]: cluster 2024-03-04T16:04:11.117419+0000 mgr.y (mgr.24428) 562 : cluster 0 pgmap v872: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:11.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:11 smithi117 bash[21480]: audit 2024-03-04T16:04:10.454053+0000 mon.a (mon.0) 2956 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-03-04T16:04:11.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:11 smithi117 bash[21480]: cluster 2024-03-04T16:04:10.460852+0000 mon.a (mon.0) 2957 : cluster 0 osdmap e535: 8 total, 8 up, 8 in 2024-03-04T16:04:11.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:11 smithi117 bash[21480]: audit 2024-03-04T16:04:10.529481+0000 mon.c (mon.2) 760 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:04:11.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:11 smithi117 bash[21480]: audit 2024-03-04T16:04:10.530219+0000 mon.a (mon.0) 2958 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:04:11.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:11 smithi117 bash[21480]: cluster 2024-03-04T16:04:11.117419+0000 mgr.y (mgr.24428) 562 : cluster 0 pgmap v872: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:11.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:11 smithi151 bash[23756]: audit 2024-03-04T16:04:10.454053+0000 mon.a (mon.0) 2956 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-03-04T16:04:11.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:11 smithi151 bash[23756]: cluster 2024-03-04T16:04:10.460852+0000 mon.a (mon.0) 2957 : cluster 0 osdmap e535: 8 total, 8 up, 8 in 2024-03-04T16:04:11.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:11 smithi151 bash[23756]: audit 2024-03-04T16:04:10.529481+0000 mon.c (mon.2) 760 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:04:11.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:11 smithi151 bash[23756]: audit 2024-03-04T16:04:10.530219+0000 mon.a (mon.0) 2958 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:04:11.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:11 smithi151 bash[23756]: cluster 2024-03-04T16:04:11.117419+0000 mgr.y (mgr.24428) 562 : cluster 0 pgmap v872: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:12.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:12 smithi117 bash[21480]: audit 2024-03-04T16:04:11.456810+0000 mon.a (mon.0) 2959 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T16:04:12.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:12 smithi117 bash[21480]: cluster 2024-03-04T16:04:11.466913+0000 mon.a (mon.0) 2960 : cluster 0 osdmap e536: 8 total, 8 up, 8 in 2024-03-04T16:04:12.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:12 smithi117 bash[21480]: audit 2024-03-04T16:04:11.468904+0000 mon.c (mon.2) 761 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-100"}]: dispatch 2024-03-04T16:04:12.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:12 smithi117 bash[21480]: audit 2024-03-04T16:04:11.472437+0000 mon.a (mon.0) 2961 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-100"}]: dispatch 2024-03-04T16:04:12.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:12 smithi117 bash[21480]: audit 2024-03-04T16:04:12.459537+0000 mon.a (mon.0) 2962 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-100"}]': finished 2024-03-04T16:04:12.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:12 smithi117 bash[21480]: cluster 2024-03-04T16:04:12.467749+0000 mon.a (mon.0) 2963 : cluster 0 osdmap e537: 8 total, 8 up, 8 in 2024-03-04T16:04:12.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:12 smithi117 bash[28843]: audit 2024-03-04T16:04:11.456810+0000 mon.a (mon.0) 2959 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T16:04:12.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:12 smithi117 bash[28843]: cluster 2024-03-04T16:04:11.466913+0000 mon.a (mon.0) 2960 : cluster 0 osdmap e536: 8 total, 8 up, 8 in 2024-03-04T16:04:12.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:12 smithi117 bash[28843]: audit 2024-03-04T16:04:11.468904+0000 mon.c (mon.2) 761 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-100"}]: dispatch 2024-03-04T16:04:12.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:12 smithi117 bash[28843]: audit 2024-03-04T16:04:11.472437+0000 mon.a (mon.0) 2961 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-100"}]: dispatch 2024-03-04T16:04:12.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:12 smithi117 bash[28843]: audit 2024-03-04T16:04:12.459537+0000 mon.a (mon.0) 2962 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-100"}]': finished 2024-03-04T16:04:12.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:12 smithi117 bash[28843]: cluster 2024-03-04T16:04:12.467749+0000 mon.a (mon.0) 2963 : cluster 0 osdmap e537: 8 total, 8 up, 8 in 2024-03-04T16:04:12.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:12 smithi151 bash[23756]: audit 2024-03-04T16:04:11.456810+0000 mon.a (mon.0) 2959 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]': finished 2024-03-04T16:04:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:12 smithi151 bash[23756]: cluster 2024-03-04T16:04:11.466913+0000 mon.a (mon.0) 2960 : cluster 0 osdmap e536: 8 total, 8 up, 8 in 2024-03-04T16:04:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:12 smithi151 bash[23756]: audit 2024-03-04T16:04:11.468904+0000 mon.c (mon.2) 761 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-100"}]: dispatch 2024-03-04T16:04:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:12 smithi151 bash[23756]: audit 2024-03-04T16:04:11.472437+0000 mon.a (mon.0) 2961 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-100"}]: dispatch 2024-03-04T16:04:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:12 smithi151 bash[23756]: audit 2024-03-04T16:04:12.459537+0000 mon.a (mon.0) 2962 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-100"}]': finished 2024-03-04T16:04:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:12 smithi151 bash[23756]: cluster 2024-03-04T16:04:12.467749+0000 mon.a (mon.0) 2963 : cluster 0 osdmap e537: 8 total, 8 up, 8 in 2024-03-04T16:04:13.815 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:13 smithi117 bash[28843]: cluster 2024-03-04T16:04:13.118225+0000 mgr.y (mgr.24428) 563 : cluster 0 pgmap v875: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-04T16:04:13.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:13 smithi117 bash[21480]: cluster 2024-03-04T16:04:13.118225+0000 mgr.y (mgr.24428) 563 : cluster 0 pgmap v875: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-04T16:04:13.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:13 smithi151 bash[23756]: cluster 2024-03-04T16:04:13.118225+0000 mgr.y (mgr.24428) 563 : cluster 0 pgmap v875: 228 pgs: 228 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-04T16:04:14.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:14 smithi117 bash[21480]: cluster 2024-03-04T16:04:13.497789+0000 mon.a (mon.0) 2964 : cluster 0 osdmap e538: 8 total, 8 up, 8 in 2024-03-04T16:04:14.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:14 smithi117 bash[28843]: cluster 2024-03-04T16:04:13.497789+0000 mon.a (mon.0) 2964 : cluster 0 osdmap e538: 8 total, 8 up, 8 in 2024-03-04T16:04:14.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:14 smithi151 bash[23756]: cluster 2024-03-04T16:04:13.497789+0000 mon.a (mon.0) 2964 : cluster 0 osdmap e538: 8 total, 8 up, 8 in 2024-03-04T16:04:15.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:15 smithi117 bash[21480]: cluster 2024-03-04T16:04:14.503062+0000 mon.a (mon.0) 2965 : cluster 0 osdmap e539: 8 total, 8 up, 8 in 2024-03-04T16:04:15.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:15 smithi117 bash[21480]: audit 2024-03-04T16:04:14.509089+0000 mon.c (mon.2) 762 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:15.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:15 smithi117 bash[21480]: audit 2024-03-04T16:04:14.538567+0000 mon.a (mon.0) 2966 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:15.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:15 smithi117 bash[21480]: audit 2024-03-04T16:04:14.655341+0000 mon.c (mon.2) 763 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:04:15.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:15 smithi117 bash[21480]: cluster 2024-03-04T16:04:15.119041+0000 mgr.y (mgr.24428) 564 : cluster 0 pgmap v878: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:04:15.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:15 smithi117 bash[21480]: audit 2024-03-04T16:04:15.496551+0000 mon.a (mon.0) 2967 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:04:15.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:15 smithi117 bash[21480]: cluster 2024-03-04T16:04:15.507759+0000 mon.a (mon.0) 2968 : cluster 0 osdmap e540: 8 total, 8 up, 8 in 2024-03-04T16:04:15.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:15 smithi117 bash[21480]: audit 2024-03-04T16:04:15.509116+0000 mon.c (mon.2) 764 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:04:15.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:15 smithi117 bash[28843]: cluster 2024-03-04T16:04:14.503062+0000 mon.a (mon.0) 2965 : cluster 0 osdmap e539: 8 total, 8 up, 8 in 2024-03-04T16:04:15.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:15 smithi117 bash[28843]: audit 2024-03-04T16:04:14.509089+0000 mon.c (mon.2) 762 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:15.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:15 smithi117 bash[28843]: audit 2024-03-04T16:04:14.538567+0000 mon.a (mon.0) 2966 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:15.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:15 smithi117 bash[28843]: audit 2024-03-04T16:04:14.655341+0000 mon.c (mon.2) 763 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:04:15.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:15 smithi117 bash[28843]: cluster 2024-03-04T16:04:15.119041+0000 mgr.y (mgr.24428) 564 : cluster 0 pgmap v878: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:04:15.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:15 smithi117 bash[28843]: audit 2024-03-04T16:04:15.496551+0000 mon.a (mon.0) 2967 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:04:15.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:15 smithi117 bash[28843]: cluster 2024-03-04T16:04:15.507759+0000 mon.a (mon.0) 2968 : cluster 0 osdmap e540: 8 total, 8 up, 8 in 2024-03-04T16:04:15.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:15 smithi117 bash[28843]: audit 2024-03-04T16:04:15.509116+0000 mon.c (mon.2) 764 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:04:15.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:15 smithi151 bash[23756]: cluster 2024-03-04T16:04:14.503062+0000 mon.a (mon.0) 2965 : cluster 0 osdmap e539: 8 total, 8 up, 8 in 2024-03-04T16:04:15.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:15 smithi151 bash[23756]: audit 2024-03-04T16:04:14.509089+0000 mon.c (mon.2) 762 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:15.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:15 smithi151 bash[23756]: audit 2024-03-04T16:04:14.538567+0000 mon.a (mon.0) 2966 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:15.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:15 smithi151 bash[23756]: audit 2024-03-04T16:04:14.655341+0000 mon.c (mon.2) 763 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:04:15.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:15 smithi151 bash[23756]: cluster 2024-03-04T16:04:15.119041+0000 mgr.y (mgr.24428) 564 : cluster 0 pgmap v878: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 662 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:04:15.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:15 smithi151 bash[23756]: audit 2024-03-04T16:04:15.496551+0000 mon.a (mon.0) 2967 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:04:15.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:15 smithi151 bash[23756]: cluster 2024-03-04T16:04:15.507759+0000 mon.a (mon.0) 2968 : cluster 0 osdmap e540: 8 total, 8 up, 8 in 2024-03-04T16:04:15.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:15 smithi151 bash[23756]: audit 2024-03-04T16:04:15.509116+0000 mon.c (mon.2) 764 : audit 0 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-03-04T16:04:16.546 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:04:16 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:04:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:04:16.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:16 smithi117 bash[21480]: audit 2024-03-04T16:04:15.524342+0000 mon.c (mon.2) 765 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:04:16.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:16 smithi117 bash[21480]: cluster 2024-03-04T16:04:15.524382+0000 mon.a (mon.0) 2969 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:16.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:16 smithi117 bash[21480]: audit 2024-03-04T16:04:15.527222+0000 mon.a (mon.0) 2970 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:04:16.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:16 smithi117 bash[21480]: audit 2024-03-04T16:04:16.499314+0000 mon.a (mon.0) 2971 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:04:16.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:16 smithi117 bash[21480]: cluster 2024-03-04T16:04:16.507127+0000 mon.a (mon.0) 2972 : cluster 0 osdmap e541: 8 total, 8 up, 8 in 2024-03-04T16:04:16.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:16 smithi117 bash[21480]: audit 2024-03-04T16:04:16.510030+0000 mon.c (mon.2) 766 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:04:16.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:16 smithi117 bash[21480]: audit 2024-03-04T16:04:16.517724+0000 mon.a (mon.0) 2973 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:04:16.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:16 smithi117 bash[28843]: audit 2024-03-04T16:04:15.524342+0000 mon.c (mon.2) 765 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:04:16.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:16 smithi117 bash[28843]: cluster 2024-03-04T16:04:15.524382+0000 mon.a (mon.0) 2969 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:16.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:16 smithi117 bash[28843]: audit 2024-03-04T16:04:15.527222+0000 mon.a (mon.0) 2970 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:04:16.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:16 smithi117 bash[28843]: audit 2024-03-04T16:04:16.499314+0000 mon.a (mon.0) 2971 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:04:16.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:16 smithi117 bash[28843]: cluster 2024-03-04T16:04:16.507127+0000 mon.a (mon.0) 2972 : cluster 0 osdmap e541: 8 total, 8 up, 8 in 2024-03-04T16:04:16.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:16 smithi117 bash[28843]: audit 2024-03-04T16:04:16.510030+0000 mon.c (mon.2) 766 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:04:16.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:16 smithi117 bash[28843]: audit 2024-03-04T16:04:16.517724+0000 mon.a (mon.0) 2973 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:04:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:16 smithi151 bash[23756]: audit 2024-03-04T16:04:15.524342+0000 mon.c (mon.2) 765 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:04:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:16 smithi151 bash[23756]: cluster 2024-03-04T16:04:15.524382+0000 mon.a (mon.0) 2969 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:16 smithi151 bash[23756]: audit 2024-03-04T16:04:15.527222+0000 mon.a (mon.0) 2970 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-03-04T16:04:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:16 smithi151 bash[23756]: audit 2024-03-04T16:04:16.499314+0000 mon.a (mon.0) 2971 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-03-04T16:04:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:16 smithi151 bash[23756]: cluster 2024-03-04T16:04:16.507127+0000 mon.a (mon.0) 2972 : cluster 0 osdmap e541: 8 total, 8 up, 8 in 2024-03-04T16:04:16.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:16 smithi151 bash[23756]: audit 2024-03-04T16:04:16.510030+0000 mon.c (mon.2) 766 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:04:16.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:16 smithi151 bash[23756]: audit 2024-03-04T16:04:16.517724+0000 mon.a (mon.0) 2973 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-03-04T16:04:17.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:04:17 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:04:17.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:17 smithi117 bash[28843]: audit 2024-03-04T16:04:17.040125+0000 mgr.y (mgr.24428) 565 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:04:17.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:17 smithi117 bash[28843]: cluster 2024-03-04T16:04:17.119843+0000 mgr.y (mgr.24428) 566 : cluster 0 pgmap v881: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:17.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:17 smithi117 bash[28843]: audit 2024-03-04T16:04:17.502217+0000 mon.a (mon.0) 2974 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-04T16:04:17.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:17 smithi117 bash[28843]: cluster 2024-03-04T16:04:17.512684+0000 mon.a (mon.0) 2975 : cluster 0 osdmap e542: 8 total, 8 up, 8 in 2024-03-04T16:04:17.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:17 smithi117 bash[28843]: audit 2024-03-04T16:04:17.514135+0000 mon.c (mon.2) 767 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:04:17.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:17 smithi117 bash[28843]: audit 2024-03-04T16:04:17.516002+0000 mon.a (mon.0) 2976 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:04:17.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:17 smithi117 bash[21480]: audit 2024-03-04T16:04:17.040125+0000 mgr.y (mgr.24428) 565 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:04:17.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:17 smithi117 bash[21480]: cluster 2024-03-04T16:04:17.119843+0000 mgr.y (mgr.24428) 566 : cluster 0 pgmap v881: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:17.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:17 smithi117 bash[21480]: audit 2024-03-04T16:04:17.502217+0000 mon.a (mon.0) 2974 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-04T16:04:17.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:17 smithi117 bash[21480]: cluster 2024-03-04T16:04:17.512684+0000 mon.a (mon.0) 2975 : cluster 0 osdmap e542: 8 total, 8 up, 8 in 2024-03-04T16:04:17.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:17 smithi117 bash[21480]: audit 2024-03-04T16:04:17.514135+0000 mon.c (mon.2) 767 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:04:17.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:17 smithi117 bash[21480]: audit 2024-03-04T16:04:17.516002+0000 mon.a (mon.0) 2976 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:04:17.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:17 smithi151 bash[23756]: audit 2024-03-04T16:04:17.040125+0000 mgr.y (mgr.24428) 565 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:04:17.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:17 smithi151 bash[23756]: cluster 2024-03-04T16:04:17.119843+0000 mgr.y (mgr.24428) 566 : cluster 0 pgmap v881: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:17.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:17 smithi151 bash[23756]: audit 2024-03-04T16:04:17.502217+0000 mon.a (mon.0) 2974 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-03-04T16:04:17.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:17 smithi151 bash[23756]: cluster 2024-03-04T16:04:17.512684+0000 mon.a (mon.0) 2975 : cluster 0 osdmap e542: 8 total, 8 up, 8 in 2024-03-04T16:04:17.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:17 smithi151 bash[23756]: audit 2024-03-04T16:04:17.514135+0000 mon.c (mon.2) 767 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:04:17.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:17 smithi151 bash[23756]: audit 2024-03-04T16:04:17.516002+0000 mon.a (mon.0) 2976 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-03-04T16:04:19.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:19 smithi117 bash[21480]: audit 2024-03-04T16:04:18.504816+0000 mon.a (mon.0) 2977 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-04T16:04:19.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:19 smithi117 bash[21480]: cluster 2024-03-04T16:04:18.512048+0000 mon.a (mon.0) 2978 : cluster 0 osdmap e543: 8 total, 8 up, 8 in 2024-03-04T16:04:19.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:19 smithi117 bash[21480]: audit 2024-03-04T16:04:18.575094+0000 mon.c (mon.2) 768 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:04:19.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:19 smithi117 bash[21480]: audit 2024-03-04T16:04:18.575820+0000 mon.a (mon.0) 2979 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:04:19.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:19 smithi117 bash[21480]: audit 2024-03-04T16:04:18.576889+0000 mon.c (mon.2) 769 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-102"}]: dispatch 2024-03-04T16:04:19.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:19 smithi117 bash[21480]: audit 2024-03-04T16:04:18.577666+0000 mon.a (mon.0) 2980 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-102"}]: dispatch 2024-03-04T16:04:19.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:19 smithi117 bash[21480]: cluster 2024-03-04T16:04:19.120678+0000 mgr.y (mgr.24428) 567 : cluster 0 pgmap v884: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:19.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:19 smithi117 bash[28843]: audit 2024-03-04T16:04:18.504816+0000 mon.a (mon.0) 2977 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-04T16:04:19.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:19 smithi117 bash[28843]: cluster 2024-03-04T16:04:18.512048+0000 mon.a (mon.0) 2978 : cluster 0 osdmap e543: 8 total, 8 up, 8 in 2024-03-04T16:04:19.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:19 smithi117 bash[28843]: audit 2024-03-04T16:04:18.575094+0000 mon.c (mon.2) 768 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:04:19.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:19 smithi117 bash[28843]: audit 2024-03-04T16:04:18.575820+0000 mon.a (mon.0) 2979 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:04:19.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:19 smithi117 bash[28843]: audit 2024-03-04T16:04:18.576889+0000 mon.c (mon.2) 769 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-102"}]: dispatch 2024-03-04T16:04:19.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:19 smithi117 bash[28843]: audit 2024-03-04T16:04:18.577666+0000 mon.a (mon.0) 2980 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-102"}]: dispatch 2024-03-04T16:04:19.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:19 smithi117 bash[28843]: cluster 2024-03-04T16:04:19.120678+0000 mgr.y (mgr.24428) 567 : cluster 0 pgmap v884: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:19.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:19 smithi151 bash[23756]: audit 2024-03-04T16:04:18.504816+0000 mon.a (mon.0) 2977 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-03-04T16:04:19.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:19 smithi151 bash[23756]: cluster 2024-03-04T16:04:18.512048+0000 mon.a (mon.0) 2978 : cluster 0 osdmap e543: 8 total, 8 up, 8 in 2024-03-04T16:04:19.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:19 smithi151 bash[23756]: audit 2024-03-04T16:04:18.575094+0000 mon.c (mon.2) 768 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:04:19.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:19 smithi151 bash[23756]: audit 2024-03-04T16:04:18.575820+0000 mon.a (mon.0) 2979 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-6"}]: dispatch 2024-03-04T16:04:19.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:19 smithi151 bash[23756]: audit 2024-03-04T16:04:18.576889+0000 mon.c (mon.2) 769 : audit 1 from='client.? 172.21.15.117:0/3689972379' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-102"}]: dispatch 2024-03-04T16:04:19.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:19 smithi151 bash[23756]: audit 2024-03-04T16:04:18.577666+0000 mon.a (mon.0) 2980 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-6", "tierpool": "test-rados-api-smithi117-64061-102"}]: dispatch 2024-03-04T16:04:19.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:19 smithi151 bash[23756]: cluster 2024-03-04T16:04:19.120678+0000 mgr.y (mgr.24428) 567 : cluster 0 pgmap v884: 228 pgs: 228 active+clean; 8.4 MiB data, 663 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:20.815 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:20 smithi117 bash[21480]: cluster 2024-03-04T16:04:19.526810+0000 mon.a (mon.0) 2981 : cluster 0 osdmap e544: 8 total, 8 up, 8 in 2024-03-04T16:04:20.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:20 smithi117 bash[28843]: cluster 2024-03-04T16:04:19.526810+0000 mon.a (mon.0) 2981 : cluster 0 osdmap e544: 8 total, 8 up, 8 in 2024-03-04T16:04:20.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:20 smithi151 bash[23756]: cluster 2024-03-04T16:04:19.526810+0000 mon.a (mon.0) 2981 : cluster 0 osdmap e544: 8 total, 8 up, 8 in 2024-03-04T16:04:21.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:21 smithi117 bash[21480]: cluster 2024-03-04T16:04:20.523152+0000 mon.a (mon.0) 2982 : cluster 0 osdmap e545: 8 total, 8 up, 8 in 2024-03-04T16:04:21.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:21 smithi117 bash[21480]: audit 2024-03-04T16:04:20.570114+0000 mon.c (mon.2) 770 : audit 1 from='client.? 172.21.15.117:0/2461560686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:21.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:21 smithi117 bash[21480]: audit 2024-03-04T16:04:20.570483+0000 mon.a (mon.0) 2983 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:21.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:21 smithi117 bash[21480]: audit 2024-03-04T16:04:20.571553+0000 mon.c (mon.2) 771 : audit 1 from='client.? 172.21.15.117:0/2461560686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:21.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:21 smithi117 bash[21480]: audit 2024-03-04T16:04:20.571885+0000 mon.a (mon.0) 2984 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:21.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:21 smithi117 bash[21480]: audit 2024-03-04T16:04:20.572427+0000 mon.c (mon.2) 772 : audit 1 from='client.? 172.21.15.117:0/2461560686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T16:04:21.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:21 smithi117 bash[21480]: audit 2024-03-04T16:04:20.572754+0000 mon.a (mon.0) 2985 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T16:04:21.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:21 smithi117 bash[21480]: cluster 2024-03-04T16:04:21.121458+0000 mgr.y (mgr.24428) 568 : cluster 0 pgmap v887: 164 pgs: 164 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:21.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:21 smithi117 bash[28843]: cluster 2024-03-04T16:04:20.523152+0000 mon.a (mon.0) 2982 : cluster 0 osdmap e545: 8 total, 8 up, 8 in 2024-03-04T16:04:21.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:21 smithi117 bash[28843]: audit 2024-03-04T16:04:20.570114+0000 mon.c (mon.2) 770 : audit 1 from='client.? 172.21.15.117:0/2461560686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:21.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:21 smithi117 bash[28843]: audit 2024-03-04T16:04:20.570483+0000 mon.a (mon.0) 2983 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:21.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:21 smithi117 bash[28843]: audit 2024-03-04T16:04:20.571553+0000 mon.c (mon.2) 771 : audit 1 from='client.? 172.21.15.117:0/2461560686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:21.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:21 smithi117 bash[28843]: audit 2024-03-04T16:04:20.571885+0000 mon.a (mon.0) 2984 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:21.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:21 smithi117 bash[28843]: audit 2024-03-04T16:04:20.572427+0000 mon.c (mon.2) 772 : audit 1 from='client.? 172.21.15.117:0/2461560686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T16:04:21.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:21 smithi117 bash[28843]: audit 2024-03-04T16:04:20.572754+0000 mon.a (mon.0) 2985 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T16:04:21.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:21 smithi117 bash[28843]: cluster 2024-03-04T16:04:21.121458+0000 mgr.y (mgr.24428) 568 : cluster 0 pgmap v887: 164 pgs: 164 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:21.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:21 smithi151 bash[23756]: cluster 2024-03-04T16:04:20.523152+0000 mon.a (mon.0) 2982 : cluster 0 osdmap e545: 8 total, 8 up, 8 in 2024-03-04T16:04:21.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:21 smithi151 bash[23756]: audit 2024-03-04T16:04:20.570114+0000 mon.c (mon.2) 770 : audit 1 from='client.? 172.21.15.117:0/2461560686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:21.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:21 smithi151 bash[23756]: audit 2024-03-04T16:04:20.570483+0000 mon.a (mon.0) 2983 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:21.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:21 smithi151 bash[23756]: audit 2024-03-04T16:04:20.571553+0000 mon.c (mon.2) 771 : audit 1 from='client.? 172.21.15.117:0/2461560686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:21.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:21 smithi151 bash[23756]: audit 2024-03-04T16:04:20.571885+0000 mon.a (mon.0) 2984 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:21.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:21 smithi151 bash[23756]: audit 2024-03-04T16:04:20.572427+0000 mon.c (mon.2) 772 : audit 1 from='client.? 172.21.15.117:0/2461560686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T16:04:21.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:21 smithi151 bash[23756]: audit 2024-03-04T16:04:20.572754+0000 mon.a (mon.0) 2985 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T16:04:21.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:21 smithi151 bash[23756]: cluster 2024-03-04T16:04:21.121458+0000 mgr.y (mgr.24428) 568 : cluster 0 pgmap v887: 164 pgs: 164 active+clean; 582 KiB data, 663 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:23.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:22 smithi117 bash[21480]: audit 2024-03-04T16:04:21.535444+0000 mon.a (mon.0) 2986 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T16:04:23.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:22 smithi117 bash[21480]: cluster 2024-03-04T16:04:21.542671+0000 mon.a (mon.0) 2987 : cluster 0 osdmap e546: 8 total, 8 up, 8 in 2024-03-04T16:04:23.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:22 smithi117 bash[21480]: audit 2024-03-04T16:04:21.543994+0000 mon.c (mon.2) 773 : audit 1 from='client.? 172.21.15.117:0/2461560686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi117-64061-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:23.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:22 smithi117 bash[21480]: audit 2024-03-04T16:04:21.545417+0000 mon.a (mon.0) 2988 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi117-64061-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:23.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:22 smithi117 bash[28843]: audit 2024-03-04T16:04:21.535444+0000 mon.a (mon.0) 2986 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T16:04:23.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:22 smithi117 bash[28843]: cluster 2024-03-04T16:04:21.542671+0000 mon.a (mon.0) 2987 : cluster 0 osdmap e546: 8 total, 8 up, 8 in 2024-03-04T16:04:23.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:22 smithi117 bash[28843]: audit 2024-03-04T16:04:21.543994+0000 mon.c (mon.2) 773 : audit 1 from='client.? 172.21.15.117:0/2461560686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi117-64061-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:23.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:22 smithi117 bash[28843]: audit 2024-03-04T16:04:21.545417+0000 mon.a (mon.0) 2988 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi117-64061-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:23.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:22 smithi151 bash[23756]: audit 2024-03-04T16:04:21.535444+0000 mon.a (mon.0) 2986 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T16:04:23.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:22 smithi151 bash[23756]: cluster 2024-03-04T16:04:21.542671+0000 mon.a (mon.0) 2987 : cluster 0 osdmap e546: 8 total, 8 up, 8 in 2024-03-04T16:04:23.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:22 smithi151 bash[23756]: audit 2024-03-04T16:04:21.543994+0000 mon.c (mon.2) 773 : audit 1 from='client.? 172.21.15.117:0/2461560686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi117-64061-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:23.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:22 smithi151 bash[23756]: audit 2024-03-04T16:04:21.545417+0000 mon.a (mon.0) 2988 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi117-64061-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:24.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:23 smithi117 bash[21480]: cluster 2024-03-04T16:04:22.989185+0000 mon.a (mon.0) 2989 : cluster 0 osdmap e547: 8 total, 8 up, 8 in 2024-03-04T16:04:24.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:23 smithi117 bash[21480]: cluster 2024-03-04T16:04:23.122161+0000 mgr.y (mgr.24428) 569 : cluster 0 pgmap v890: 164 pgs: 164 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:24.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:23 smithi117 bash[21480]: audit 2024-03-04T16:04:23.541916+0000 mon.a (mon.0) 2990 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi117-64061-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi117-64061-104"}]': finished 2024-03-04T16:04:24.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:23 smithi117 bash[21480]: cluster 2024-03-04T16:04:23.552230+0000 mon.a (mon.0) 2991 : cluster 0 osdmap e548: 8 total, 8 up, 8 in 2024-03-04T16:04:24.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:23 smithi117 bash[28843]: cluster 2024-03-04T16:04:22.989185+0000 mon.a (mon.0) 2989 : cluster 0 osdmap e547: 8 total, 8 up, 8 in 2024-03-04T16:04:24.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:23 smithi117 bash[28843]: cluster 2024-03-04T16:04:23.122161+0000 mgr.y (mgr.24428) 569 : cluster 0 pgmap v890: 164 pgs: 164 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:24.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:23 smithi117 bash[28843]: audit 2024-03-04T16:04:23.541916+0000 mon.a (mon.0) 2990 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi117-64061-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi117-64061-104"}]': finished 2024-03-04T16:04:24.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:23 smithi117 bash[28843]: cluster 2024-03-04T16:04:23.552230+0000 mon.a (mon.0) 2991 : cluster 0 osdmap e548: 8 total, 8 up, 8 in 2024-03-04T16:04:24.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:23 smithi151 bash[23756]: cluster 2024-03-04T16:04:22.989185+0000 mon.a (mon.0) 2989 : cluster 0 osdmap e547: 8 total, 8 up, 8 in 2024-03-04T16:04:24.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:23 smithi151 bash[23756]: cluster 2024-03-04T16:04:23.122161+0000 mgr.y (mgr.24428) 569 : cluster 0 pgmap v890: 164 pgs: 164 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:24.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:23 smithi151 bash[23756]: audit 2024-03-04T16:04:23.541916+0000 mon.a (mon.0) 2990 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi117-64061-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi117-64061-104"}]': finished 2024-03-04T16:04:24.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:23 smithi151 bash[23756]: cluster 2024-03-04T16:04:23.552230+0000 mon.a (mon.0) 2991 : cluster 0 osdmap e548: 8 total, 8 up, 8 in 2024-03-04T16:04:25.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:25 smithi117 bash[21480]: cluster 2024-03-04T16:04:24.065111+0000 mon.a (mon.0) 2992 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:25.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:25 smithi117 bash[21480]: cluster 2024-03-04T16:04:24.552830+0000 mon.a (mon.0) 2993 : cluster 0 osdmap e549: 8 total, 8 up, 8 in 2024-03-04T16:04:25.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:25 smithi117 bash[28843]: cluster 2024-03-04T16:04:24.065111+0000 mon.a (mon.0) 2992 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:25.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:25 smithi117 bash[28843]: cluster 2024-03-04T16:04:24.552830+0000 mon.a (mon.0) 2993 : cluster 0 osdmap e549: 8 total, 8 up, 8 in 2024-03-04T16:04:25.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:25 smithi151 bash[23756]: cluster 2024-03-04T16:04:24.065111+0000 mon.a (mon.0) 2992 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:25.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:25 smithi151 bash[23756]: cluster 2024-03-04T16:04:24.552830+0000 mon.a (mon.0) 2993 : cluster 0 osdmap e549: 8 total, 8 up, 8 in 2024-03-04T16:04:26.315 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:04:26 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:04:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:04:26.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:26 smithi117 bash[21480]: cluster 2024-03-04T16:04:25.122879+0000 mgr.y (mgr.24428) 570 : cluster 0 pgmap v893: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:26.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:26 smithi117 bash[21480]: cluster 2024-03-04T16:04:25.560915+0000 mon.a (mon.0) 2994 : cluster 0 osdmap e550: 8 total, 8 up, 8 in 2024-03-04T16:04:26.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:26 smithi117 bash[21480]: audit 2024-03-04T16:04:25.566400+0000 mon.a (mon.0) 2995 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:26.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:26 smithi117 bash[28843]: cluster 2024-03-04T16:04:25.122879+0000 mgr.y (mgr.24428) 570 : cluster 0 pgmap v893: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:26.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:26 smithi117 bash[28843]: cluster 2024-03-04T16:04:25.560915+0000 mon.a (mon.0) 2994 : cluster 0 osdmap e550: 8 total, 8 up, 8 in 2024-03-04T16:04:26.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:26 smithi117 bash[28843]: audit 2024-03-04T16:04:25.566400+0000 mon.a (mon.0) 2995 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:26.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:26 smithi151 bash[23756]: cluster 2024-03-04T16:04:25.122879+0000 mgr.y (mgr.24428) 570 : cluster 0 pgmap v893: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:26.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:26 smithi151 bash[23756]: cluster 2024-03-04T16:04:25.560915+0000 mon.a (mon.0) 2994 : cluster 0 osdmap e550: 8 total, 8 up, 8 in 2024-03-04T16:04:26.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:26 smithi151 bash[23756]: audit 2024-03-04T16:04:25.566400+0000 mon.a (mon.0) 2995 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:27.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:04:27 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:04:27.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:27 smithi117 bash[21480]: audit 2024-03-04T16:04:26.552788+0000 mon.a (mon.0) 2996 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:04:27.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:27 smithi117 bash[21480]: cluster 2024-03-04T16:04:26.562122+0000 mon.a (mon.0) 2997 : cluster 0 osdmap e551: 8 total, 8 up, 8 in 2024-03-04T16:04:27.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:27 smithi117 bash[21480]: audit 2024-03-04T16:04:27.044942+0000 mgr.y (mgr.24428) 571 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:04:27.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:27 smithi117 bash[21480]: cluster 2024-03-04T16:04:27.123790+0000 mgr.y (mgr.24428) 572 : cluster 0 pgmap v896: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:04:27.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:27 smithi117 bash[28843]: audit 2024-03-04T16:04:26.552788+0000 mon.a (mon.0) 2996 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:04:27.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:27 smithi117 bash[28843]: cluster 2024-03-04T16:04:26.562122+0000 mon.a (mon.0) 2997 : cluster 0 osdmap e551: 8 total, 8 up, 8 in 2024-03-04T16:04:27.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:27 smithi117 bash[28843]: audit 2024-03-04T16:04:27.044942+0000 mgr.y (mgr.24428) 571 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:04:27.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:27 smithi117 bash[28843]: cluster 2024-03-04T16:04:27.123790+0000 mgr.y (mgr.24428) 572 : cluster 0 pgmap v896: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:04:27.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:27 smithi151 bash[23756]: audit 2024-03-04T16:04:26.552788+0000 mon.a (mon.0) 2996 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:04:27.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:27 smithi151 bash[23756]: cluster 2024-03-04T16:04:26.562122+0000 mon.a (mon.0) 2997 : cluster 0 osdmap e551: 8 total, 8 up, 8 in 2024-03-04T16:04:27.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:27 smithi151 bash[23756]: audit 2024-03-04T16:04:27.044942+0000 mgr.y (mgr.24428) 571 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:04:27.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:27 smithi151 bash[23756]: cluster 2024-03-04T16:04:27.123790+0000 mgr.y (mgr.24428) 572 : cluster 0 pgmap v896: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:04:28.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:28 smithi117 bash[21480]: cluster 2024-03-04T16:04:27.575137+0000 mon.a (mon.0) 2998 : cluster 0 osdmap e552: 8 total, 8 up, 8 in 2024-03-04T16:04:28.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:28 smithi117 bash[21480]: audit 2024-03-04T16:04:27.577399+0000 mon.a (mon.0) 2999 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:28.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:28 smithi117 bash[28843]: cluster 2024-03-04T16:04:27.575137+0000 mon.a (mon.0) 2998 : cluster 0 osdmap e552: 8 total, 8 up, 8 in 2024-03-04T16:04:28.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:28 smithi117 bash[28843]: audit 2024-03-04T16:04:27.577399+0000 mon.a (mon.0) 2999 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:28.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:28 smithi151 bash[23756]: cluster 2024-03-04T16:04:27.575137+0000 mon.a (mon.0) 2998 : cluster 0 osdmap e552: 8 total, 8 up, 8 in 2024-03-04T16:04:28.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:28 smithi151 bash[23756]: audit 2024-03-04T16:04:27.577399+0000 mon.a (mon.0) 2999 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:29.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:29 smithi151 bash[23756]: audit 2024-03-04T16:04:28.576069+0000 mon.a (mon.0) 3000 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:04:29.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:29 smithi151 bash[23756]: cluster 2024-03-04T16:04:28.584559+0000 mon.a (mon.0) 3001 : cluster 0 osdmap e553: 8 total, 8 up, 8 in 2024-03-04T16:04:29.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:29 smithi151 bash[23756]: audit 2024-03-04T16:04:28.590923+0000 mon.a (mon.0) 3002 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-107", "tierpool": "test-rados-api-smithi117-64061-107-cache"}]: dispatch 2024-03-04T16:04:29.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:29 smithi151 bash[23756]: cluster 2024-03-04T16:04:29.066492+0000 mon.a (mon.0) 3003 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:29.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:29 smithi151 bash[23756]: cluster 2024-03-04T16:04:29.124625+0000 mgr.y (mgr.24428) 573 : cluster 0 pgmap v899: 236 pgs: 32 unknown, 32 creating+peering, 172 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:04:30.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:29 smithi117 bash[21480]: audit 2024-03-04T16:04:28.576069+0000 mon.a (mon.0) 3000 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:04:30.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:29 smithi117 bash[21480]: cluster 2024-03-04T16:04:28.584559+0000 mon.a (mon.0) 3001 : cluster 0 osdmap e553: 8 total, 8 up, 8 in 2024-03-04T16:04:30.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:29 smithi117 bash[21480]: audit 2024-03-04T16:04:28.590923+0000 mon.a (mon.0) 3002 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-107", "tierpool": "test-rados-api-smithi117-64061-107-cache"}]: dispatch 2024-03-04T16:04:30.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:29 smithi117 bash[21480]: cluster 2024-03-04T16:04:29.066492+0000 mon.a (mon.0) 3003 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:30.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:29 smithi117 bash[21480]: cluster 2024-03-04T16:04:29.124625+0000 mgr.y (mgr.24428) 573 : cluster 0 pgmap v899: 236 pgs: 32 unknown, 32 creating+peering, 172 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:04:30.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:29 smithi117 bash[28843]: audit 2024-03-04T16:04:28.576069+0000 mon.a (mon.0) 3000 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:04:30.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:29 smithi117 bash[28843]: cluster 2024-03-04T16:04:28.584559+0000 mon.a (mon.0) 3001 : cluster 0 osdmap e553: 8 total, 8 up, 8 in 2024-03-04T16:04:30.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:29 smithi117 bash[28843]: audit 2024-03-04T16:04:28.590923+0000 mon.a (mon.0) 3002 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-107", "tierpool": "test-rados-api-smithi117-64061-107-cache"}]: dispatch 2024-03-04T16:04:30.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:29 smithi117 bash[28843]: cluster 2024-03-04T16:04:29.066492+0000 mon.a (mon.0) 3003 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:30.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:29 smithi117 bash[28843]: cluster 2024-03-04T16:04:29.124625+0000 mgr.y (mgr.24428) 573 : cluster 0 pgmap v899: 236 pgs: 32 unknown, 32 creating+peering, 172 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:04:30.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:30 smithi151 bash[23756]: audit 2024-03-04T16:04:29.585066+0000 mon.a (mon.0) 3004 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-107", "tierpool": "test-rados-api-smithi117-64061-107-cache"}]': finished 2024-03-04T16:04:30.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:30 smithi151 bash[23756]: cluster 2024-03-04T16:04:29.594410+0000 mon.a (mon.0) 3005 : cluster 0 osdmap e554: 8 total, 8 up, 8 in 2024-03-04T16:04:30.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:30 smithi151 bash[23756]: audit 2024-03-04T16:04:29.598817+0000 mon.a (mon.0) 3006 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-107", "overlaypool": "test-rados-api-smithi117-64061-107-cache"}]: dispatch 2024-03-04T16:04:30.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:30 smithi151 bash[23756]: audit 2024-03-04T16:04:29.689272+0000 mon.c (mon.2) 774 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:04:31.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:30 smithi117 bash[28843]: audit 2024-03-04T16:04:29.585066+0000 mon.a (mon.0) 3004 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-107", "tierpool": "test-rados-api-smithi117-64061-107-cache"}]': finished 2024-03-04T16:04:31.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:30 smithi117 bash[28843]: cluster 2024-03-04T16:04:29.594410+0000 mon.a (mon.0) 3005 : cluster 0 osdmap e554: 8 total, 8 up, 8 in 2024-03-04T16:04:31.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:30 smithi117 bash[28843]: audit 2024-03-04T16:04:29.598817+0000 mon.a (mon.0) 3006 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-107", "overlaypool": "test-rados-api-smithi117-64061-107-cache"}]: dispatch 2024-03-04T16:04:31.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:30 smithi117 bash[28843]: audit 2024-03-04T16:04:29.689272+0000 mon.c (mon.2) 774 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:04:31.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:30 smithi117 bash[21480]: audit 2024-03-04T16:04:29.585066+0000 mon.a (mon.0) 3004 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-107", "tierpool": "test-rados-api-smithi117-64061-107-cache"}]': finished 2024-03-04T16:04:31.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:30 smithi117 bash[21480]: cluster 2024-03-04T16:04:29.594410+0000 mon.a (mon.0) 3005 : cluster 0 osdmap e554: 8 total, 8 up, 8 in 2024-03-04T16:04:31.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:30 smithi117 bash[21480]: audit 2024-03-04T16:04:29.598817+0000 mon.a (mon.0) 3006 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-107", "overlaypool": "test-rados-api-smithi117-64061-107-cache"}]: dispatch 2024-03-04T16:04:31.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:30 smithi117 bash[21480]: audit 2024-03-04T16:04:29.689272+0000 mon.c (mon.2) 774 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:04:31.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:31 smithi151 bash[23756]: audit 2024-03-04T16:04:30.587720+0000 mon.a (mon.0) 3007 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-107", "overlaypool": "test-rados-api-smithi117-64061-107-cache"}]': finished 2024-03-04T16:04:31.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:31 smithi151 bash[23756]: cluster 2024-03-04T16:04:30.594897+0000 mon.a (mon.0) 3008 : cluster 0 osdmap e555: 8 total, 8 up, 8 in 2024-03-04T16:04:31.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:31 smithi151 bash[23756]: audit 2024-03-04T16:04:30.596651+0000 mon.a (mon.0) 3009 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-107-cache", "mode": "writeback"}]: dispatch 2024-03-04T16:04:31.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:31 smithi151 bash[23756]: cluster 2024-03-04T16:04:31.125479+0000 mgr.y (mgr.24428) 574 : cluster 0 pgmap v902: 236 pgs: 32 unknown, 32 creating+peering, 172 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:31.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:31 smithi151 bash[23756]: cluster 2024-03-04T16:04:31.588025+0000 mon.a (mon.0) 3010 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:04:31.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:31 smithi151 bash[23756]: audit 2024-03-04T16:04:31.590346+0000 mon.a (mon.0) 3011 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-107-cache", "mode": "writeback"}]': finished 2024-03-04T16:04:31.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:31 smithi151 bash[23756]: cluster 2024-03-04T16:04:31.598398+0000 mon.a (mon.0) 3012 : cluster 0 osdmap e556: 8 total, 8 up, 8 in 2024-03-04T16:04:32.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:31 smithi117 bash[28843]: audit 2024-03-04T16:04:30.587720+0000 mon.a (mon.0) 3007 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-107", "overlaypool": "test-rados-api-smithi117-64061-107-cache"}]': finished 2024-03-04T16:04:32.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:31 smithi117 bash[28843]: cluster 2024-03-04T16:04:30.594897+0000 mon.a (mon.0) 3008 : cluster 0 osdmap e555: 8 total, 8 up, 8 in 2024-03-04T16:04:32.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:31 smithi117 bash[28843]: audit 2024-03-04T16:04:30.596651+0000 mon.a (mon.0) 3009 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-107-cache", "mode": "writeback"}]: dispatch 2024-03-04T16:04:32.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:31 smithi117 bash[28843]: cluster 2024-03-04T16:04:31.125479+0000 mgr.y (mgr.24428) 574 : cluster 0 pgmap v902: 236 pgs: 32 unknown, 32 creating+peering, 172 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:32.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:31 smithi117 bash[28843]: cluster 2024-03-04T16:04:31.588025+0000 mon.a (mon.0) 3010 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:04:32.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:31 smithi117 bash[28843]: audit 2024-03-04T16:04:31.590346+0000 mon.a (mon.0) 3011 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-107-cache", "mode": "writeback"}]': finished 2024-03-04T16:04:32.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:31 smithi117 bash[28843]: cluster 2024-03-04T16:04:31.598398+0000 mon.a (mon.0) 3012 : cluster 0 osdmap e556: 8 total, 8 up, 8 in 2024-03-04T16:04:32.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:31 smithi117 bash[21480]: audit 2024-03-04T16:04:30.587720+0000 mon.a (mon.0) 3007 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-107", "overlaypool": "test-rados-api-smithi117-64061-107-cache"}]': finished 2024-03-04T16:04:32.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:31 smithi117 bash[21480]: cluster 2024-03-04T16:04:30.594897+0000 mon.a (mon.0) 3008 : cluster 0 osdmap e555: 8 total, 8 up, 8 in 2024-03-04T16:04:32.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:31 smithi117 bash[21480]: audit 2024-03-04T16:04:30.596651+0000 mon.a (mon.0) 3009 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-107-cache", "mode": "writeback"}]: dispatch 2024-03-04T16:04:32.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:31 smithi117 bash[21480]: cluster 2024-03-04T16:04:31.125479+0000 mgr.y (mgr.24428) 574 : cluster 0 pgmap v902: 236 pgs: 32 unknown, 32 creating+peering, 172 active+clean; 582 KiB data, 638 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:32.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:31 smithi117 bash[21480]: cluster 2024-03-04T16:04:31.588025+0000 mon.a (mon.0) 3010 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:04:32.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:31 smithi117 bash[21480]: audit 2024-03-04T16:04:31.590346+0000 mon.a (mon.0) 3011 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-107-cache", "mode": "writeback"}]': finished 2024-03-04T16:04:32.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:31 smithi117 bash[21480]: cluster 2024-03-04T16:04:31.598398+0000 mon.a (mon.0) 3012 : cluster 0 osdmap e556: 8 total, 8 up, 8 in 2024-03-04T16:04:32.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:32 smithi151 bash[23756]: audit 2024-03-04T16:04:31.640657+0000 mon.a (mon.0) 3013 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-107"}]: dispatch 2024-03-04T16:04:33.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:32 smithi117 bash[21480]: audit 2024-03-04T16:04:31.640657+0000 mon.a (mon.0) 3013 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-107"}]: dispatch 2024-03-04T16:04:33.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:32 smithi117 bash[28843]: audit 2024-03-04T16:04:31.640657+0000 mon.a (mon.0) 3013 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-107"}]: dispatch 2024-03-04T16:04:33.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:33 smithi151 bash[23756]: audit 2024-03-04T16:04:32.625569+0000 mon.a (mon.0) 3014 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-107"}]': finished 2024-03-04T16:04:33.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:33 smithi151 bash[23756]: cluster 2024-03-04T16:04:32.634115+0000 mon.a (mon.0) 3015 : cluster 0 osdmap e557: 8 total, 8 up, 8 in 2024-03-04T16:04:33.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:33 smithi151 bash[23756]: audit 2024-03-04T16:04:32.635819+0000 mon.a (mon.0) 3016 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-107", "tierpool": "test-rados-api-smithi117-64061-107-cache"}]: dispatch 2024-03-04T16:04:33.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:33 smithi151 bash[23756]: cluster 2024-03-04T16:04:33.126330+0000 mgr.y (mgr.24428) 575 : cluster 0 pgmap v905: 236 pgs: 236 active+clean; 582 KiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:34.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:33 smithi117 bash[28843]: audit 2024-03-04T16:04:32.625569+0000 mon.a (mon.0) 3014 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-107"}]': finished 2024-03-04T16:04:34.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:33 smithi117 bash[28843]: cluster 2024-03-04T16:04:32.634115+0000 mon.a (mon.0) 3015 : cluster 0 osdmap e557: 8 total, 8 up, 8 in 2024-03-04T16:04:34.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:33 smithi117 bash[28843]: audit 2024-03-04T16:04:32.635819+0000 mon.a (mon.0) 3016 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-107", "tierpool": "test-rados-api-smithi117-64061-107-cache"}]: dispatch 2024-03-04T16:04:34.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:33 smithi117 bash[28843]: cluster 2024-03-04T16:04:33.126330+0000 mgr.y (mgr.24428) 575 : cluster 0 pgmap v905: 236 pgs: 236 active+clean; 582 KiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:34.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:33 smithi117 bash[21480]: audit 2024-03-04T16:04:32.625569+0000 mon.a (mon.0) 3014 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-107"}]': finished 2024-03-04T16:04:34.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:33 smithi117 bash[21480]: cluster 2024-03-04T16:04:32.634115+0000 mon.a (mon.0) 3015 : cluster 0 osdmap e557: 8 total, 8 up, 8 in 2024-03-04T16:04:34.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:33 smithi117 bash[21480]: audit 2024-03-04T16:04:32.635819+0000 mon.a (mon.0) 3016 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-107", "tierpool": "test-rados-api-smithi117-64061-107-cache"}]: dispatch 2024-03-04T16:04:34.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:33 smithi117 bash[21480]: cluster 2024-03-04T16:04:33.126330+0000 mgr.y (mgr.24428) 575 : cluster 0 pgmap v905: 236 pgs: 236 active+clean; 582 KiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:34.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:34 smithi151 bash[23756]: cluster 2024-03-04T16:04:33.625872+0000 mon.a (mon.0) 3017 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:04:34.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:34 smithi151 bash[23756]: audit 2024-03-04T16:04:33.628328+0000 mon.a (mon.0) 3018 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-107", "tierpool": "test-rados-api-smithi117-64061-107-cache"}]': finished 2024-03-04T16:04:34.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:34 smithi151 bash[23756]: cluster 2024-03-04T16:04:33.634732+0000 mon.a (mon.0) 3019 : cluster 0 osdmap e558: 8 total, 8 up, 8 in 2024-03-04T16:04:34.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:34 smithi151 bash[23756]: cluster 2024-03-04T16:04:34.637656+0000 mon.a (mon.0) 3020 : cluster 0 osdmap e559: 8 total, 8 up, 8 in 2024-03-04T16:04:35.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:34 smithi117 bash[28843]: cluster 2024-03-04T16:04:33.625872+0000 mon.a (mon.0) 3017 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:04:35.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:34 smithi117 bash[28843]: audit 2024-03-04T16:04:33.628328+0000 mon.a (mon.0) 3018 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-107", "tierpool": "test-rados-api-smithi117-64061-107-cache"}]': finished 2024-03-04T16:04:35.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:34 smithi117 bash[28843]: cluster 2024-03-04T16:04:33.634732+0000 mon.a (mon.0) 3019 : cluster 0 osdmap e558: 8 total, 8 up, 8 in 2024-03-04T16:04:35.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:34 smithi117 bash[28843]: cluster 2024-03-04T16:04:34.637656+0000 mon.a (mon.0) 3020 : cluster 0 osdmap e559: 8 total, 8 up, 8 in 2024-03-04T16:04:35.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:34 smithi117 bash[21480]: cluster 2024-03-04T16:04:33.625872+0000 mon.a (mon.0) 3017 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:04:35.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:34 smithi117 bash[21480]: audit 2024-03-04T16:04:33.628328+0000 mon.a (mon.0) 3018 : audit 1 from='client.? 172.21.15.117:0/3210970924' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-107", "tierpool": "test-rados-api-smithi117-64061-107-cache"}]': finished 2024-03-04T16:04:35.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:34 smithi117 bash[21480]: cluster 2024-03-04T16:04:33.634732+0000 mon.a (mon.0) 3019 : cluster 0 osdmap e558: 8 total, 8 up, 8 in 2024-03-04T16:04:35.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:34 smithi117 bash[21480]: cluster 2024-03-04T16:04:34.637656+0000 mon.a (mon.0) 3020 : cluster 0 osdmap e559: 8 total, 8 up, 8 in 2024-03-04T16:04:35.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:35 smithi151 bash[23756]: cluster 2024-03-04T16:04:35.127180+0000 mgr.y (mgr.24428) 576 : cluster 0 pgmap v908: 204 pgs: 204 active+clean; 582 KiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:35.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:35 smithi151 bash[23756]: cluster 2024-03-04T16:04:35.642804+0000 mon.a (mon.0) 3021 : cluster 0 osdmap e560: 8 total, 8 up, 8 in 2024-03-04T16:04:36.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:35 smithi117 bash[21480]: cluster 2024-03-04T16:04:35.127180+0000 mgr.y (mgr.24428) 576 : cluster 0 pgmap v908: 204 pgs: 204 active+clean; 582 KiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:36.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:35 smithi117 bash[21480]: cluster 2024-03-04T16:04:35.642804+0000 mon.a (mon.0) 3021 : cluster 0 osdmap e560: 8 total, 8 up, 8 in 2024-03-04T16:04:36.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:35 smithi117 bash[28843]: cluster 2024-03-04T16:04:35.127180+0000 mgr.y (mgr.24428) 576 : cluster 0 pgmap v908: 204 pgs: 204 active+clean; 582 KiB data, 639 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:36.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:35 smithi117 bash[28843]: cluster 2024-03-04T16:04:35.642804+0000 mon.a (mon.0) 3021 : cluster 0 osdmap e560: 8 total, 8 up, 8 in 2024-03-04T16:04:36.566 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:04:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:04:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:04:36.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:36 smithi151 bash[23756]: audit 2024-03-04T16:04:35.699247+0000 mon.c (mon.2) 775 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:36.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:36 smithi151 bash[23756]: audit 2024-03-04T16:04:35.699636+0000 mon.a (mon.0) 3022 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:36.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:36 smithi151 bash[23756]: audit 2024-03-04T16:04:35.700950+0000 mon.c (mon.2) 776 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:36.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:36 smithi151 bash[23756]: audit 2024-03-04T16:04:35.701312+0000 mon.a (mon.0) 3023 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:36.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:36 smithi151 bash[23756]: audit 2024-03-04T16:04:35.701820+0000 mon.c (mon.2) 777 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-64061-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T16:04:36.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:36 smithi151 bash[23756]: audit 2024-03-04T16:04:35.702091+0000 mon.a (mon.0) 3024 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-64061-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T16:04:37.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:36 smithi117 bash[28843]: audit 2024-03-04T16:04:35.699247+0000 mon.c (mon.2) 775 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:37.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:36 smithi117 bash[28843]: audit 2024-03-04T16:04:35.699636+0000 mon.a (mon.0) 3022 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:37.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:36 smithi117 bash[28843]: audit 2024-03-04T16:04:35.700950+0000 mon.c (mon.2) 776 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:37.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:36 smithi117 bash[28843]: audit 2024-03-04T16:04:35.701312+0000 mon.a (mon.0) 3023 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:37.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:36 smithi117 bash[28843]: audit 2024-03-04T16:04:35.701820+0000 mon.c (mon.2) 777 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-64061-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T16:04:37.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:36 smithi117 bash[28843]: audit 2024-03-04T16:04:35.702091+0000 mon.a (mon.0) 3024 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-64061-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T16:04:37.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:36 smithi117 bash[21480]: audit 2024-03-04T16:04:35.699247+0000 mon.c (mon.2) 775 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:37.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:36 smithi117 bash[21480]: audit 2024-03-04T16:04:35.699636+0000 mon.a (mon.0) 3022 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:37.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:36 smithi117 bash[21480]: audit 2024-03-04T16:04:35.700950+0000 mon.c (mon.2) 776 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:37.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:36 smithi117 bash[21480]: audit 2024-03-04T16:04:35.701312+0000 mon.a (mon.0) 3023 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:37.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:36 smithi117 bash[21480]: audit 2024-03-04T16:04:35.701820+0000 mon.c (mon.2) 777 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-64061-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T16:04:37.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:36 smithi117 bash[21480]: audit 2024-03-04T16:04:35.702091+0000 mon.a (mon.0) 3024 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-64061-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T16:04:37.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:04:37 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:04:38.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:37 smithi117 bash[21480]: audit 2024-03-04T16:04:36.672516+0000 mon.a (mon.0) 3025 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-64061-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T16:04:38.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:37 smithi117 bash[21480]: cluster 2024-03-04T16:04:36.680211+0000 mon.a (mon.0) 3026 : cluster 0 osdmap e561: 8 total, 8 up, 8 in 2024-03-04T16:04:38.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:37 smithi117 bash[21480]: audit 2024-03-04T16:04:36.684668+0000 mon.c (mon.2) 778 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64061-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:38.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:37 smithi117 bash[21480]: audit 2024-03-04T16:04:36.692543+0000 mon.a (mon.0) 3027 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64061-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:38.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:37 smithi117 bash[21480]: audit 2024-03-04T16:04:37.048117+0000 mgr.y (mgr.24428) 577 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:04:38.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:37 smithi117 bash[21480]: cluster 2024-03-04T16:04:37.127946+0000 mgr.y (mgr.24428) 578 : cluster 0 pgmap v911: 172 pgs: 172 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:38.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:37 smithi117 bash[28843]: audit 2024-03-04T16:04:36.672516+0000 mon.a (mon.0) 3025 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-64061-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T16:04:38.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:37 smithi117 bash[28843]: cluster 2024-03-04T16:04:36.680211+0000 mon.a (mon.0) 3026 : cluster 0 osdmap e561: 8 total, 8 up, 8 in 2024-03-04T16:04:38.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:37 smithi117 bash[28843]: audit 2024-03-04T16:04:36.684668+0000 mon.c (mon.2) 778 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64061-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:38.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:37 smithi117 bash[28843]: audit 2024-03-04T16:04:36.692543+0000 mon.a (mon.0) 3027 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64061-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:38.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:37 smithi117 bash[28843]: audit 2024-03-04T16:04:37.048117+0000 mgr.y (mgr.24428) 577 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:04:38.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:37 smithi117 bash[28843]: cluster 2024-03-04T16:04:37.127946+0000 mgr.y (mgr.24428) 578 : cluster 0 pgmap v911: 172 pgs: 172 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:38.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:37 smithi151 bash[23756]: audit 2024-03-04T16:04:36.672516+0000 mon.a (mon.0) 3025 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-64061-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T16:04:38.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:37 smithi151 bash[23756]: cluster 2024-03-04T16:04:36.680211+0000 mon.a (mon.0) 3026 : cluster 0 osdmap e561: 8 total, 8 up, 8 in 2024-03-04T16:04:38.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:37 smithi151 bash[23756]: audit 2024-03-04T16:04:36.684668+0000 mon.c (mon.2) 778 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64061-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:38.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:37 smithi151 bash[23756]: audit 2024-03-04T16:04:36.692543+0000 mon.a (mon.0) 3027 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64061-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:38.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:37 smithi151 bash[23756]: audit 2024-03-04T16:04:37.048117+0000 mgr.y (mgr.24428) 577 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:04:38.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:37 smithi151 bash[23756]: cluster 2024-03-04T16:04:37.127946+0000 mgr.y (mgr.24428) 578 : cluster 0 pgmap v911: 172 pgs: 172 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:39.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:38 smithi117 bash[28843]: cluster 2024-03-04T16:04:38.156992+0000 mon.a (mon.0) 3028 : cluster 0 osdmap e562: 8 total, 8 up, 8 in 2024-03-04T16:04:39.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:38 smithi117 bash[21480]: cluster 2024-03-04T16:04:38.156992+0000 mon.a (mon.0) 3028 : cluster 0 osdmap e562: 8 total, 8 up, 8 in 2024-03-04T16:04:39.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:38 smithi151 bash[23756]: cluster 2024-03-04T16:04:38.156992+0000 mon.a (mon.0) 3028 : cluster 0 osdmap e562: 8 total, 8 up, 8 in 2024-03-04T16:04:40.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:39 smithi117 bash[21480]: audit 2024-03-04T16:04:38.700129+0000 mon.a (mon.0) 3029 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64061-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-64061-109"}]': finished 2024-03-04T16:04:40.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:39 smithi117 bash[21480]: cluster 2024-03-04T16:04:38.709894+0000 mon.a (mon.0) 3030 : cluster 0 osdmap e563: 8 total, 8 up, 8 in 2024-03-04T16:04:40.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:39 smithi117 bash[21480]: cluster 2024-03-04T16:04:39.128716+0000 mgr.y (mgr.24428) 579 : cluster 0 pgmap v914: 180 pgs: 8 unknown, 172 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:40.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:39 smithi117 bash[28843]: audit 2024-03-04T16:04:38.700129+0000 mon.a (mon.0) 3029 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64061-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-64061-109"}]': finished 2024-03-04T16:04:40.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:39 smithi117 bash[28843]: cluster 2024-03-04T16:04:38.709894+0000 mon.a (mon.0) 3030 : cluster 0 osdmap e563: 8 total, 8 up, 8 in 2024-03-04T16:04:40.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:39 smithi117 bash[28843]: cluster 2024-03-04T16:04:39.128716+0000 mgr.y (mgr.24428) 579 : cluster 0 pgmap v914: 180 pgs: 8 unknown, 172 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:40.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:39 smithi151 bash[23756]: audit 2024-03-04T16:04:38.700129+0000 mon.a (mon.0) 3029 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64061-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-64061-109"}]': finished 2024-03-04T16:04:40.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:39 smithi151 bash[23756]: cluster 2024-03-04T16:04:38.709894+0000 mon.a (mon.0) 3030 : cluster 0 osdmap e563: 8 total, 8 up, 8 in 2024-03-04T16:04:40.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:39 smithi151 bash[23756]: cluster 2024-03-04T16:04:39.128716+0000 mgr.y (mgr.24428) 579 : cluster 0 pgmap v914: 180 pgs: 8 unknown, 172 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:41.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:40 smithi117 bash[21480]: cluster 2024-03-04T16:04:39.719075+0000 mon.a (mon.0) 3031 : cluster 0 osdmap e564: 8 total, 8 up, 8 in 2024-03-04T16:04:41.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:40 smithi117 bash[21480]: cluster 2024-03-04T16:04:39.722545+0000 mon.a (mon.0) 3032 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:41.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:40 smithi117 bash[21480]: audit 2024-03-04T16:04:39.737108+0000 mon.c (mon.2) 779 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:41.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:40 smithi117 bash[21480]: audit 2024-03-04T16:04:39.747533+0000 mon.a (mon.0) 3033 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:41.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:40 smithi117 bash[21480]: audit 2024-03-04T16:04:40.707390+0000 mon.a (mon.0) 3034 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:04:41.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:40 smithi117 bash[21480]: cluster 2024-03-04T16:04:40.716454+0000 mon.a (mon.0) 3035 : cluster 0 osdmap e565: 8 total, 8 up, 8 in 2024-03-04T16:04:41.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:40 smithi117 bash[21480]: audit 2024-03-04T16:04:40.718016+0000 mon.c (mon.2) 780 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-109", "tierpool": "test-rados-api-smithi117-64061-109-cache"}]: dispatch 2024-03-04T16:04:41.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:40 smithi117 bash[21480]: audit 2024-03-04T16:04:40.720254+0000 mon.a (mon.0) 3036 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-109", "tierpool": "test-rados-api-smithi117-64061-109-cache"}]: dispatch 2024-03-04T16:04:41.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:40 smithi117 bash[28843]: cluster 2024-03-04T16:04:39.719075+0000 mon.a (mon.0) 3031 : cluster 0 osdmap e564: 8 total, 8 up, 8 in 2024-03-04T16:04:41.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:40 smithi117 bash[28843]: cluster 2024-03-04T16:04:39.722545+0000 mon.a (mon.0) 3032 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:41.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:40 smithi117 bash[28843]: audit 2024-03-04T16:04:39.737108+0000 mon.c (mon.2) 779 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:41.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:40 smithi117 bash[28843]: audit 2024-03-04T16:04:39.747533+0000 mon.a (mon.0) 3033 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:41.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:40 smithi117 bash[28843]: audit 2024-03-04T16:04:40.707390+0000 mon.a (mon.0) 3034 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:04:41.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:40 smithi117 bash[28843]: cluster 2024-03-04T16:04:40.716454+0000 mon.a (mon.0) 3035 : cluster 0 osdmap e565: 8 total, 8 up, 8 in 2024-03-04T16:04:41.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:40 smithi117 bash[28843]: audit 2024-03-04T16:04:40.718016+0000 mon.c (mon.2) 780 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-109", "tierpool": "test-rados-api-smithi117-64061-109-cache"}]: dispatch 2024-03-04T16:04:41.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:40 smithi117 bash[28843]: audit 2024-03-04T16:04:40.720254+0000 mon.a (mon.0) 3036 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-109", "tierpool": "test-rados-api-smithi117-64061-109-cache"}]: dispatch 2024-03-04T16:04:41.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:40 smithi151 bash[23756]: cluster 2024-03-04T16:04:39.719075+0000 mon.a (mon.0) 3031 : cluster 0 osdmap e564: 8 total, 8 up, 8 in 2024-03-04T16:04:41.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:40 smithi151 bash[23756]: cluster 2024-03-04T16:04:39.722545+0000 mon.a (mon.0) 3032 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:41.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:40 smithi151 bash[23756]: audit 2024-03-04T16:04:39.737108+0000 mon.c (mon.2) 779 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:41.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:40 smithi151 bash[23756]: audit 2024-03-04T16:04:39.747533+0000 mon.a (mon.0) 3033 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:04:41.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:40 smithi151 bash[23756]: audit 2024-03-04T16:04:40.707390+0000 mon.a (mon.0) 3034 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:04:41.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:40 smithi151 bash[23756]: cluster 2024-03-04T16:04:40.716454+0000 mon.a (mon.0) 3035 : cluster 0 osdmap e565: 8 total, 8 up, 8 in 2024-03-04T16:04:41.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:40 smithi151 bash[23756]: audit 2024-03-04T16:04:40.718016+0000 mon.c (mon.2) 780 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-109", "tierpool": "test-rados-api-smithi117-64061-109-cache"}]: dispatch 2024-03-04T16:04:41.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:40 smithi151 bash[23756]: audit 2024-03-04T16:04:40.720254+0000 mon.a (mon.0) 3036 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-109", "tierpool": "test-rados-api-smithi117-64061-109-cache"}]: dispatch 2024-03-04T16:04:42.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:41 smithi117 bash[21480]: cluster 2024-03-04T16:04:41.129606+0000 mgr.y (mgr.24428) 580 : cluster 0 pgmap v917: 212 pgs: 40 unknown, 172 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:42.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:41 smithi117 bash[21480]: audit 2024-03-04T16:04:41.710545+0000 mon.a (mon.0) 3037 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-109", "tierpool": "test-rados-api-smithi117-64061-109-cache"}]': finished 2024-03-04T16:04:42.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:41 smithi117 bash[21480]: cluster 2024-03-04T16:04:41.718490+0000 mon.a (mon.0) 3038 : cluster 0 osdmap e566: 8 total, 8 up, 8 in 2024-03-04T16:04:42.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:41 smithi117 bash[21480]: audit 2024-03-04T16:04:41.720125+0000 mon.c (mon.2) 781 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-109", "overlaypool": "test-rados-api-smithi117-64061-109-cache"}]: dispatch 2024-03-04T16:04:42.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:41 smithi117 bash[21480]: audit 2024-03-04T16:04:41.721327+0000 mon.a (mon.0) 3039 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-109", "overlaypool": "test-rados-api-smithi117-64061-109-cache"}]: dispatch 2024-03-04T16:04:42.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:41 smithi117 bash[28843]: cluster 2024-03-04T16:04:41.129606+0000 mgr.y (mgr.24428) 580 : cluster 0 pgmap v917: 212 pgs: 40 unknown, 172 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:42.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:41 smithi117 bash[28843]: audit 2024-03-04T16:04:41.710545+0000 mon.a (mon.0) 3037 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-109", "tierpool": "test-rados-api-smithi117-64061-109-cache"}]': finished 2024-03-04T16:04:42.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:41 smithi117 bash[28843]: cluster 2024-03-04T16:04:41.718490+0000 mon.a (mon.0) 3038 : cluster 0 osdmap e566: 8 total, 8 up, 8 in 2024-03-04T16:04:42.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:41 smithi117 bash[28843]: audit 2024-03-04T16:04:41.720125+0000 mon.c (mon.2) 781 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-109", "overlaypool": "test-rados-api-smithi117-64061-109-cache"}]: dispatch 2024-03-04T16:04:42.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:41 smithi117 bash[28843]: audit 2024-03-04T16:04:41.721327+0000 mon.a (mon.0) 3039 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-109", "overlaypool": "test-rados-api-smithi117-64061-109-cache"}]: dispatch 2024-03-04T16:04:42.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:41 smithi151 bash[23756]: cluster 2024-03-04T16:04:41.129606+0000 mgr.y (mgr.24428) 580 : cluster 0 pgmap v917: 212 pgs: 40 unknown, 172 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:42.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:41 smithi151 bash[23756]: audit 2024-03-04T16:04:41.710545+0000 mon.a (mon.0) 3037 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-109", "tierpool": "test-rados-api-smithi117-64061-109-cache"}]': finished 2024-03-04T16:04:42.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:41 smithi151 bash[23756]: cluster 2024-03-04T16:04:41.718490+0000 mon.a (mon.0) 3038 : cluster 0 osdmap e566: 8 total, 8 up, 8 in 2024-03-04T16:04:42.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:41 smithi151 bash[23756]: audit 2024-03-04T16:04:41.720125+0000 mon.c (mon.2) 781 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-109", "overlaypool": "test-rados-api-smithi117-64061-109-cache"}]: dispatch 2024-03-04T16:04:42.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:41 smithi151 bash[23756]: audit 2024-03-04T16:04:41.721327+0000 mon.a (mon.0) 3039 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-109", "overlaypool": "test-rados-api-smithi117-64061-109-cache"}]: dispatch 2024-03-04T16:04:44.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:43 smithi117 bash[21480]: audit 2024-03-04T16:04:42.713438+0000 mon.a (mon.0) 3040 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-109", "overlaypool": "test-rados-api-smithi117-64061-109-cache"}]': finished 2024-03-04T16:04:44.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:43 smithi117 bash[21480]: cluster 2024-03-04T16:04:42.725392+0000 mon.a (mon.0) 3041 : cluster 0 osdmap e567: 8 total, 8 up, 8 in 2024-03-04T16:04:44.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:43 smithi117 bash[21480]: audit 2024-03-04T16:04:42.727304+0000 mon.c (mon.2) 782 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-109-cache", "mode": "writeback"}]: dispatch 2024-03-04T16:04:44.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:43 smithi117 bash[21480]: audit 2024-03-04T16:04:42.728228+0000 mon.a (mon.0) 3042 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-109-cache", "mode": "writeback"}]: dispatch 2024-03-04T16:04:44.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:43 smithi117 bash[21480]: cluster 2024-03-04T16:04:43.130504+0000 mgr.y (mgr.24428) 581 : cluster 0 pgmap v920: 212 pgs: 212 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:44.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:43 smithi117 bash[28843]: audit 2024-03-04T16:04:42.713438+0000 mon.a (mon.0) 3040 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-109", "overlaypool": "test-rados-api-smithi117-64061-109-cache"}]': finished 2024-03-04T16:04:44.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:43 smithi117 bash[28843]: cluster 2024-03-04T16:04:42.725392+0000 mon.a (mon.0) 3041 : cluster 0 osdmap e567: 8 total, 8 up, 8 in 2024-03-04T16:04:44.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:43 smithi117 bash[28843]: audit 2024-03-04T16:04:42.727304+0000 mon.c (mon.2) 782 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-109-cache", "mode": "writeback"}]: dispatch 2024-03-04T16:04:44.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:43 smithi117 bash[28843]: audit 2024-03-04T16:04:42.728228+0000 mon.a (mon.0) 3042 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-109-cache", "mode": "writeback"}]: dispatch 2024-03-04T16:04:44.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:43 smithi117 bash[28843]: cluster 2024-03-04T16:04:43.130504+0000 mgr.y (mgr.24428) 581 : cluster 0 pgmap v920: 212 pgs: 212 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:44.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:43 smithi151 bash[23756]: audit 2024-03-04T16:04:42.713438+0000 mon.a (mon.0) 3040 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-109", "overlaypool": "test-rados-api-smithi117-64061-109-cache"}]': finished 2024-03-04T16:04:44.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:43 smithi151 bash[23756]: cluster 2024-03-04T16:04:42.725392+0000 mon.a (mon.0) 3041 : cluster 0 osdmap e567: 8 total, 8 up, 8 in 2024-03-04T16:04:44.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:43 smithi151 bash[23756]: audit 2024-03-04T16:04:42.727304+0000 mon.c (mon.2) 782 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-109-cache", "mode": "writeback"}]: dispatch 2024-03-04T16:04:44.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:43 smithi151 bash[23756]: audit 2024-03-04T16:04:42.728228+0000 mon.a (mon.0) 3042 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-109-cache", "mode": "writeback"}]: dispatch 2024-03-04T16:04:44.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:43 smithi151 bash[23756]: cluster 2024-03-04T16:04:43.130504+0000 mgr.y (mgr.24428) 581 : cluster 0 pgmap v920: 212 pgs: 212 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:45.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:44 smithi117 bash[21480]: cluster 2024-03-04T16:04:43.714296+0000 mon.a (mon.0) 3043 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:04:45.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:44 smithi117 bash[21480]: audit 2024-03-04T16:04:43.724488+0000 mon.a (mon.0) 3044 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-109-cache", "mode": "writeback"}]': finished 2024-03-04T16:04:45.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:44 smithi117 bash[21480]: audit 2024-03-04T16:04:43.736633+0000 mon.c (mon.2) 783 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:04:45.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:44 smithi117 bash[21480]: cluster 2024-03-04T16:04:43.743802+0000 mon.a (mon.0) 3045 : cluster 0 osdmap e568: 8 total, 8 up, 8 in 2024-03-04T16:04:45.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:44 smithi117 bash[21480]: audit 2024-03-04T16:04:43.745816+0000 mon.a (mon.0) 3046 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:04:45.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:44 smithi117 bash[28843]: cluster 2024-03-04T16:04:43.714296+0000 mon.a (mon.0) 3043 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:04:45.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:44 smithi117 bash[28843]: audit 2024-03-04T16:04:43.724488+0000 mon.a (mon.0) 3044 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-109-cache", "mode": "writeback"}]': finished 2024-03-04T16:04:45.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:44 smithi117 bash[28843]: audit 2024-03-04T16:04:43.736633+0000 mon.c (mon.2) 783 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:04:45.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:44 smithi117 bash[28843]: cluster 2024-03-04T16:04:43.743802+0000 mon.a (mon.0) 3045 : cluster 0 osdmap e568: 8 total, 8 up, 8 in 2024-03-04T16:04:45.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:44 smithi117 bash[28843]: audit 2024-03-04T16:04:43.745816+0000 mon.a (mon.0) 3046 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:04:45.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:44 smithi151 bash[23756]: cluster 2024-03-04T16:04:43.714296+0000 mon.a (mon.0) 3043 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:04:45.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:44 smithi151 bash[23756]: audit 2024-03-04T16:04:43.724488+0000 mon.a (mon.0) 3044 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-109-cache", "mode": "writeback"}]': finished 2024-03-04T16:04:45.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:44 smithi151 bash[23756]: audit 2024-03-04T16:04:43.736633+0000 mon.c (mon.2) 783 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:04:45.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:44 smithi151 bash[23756]: cluster 2024-03-04T16:04:43.743802+0000 mon.a (mon.0) 3045 : cluster 0 osdmap e568: 8 total, 8 up, 8 in 2024-03-04T16:04:45.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:44 smithi151 bash[23756]: audit 2024-03-04T16:04:43.745816+0000 mon.a (mon.0) 3046 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:04:46.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:45 smithi117 bash[21480]: audit 2024-03-04T16:04:44.730204+0000 mon.a (mon.0) 3047 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:04:46.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:45 smithi117 bash[21480]: audit 2024-03-04T16:04:44.733336+0000 mon.a (mon.0) 3048 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-03-04T16:04:46.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:45 smithi117 bash[21480]: audit 2024-03-04T16:04:44.743495+0000 mon.c (mon.2) 784 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:04:46.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:45 smithi117 bash[21480]: cluster 2024-03-04T16:04:44.744360+0000 mon.a (mon.0) 3049 : cluster 0 osdmap e569: 8 total, 8 up, 8 in 2024-03-04T16:04:46.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:45 smithi117 bash[21480]: audit 2024-03-04T16:04:44.746367+0000 mon.c (mon.2) 785 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:04:46.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:45 smithi117 bash[21480]: audit 2024-03-04T16:04:44.748404+0000 mon.a (mon.0) 3050 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:04:46.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:45 smithi117 bash[21480]: cluster 2024-03-04T16:04:45.131346+0000 mgr.y (mgr.24428) 582 : cluster 0 pgmap v923: 212 pgs: 212 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:46.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:45 smithi117 bash[28843]: audit 2024-03-04T16:04:44.730204+0000 mon.a (mon.0) 3047 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:04:46.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:45 smithi117 bash[28843]: audit 2024-03-04T16:04:44.733336+0000 mon.a (mon.0) 3048 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-03-04T16:04:46.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:45 smithi117 bash[28843]: audit 2024-03-04T16:04:44.743495+0000 mon.c (mon.2) 784 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:04:46.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:45 smithi117 bash[28843]: cluster 2024-03-04T16:04:44.744360+0000 mon.a (mon.0) 3049 : cluster 0 osdmap e569: 8 total, 8 up, 8 in 2024-03-04T16:04:46.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:45 smithi117 bash[28843]: audit 2024-03-04T16:04:44.746367+0000 mon.c (mon.2) 785 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:04:46.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:45 smithi117 bash[28843]: audit 2024-03-04T16:04:44.748404+0000 mon.a (mon.0) 3050 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:04:46.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:45 smithi117 bash[28843]: cluster 2024-03-04T16:04:45.131346+0000 mgr.y (mgr.24428) 582 : cluster 0 pgmap v923: 212 pgs: 212 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:46.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:45 smithi151 bash[23756]: audit 2024-03-04T16:04:44.730204+0000 mon.a (mon.0) 3047 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:04:46.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:45 smithi151 bash[23756]: audit 2024-03-04T16:04:44.733336+0000 mon.a (mon.0) 3048 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-03-04T16:04:46.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:45 smithi151 bash[23756]: audit 2024-03-04T16:04:44.743495+0000 mon.c (mon.2) 784 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:04:46.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:45 smithi151 bash[23756]: cluster 2024-03-04T16:04:44.744360+0000 mon.a (mon.0) 3049 : cluster 0 osdmap e569: 8 total, 8 up, 8 in 2024-03-04T16:04:46.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:45 smithi151 bash[23756]: audit 2024-03-04T16:04:44.746367+0000 mon.c (mon.2) 785 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:04:46.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:45 smithi151 bash[23756]: audit 2024-03-04T16:04:44.748404+0000 mon.a (mon.0) 3050 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:04:46.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:45 smithi151 bash[23756]: cluster 2024-03-04T16:04:45.131346+0000 mgr.y (mgr.24428) 582 : cluster 0 pgmap v923: 212 pgs: 212 active+clean; 582 KiB data, 640 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:46.566 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:04:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:04:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:04:47.052 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:46 smithi151 bash[23756]: audit 2024-03-04T16:04:45.742292+0000 mon.a (mon.0) 3051 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-03-04T16:04:47.052 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:46 smithi151 bash[23756]: cluster 2024-03-04T16:04:45.752384+0000 mon.a (mon.0) 3052 : cluster 0 osdmap e570: 8 total, 8 up, 8 in 2024-03-04T16:04:47.053 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:46 smithi151 bash[23756]: audit 2024-03-04T16:04:45.754549+0000 mon.c (mon.2) 786 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T16:04:47.053 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:46 smithi151 bash[23756]: audit 2024-03-04T16:04:45.755582+0000 mon.a (mon.0) 3053 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T16:04:47.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:46 smithi117 bash[21480]: audit 2024-03-04T16:04:45.742292+0000 mon.a (mon.0) 3051 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-03-04T16:04:47.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:46 smithi117 bash[21480]: cluster 2024-03-04T16:04:45.752384+0000 mon.a (mon.0) 3052 : cluster 0 osdmap e570: 8 total, 8 up, 8 in 2024-03-04T16:04:47.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:46 smithi117 bash[21480]: audit 2024-03-04T16:04:45.754549+0000 mon.c (mon.2) 786 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T16:04:47.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:46 smithi117 bash[21480]: audit 2024-03-04T16:04:45.755582+0000 mon.a (mon.0) 3053 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T16:04:47.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:46 smithi117 bash[28843]: audit 2024-03-04T16:04:45.742292+0000 mon.a (mon.0) 3051 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-03-04T16:04:47.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:46 smithi117 bash[28843]: cluster 2024-03-04T16:04:45.752384+0000 mon.a (mon.0) 3052 : cluster 0 osdmap e570: 8 total, 8 up, 8 in 2024-03-04T16:04:47.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:46 smithi117 bash[28843]: audit 2024-03-04T16:04:45.754549+0000 mon.c (mon.2) 786 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T16:04:47.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:46 smithi117 bash[28843]: audit 2024-03-04T16:04:45.755582+0000 mon.a (mon.0) 3053 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T16:04:47.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:04:47 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:04:48.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:47 smithi117 bash[21480]: cluster 2024-03-04T16:04:46.743280+0000 mon.a (mon.0) 3054 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:04:48.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:47 smithi117 bash[21480]: audit 2024-03-04T16:04:46.755957+0000 mon.a (mon.0) 3055 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-04T16:04:48.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:47 smithi117 bash[21480]: cluster 2024-03-04T16:04:46.764738+0000 mon.a (mon.0) 3056 : cluster 0 osdmap e571: 8 total, 8 up, 8 in 2024-03-04T16:04:48.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:47 smithi117 bash[21480]: audit 2024-03-04T16:04:46.767151+0000 mon.c (mon.2) 787 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-04T16:04:48.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:47 smithi117 bash[21480]: audit 2024-03-04T16:04:46.769261+0000 mon.a (mon.0) 3057 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-04T16:04:48.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:47 smithi117 bash[21480]: audit 2024-03-04T16:04:47.056698+0000 mgr.y (mgr.24428) 583 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:04:48.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:47 smithi117 bash[21480]: cluster 2024-03-04T16:04:47.132140+0000 mgr.y (mgr.24428) 584 : cluster 0 pgmap v926: 212 pgs: 212 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:48.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:47 smithi117 bash[28843]: cluster 2024-03-04T16:04:46.743280+0000 mon.a (mon.0) 3054 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:04:48.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:47 smithi117 bash[28843]: audit 2024-03-04T16:04:46.755957+0000 mon.a (mon.0) 3055 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-04T16:04:48.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:47 smithi117 bash[28843]: cluster 2024-03-04T16:04:46.764738+0000 mon.a (mon.0) 3056 : cluster 0 osdmap e571: 8 total, 8 up, 8 in 2024-03-04T16:04:48.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:47 smithi117 bash[28843]: audit 2024-03-04T16:04:46.767151+0000 mon.c (mon.2) 787 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-04T16:04:48.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:47 smithi117 bash[28843]: audit 2024-03-04T16:04:46.769261+0000 mon.a (mon.0) 3057 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-04T16:04:48.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:47 smithi117 bash[28843]: audit 2024-03-04T16:04:47.056698+0000 mgr.y (mgr.24428) 583 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:04:48.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:47 smithi117 bash[28843]: cluster 2024-03-04T16:04:47.132140+0000 mgr.y (mgr.24428) 584 : cluster 0 pgmap v926: 212 pgs: 212 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:48.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:47 smithi151 bash[23756]: cluster 2024-03-04T16:04:46.743280+0000 mon.a (mon.0) 3054 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:04:48.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:47 smithi151 bash[23756]: audit 2024-03-04T16:04:46.755957+0000 mon.a (mon.0) 3055 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-04T16:04:48.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:47 smithi151 bash[23756]: cluster 2024-03-04T16:04:46.764738+0000 mon.a (mon.0) 3056 : cluster 0 osdmap e571: 8 total, 8 up, 8 in 2024-03-04T16:04:48.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:47 smithi151 bash[23756]: audit 2024-03-04T16:04:46.767151+0000 mon.c (mon.2) 787 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-04T16:04:48.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:47 smithi151 bash[23756]: audit 2024-03-04T16:04:46.769261+0000 mon.a (mon.0) 3057 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-03-04T16:04:48.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:47 smithi151 bash[23756]: audit 2024-03-04T16:04:47.056698+0000 mgr.y (mgr.24428) 583 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:04:48.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:47 smithi151 bash[23756]: cluster 2024-03-04T16:04:47.132140+0000 mgr.y (mgr.24428) 584 : cluster 0 pgmap v926: 212 pgs: 212 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:49.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:48 smithi117 bash[21480]: audit 2024-03-04T16:04:47.767619+0000 mon.a (mon.0) 3058 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-03-04T16:04:49.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:48 smithi117 bash[21480]: cluster 2024-03-04T16:04:47.773324+0000 mon.a (mon.0) 3059 : cluster 0 osdmap e572: 8 total, 8 up, 8 in 2024-03-04T16:04:49.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:48 smithi117 bash[21480]: audit 2024-03-04T16:04:47.867178+0000 mon.c (mon.2) 788 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:49.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:48 smithi117 bash[21480]: audit 2024-03-04T16:04:47.867869+0000 mon.a (mon.0) 3060 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:49.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:48 smithi117 bash[28843]: audit 2024-03-04T16:04:47.767619+0000 mon.a (mon.0) 3058 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-03-04T16:04:49.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:48 smithi117 bash[28843]: cluster 2024-03-04T16:04:47.773324+0000 mon.a (mon.0) 3059 : cluster 0 osdmap e572: 8 total, 8 up, 8 in 2024-03-04T16:04:49.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:48 smithi117 bash[28843]: audit 2024-03-04T16:04:47.867178+0000 mon.c (mon.2) 788 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:49.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:48 smithi117 bash[28843]: audit 2024-03-04T16:04:47.867869+0000 mon.a (mon.0) 3060 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:49.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:48 smithi151 bash[23756]: audit 2024-03-04T16:04:47.767619+0000 mon.a (mon.0) 3058 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-03-04T16:04:49.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:48 smithi151 bash[23756]: cluster 2024-03-04T16:04:47.773324+0000 mon.a (mon.0) 3059 : cluster 0 osdmap e572: 8 total, 8 up, 8 in 2024-03-04T16:04:49.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:48 smithi151 bash[23756]: audit 2024-03-04T16:04:47.867178+0000 mon.c (mon.2) 788 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:49.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:48 smithi151 bash[23756]: audit 2024-03-04T16:04:47.867869+0000 mon.a (mon.0) 3060 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:50.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:49 smithi117 bash[21480]: audit 2024-03-04T16:04:48.780499+0000 mon.a (mon.0) 3061 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-109"}]': finished 2024-03-04T16:04:50.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:49 smithi117 bash[21480]: cluster 2024-03-04T16:04:48.789691+0000 mon.a (mon.0) 3062 : cluster 0 osdmap e573: 8 total, 8 up, 8 in 2024-03-04T16:04:50.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:49 smithi117 bash[21480]: audit 2024-03-04T16:04:48.792198+0000 mon.c (mon.2) 789 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-109", "tierpool": "test-rados-api-smithi117-64061-109-cache"}]: dispatch 2024-03-04T16:04:50.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:49 smithi117 bash[21480]: audit 2024-03-04T16:04:48.793093+0000 mon.a (mon.0) 3063 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-109", "tierpool": "test-rados-api-smithi117-64061-109-cache"}]: dispatch 2024-03-04T16:04:50.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:49 smithi117 bash[21480]: cluster 2024-03-04T16:04:49.132993+0000 mgr.y (mgr.24428) 585 : cluster 0 pgmap v929: 212 pgs: 212 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:50.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:49 smithi117 bash[28843]: audit 2024-03-04T16:04:48.780499+0000 mon.a (mon.0) 3061 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-109"}]': finished 2024-03-04T16:04:50.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:49 smithi117 bash[28843]: cluster 2024-03-04T16:04:48.789691+0000 mon.a (mon.0) 3062 : cluster 0 osdmap e573: 8 total, 8 up, 8 in 2024-03-04T16:04:50.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:49 smithi117 bash[28843]: audit 2024-03-04T16:04:48.792198+0000 mon.c (mon.2) 789 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-109", "tierpool": "test-rados-api-smithi117-64061-109-cache"}]: dispatch 2024-03-04T16:04:50.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:49 smithi117 bash[28843]: audit 2024-03-04T16:04:48.793093+0000 mon.a (mon.0) 3063 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-109", "tierpool": "test-rados-api-smithi117-64061-109-cache"}]: dispatch 2024-03-04T16:04:50.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:49 smithi117 bash[28843]: cluster 2024-03-04T16:04:49.132993+0000 mgr.y (mgr.24428) 585 : cluster 0 pgmap v929: 212 pgs: 212 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:50.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:49 smithi151 bash[23756]: audit 2024-03-04T16:04:48.780499+0000 mon.a (mon.0) 3061 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-109"}]': finished 2024-03-04T16:04:50.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:49 smithi151 bash[23756]: cluster 2024-03-04T16:04:48.789691+0000 mon.a (mon.0) 3062 : cluster 0 osdmap e573: 8 total, 8 up, 8 in 2024-03-04T16:04:50.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:49 smithi151 bash[23756]: audit 2024-03-04T16:04:48.792198+0000 mon.c (mon.2) 789 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-109", "tierpool": "test-rados-api-smithi117-64061-109-cache"}]: dispatch 2024-03-04T16:04:50.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:49 smithi151 bash[23756]: audit 2024-03-04T16:04:48.793093+0000 mon.a (mon.0) 3063 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-109", "tierpool": "test-rados-api-smithi117-64061-109-cache"}]: dispatch 2024-03-04T16:04:50.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:49 smithi151 bash[23756]: cluster 2024-03-04T16:04:49.132993+0000 mgr.y (mgr.24428) 585 : cluster 0 pgmap v929: 212 pgs: 212 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:51.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:50 smithi117 bash[21480]: audit 2024-03-04T16:04:49.789559+0000 mon.a (mon.0) 3064 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-109", "tierpool": "test-rados-api-smithi117-64061-109-cache"}]': finished 2024-03-04T16:04:51.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:50 smithi117 bash[21480]: cluster 2024-03-04T16:04:49.799748+0000 mon.a (mon.0) 3065 : cluster 0 osdmap e574: 8 total, 8 up, 8 in 2024-03-04T16:04:51.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:50 smithi117 bash[28843]: audit 2024-03-04T16:04:49.789559+0000 mon.a (mon.0) 3064 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-109", "tierpool": "test-rados-api-smithi117-64061-109-cache"}]': finished 2024-03-04T16:04:51.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:50 smithi117 bash[28843]: cluster 2024-03-04T16:04:49.799748+0000 mon.a (mon.0) 3065 : cluster 0 osdmap e574: 8 total, 8 up, 8 in 2024-03-04T16:04:51.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:50 smithi151 bash[23756]: audit 2024-03-04T16:04:49.789559+0000 mon.a (mon.0) 3064 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-109", "tierpool": "test-rados-api-smithi117-64061-109-cache"}]': finished 2024-03-04T16:04:51.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:50 smithi151 bash[23756]: cluster 2024-03-04T16:04:49.799748+0000 mon.a (mon.0) 3065 : cluster 0 osdmap e574: 8 total, 8 up, 8 in 2024-03-04T16:04:52.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:51 smithi117 bash[21480]: cluster 2024-03-04T16:04:50.799318+0000 mon.a (mon.0) 3066 : cluster 0 osdmap e575: 8 total, 8 up, 8 in 2024-03-04T16:04:52.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:51 smithi117 bash[21480]: cluster 2024-03-04T16:04:51.133858+0000 mgr.y (mgr.24428) 586 : cluster 0 pgmap v932: 180 pgs: 180 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:52.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:51 smithi117 bash[21480]: audit 2024-03-04T16:04:51.596599+0000 mon.c (mon.2) 790 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:04:52.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:51 smithi117 bash[28843]: cluster 2024-03-04T16:04:50.799318+0000 mon.a (mon.0) 3066 : cluster 0 osdmap e575: 8 total, 8 up, 8 in 2024-03-04T16:04:52.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:51 smithi117 bash[28843]: cluster 2024-03-04T16:04:51.133858+0000 mgr.y (mgr.24428) 586 : cluster 0 pgmap v932: 180 pgs: 180 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:52.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:51 smithi117 bash[28843]: audit 2024-03-04T16:04:51.596599+0000 mon.c (mon.2) 790 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:04:52.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:51 smithi151 bash[23756]: cluster 2024-03-04T16:04:50.799318+0000 mon.a (mon.0) 3066 : cluster 0 osdmap e575: 8 total, 8 up, 8 in 2024-03-04T16:04:52.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:51 smithi151 bash[23756]: cluster 2024-03-04T16:04:51.133858+0000 mgr.y (mgr.24428) 586 : cluster 0 pgmap v932: 180 pgs: 180 active+clean; 582 KiB data, 641 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:04:52.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:51 smithi151 bash[23756]: audit 2024-03-04T16:04:51.596599+0000 mon.c (mon.2) 790 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:04:53.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:52 smithi151 bash[23756]: cluster 2024-03-04T16:04:51.824561+0000 mon.a (mon.0) 3067 : cluster 0 osdmap e576: 8 total, 8 up, 8 in 2024-03-04T16:04:53.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:52 smithi151 bash[23756]: audit 2024-03-04T16:04:51.831662+0000 mon.c (mon.2) 791 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:53.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:52 smithi151 bash[23756]: audit 2024-03-04T16:04:51.834421+0000 mon.a (mon.0) 3068 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:53.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:52 smithi151 bash[23756]: audit 2024-03-04T16:04:52.378697+0000 mon.c (mon.2) 792 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:04:53.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:52 smithi151 bash[23756]: audit 2024-03-04T16:04:52.380055+0000 mon.c (mon.2) 793 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:04:53.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:52 smithi151 bash[23756]: audit 2024-03-04T16:04:52.388705+0000 mon.a (mon.0) 3069 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:04:53.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:52 smithi117 bash[21480]: cluster 2024-03-04T16:04:51.824561+0000 mon.a (mon.0) 3067 : cluster 0 osdmap e576: 8 total, 8 up, 8 in 2024-03-04T16:04:53.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:52 smithi117 bash[21480]: audit 2024-03-04T16:04:51.831662+0000 mon.c (mon.2) 791 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:53.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:52 smithi117 bash[21480]: audit 2024-03-04T16:04:51.834421+0000 mon.a (mon.0) 3068 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:53.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:52 smithi117 bash[21480]: audit 2024-03-04T16:04:52.378697+0000 mon.c (mon.2) 792 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:04:53.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:52 smithi117 bash[21480]: audit 2024-03-04T16:04:52.380055+0000 mon.c (mon.2) 793 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:04:53.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:52 smithi117 bash[21480]: audit 2024-03-04T16:04:52.388705+0000 mon.a (mon.0) 3069 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:04:53.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:52 smithi117 bash[28843]: cluster 2024-03-04T16:04:51.824561+0000 mon.a (mon.0) 3067 : cluster 0 osdmap e576: 8 total, 8 up, 8 in 2024-03-04T16:04:53.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:52 smithi117 bash[28843]: audit 2024-03-04T16:04:51.831662+0000 mon.c (mon.2) 791 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:53.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:52 smithi117 bash[28843]: audit 2024-03-04T16:04:51.834421+0000 mon.a (mon.0) 3068 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:53.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:52 smithi117 bash[28843]: audit 2024-03-04T16:04:52.378697+0000 mon.c (mon.2) 792 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:04:53.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:52 smithi117 bash[28843]: audit 2024-03-04T16:04:52.380055+0000 mon.c (mon.2) 793 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:04:53.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:52 smithi117 bash[28843]: audit 2024-03-04T16:04:52.388705+0000 mon.a (mon.0) 3069 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:04:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:53 smithi151 bash[23756]: audit 2024-03-04T16:04:52.821507+0000 mon.a (mon.0) 3070 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-109"}]': finished 2024-03-04T16:04:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:53 smithi151 bash[23756]: cluster 2024-03-04T16:04:52.828537+0000 mon.a (mon.0) 3071 : cluster 0 osdmap e577: 8 total, 8 up, 8 in 2024-03-04T16:04:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:53 smithi151 bash[23756]: audit 2024-03-04T16:04:52.832063+0000 mon.c (mon.2) 794 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:53 smithi151 bash[23756]: audit 2024-03-04T16:04:52.842165+0000 mon.a (mon.0) 3072 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:53 smithi151 bash[23756]: cluster 2024-03-04T16:04:53.134723+0000 mgr.y (mgr.24428) 587 : cluster 0 pgmap v935: 172 pgs: 172 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:54.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:53 smithi117 bash[21480]: audit 2024-03-04T16:04:52.821507+0000 mon.a (mon.0) 3070 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-109"}]': finished 2024-03-04T16:04:54.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:53 smithi117 bash[21480]: cluster 2024-03-04T16:04:52.828537+0000 mon.a (mon.0) 3071 : cluster 0 osdmap e577: 8 total, 8 up, 8 in 2024-03-04T16:04:54.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:53 smithi117 bash[21480]: audit 2024-03-04T16:04:52.832063+0000 mon.c (mon.2) 794 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:54.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:53 smithi117 bash[21480]: audit 2024-03-04T16:04:52.842165+0000 mon.a (mon.0) 3072 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:54.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:53 smithi117 bash[21480]: cluster 2024-03-04T16:04:53.134723+0000 mgr.y (mgr.24428) 587 : cluster 0 pgmap v935: 172 pgs: 172 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:54.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:53 smithi117 bash[28843]: audit 2024-03-04T16:04:52.821507+0000 mon.a (mon.0) 3070 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-109"}]': finished 2024-03-04T16:04:54.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:53 smithi117 bash[28843]: cluster 2024-03-04T16:04:52.828537+0000 mon.a (mon.0) 3071 : cluster 0 osdmap e577: 8 total, 8 up, 8 in 2024-03-04T16:04:54.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:53 smithi117 bash[28843]: audit 2024-03-04T16:04:52.832063+0000 mon.c (mon.2) 794 : audit 1 from='client.? 172.21.15.117:0/3706884503' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:54.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:53 smithi117 bash[28843]: audit 2024-03-04T16:04:52.842165+0000 mon.a (mon.0) 3072 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-109"}]: dispatch 2024-03-04T16:04:54.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:53 smithi117 bash[28843]: cluster 2024-03-04T16:04:53.134723+0000 mgr.y (mgr.24428) 587 : cluster 0 pgmap v935: 172 pgs: 172 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:04:55.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:54 smithi151 bash[23756]: cluster 2024-03-04T16:04:53.835206+0000 mon.a (mon.0) 3073 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:55.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:54 smithi151 bash[23756]: audit 2024-03-04T16:04:53.846440+0000 mon.a (mon.0) 3074 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-109"}]': finished 2024-03-04T16:04:55.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:54 smithi151 bash[23756]: cluster 2024-03-04T16:04:53.853728+0000 mon.a (mon.0) 3075 : cluster 0 osdmap e578: 8 total, 8 up, 8 in 2024-03-04T16:04:55.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:54 smithi117 bash[21480]: cluster 2024-03-04T16:04:53.835206+0000 mon.a (mon.0) 3073 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:55.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:54 smithi117 bash[21480]: audit 2024-03-04T16:04:53.846440+0000 mon.a (mon.0) 3074 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-109"}]': finished 2024-03-04T16:04:55.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:54 smithi117 bash[21480]: cluster 2024-03-04T16:04:53.853728+0000 mon.a (mon.0) 3075 : cluster 0 osdmap e578: 8 total, 8 up, 8 in 2024-03-04T16:04:55.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:54 smithi117 bash[28843]: cluster 2024-03-04T16:04:53.835206+0000 mon.a (mon.0) 3073 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:04:55.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:54 smithi117 bash[28843]: audit 2024-03-04T16:04:53.846440+0000 mon.a (mon.0) 3074 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-109"}]': finished 2024-03-04T16:04:55.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:54 smithi117 bash[28843]: cluster 2024-03-04T16:04:53.853728+0000 mon.a (mon.0) 3075 : cluster 0 osdmap e578: 8 total, 8 up, 8 in 2024-03-04T16:04:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:55 smithi151 bash[23756]: cluster 2024-03-04T16:04:54.855698+0000 mon.a (mon.0) 3076 : cluster 0 osdmap e579: 8 total, 8 up, 8 in 2024-03-04T16:04:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:55 smithi151 bash[23756]: audit 2024-03-04T16:04:54.859882+0000 mon.c (mon.2) 795 : audit 1 from='client.? 172.21.15.117:0/2461560686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:55 smithi151 bash[23756]: audit 2024-03-04T16:04:54.860722+0000 mon.a (mon.0) 3077 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:55 smithi151 bash[23756]: cluster 2024-03-04T16:04:55.135505+0000 mgr.y (mgr.24428) 588 : cluster 0 pgmap v938: 164 pgs: 164 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-04T16:04:56.316 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:04:56 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:04:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:04:56.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:55 smithi117 bash[21480]: cluster 2024-03-04T16:04:54.855698+0000 mon.a (mon.0) 3076 : cluster 0 osdmap e579: 8 total, 8 up, 8 in 2024-03-04T16:04:56.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:55 smithi117 bash[21480]: audit 2024-03-04T16:04:54.859882+0000 mon.c (mon.2) 795 : audit 1 from='client.? 172.21.15.117:0/2461560686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:56.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:55 smithi117 bash[21480]: audit 2024-03-04T16:04:54.860722+0000 mon.a (mon.0) 3077 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:56.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:55 smithi117 bash[21480]: cluster 2024-03-04T16:04:55.135505+0000 mgr.y (mgr.24428) 588 : cluster 0 pgmap v938: 164 pgs: 164 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-04T16:04:56.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:55 smithi117 bash[28843]: cluster 2024-03-04T16:04:54.855698+0000 mon.a (mon.0) 3076 : cluster 0 osdmap e579: 8 total, 8 up, 8 in 2024-03-04T16:04:56.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:55 smithi117 bash[28843]: audit 2024-03-04T16:04:54.859882+0000 mon.c (mon.2) 795 : audit 1 from='client.? 172.21.15.117:0/2461560686' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:56.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:55 smithi117 bash[28843]: audit 2024-03-04T16:04:54.860722+0000 mon.a (mon.0) 3077 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:56.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:55 smithi117 bash[28843]: cluster 2024-03-04T16:04:55.135505+0000 mgr.y (mgr.24428) 588 : cluster 0 pgmap v938: 164 pgs: 164 active+clean; 582 KiB data, 646 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-04T16:04:57.181 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:04:57 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:04:57.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:56 smithi151 bash[23756]: audit 2024-03-04T16:04:55.860498+0000 mon.a (mon.0) 3078 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104"}]': finished 2024-03-04T16:04:57.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:56 smithi151 bash[23756]: cluster 2024-03-04T16:04:55.869964+0000 mon.a (mon.0) 3079 : cluster 0 osdmap e580: 8 total, 8 up, 8 in 2024-03-04T16:04:57.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:56 smithi151 bash[23756]: audit 2024-03-04T16:04:55.871433+0000 mon.c (mon.2) 796 : audit 1 from='client.? 172.21.15.117:0/2461560686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:57.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:56 smithi151 bash[23756]: audit 2024-03-04T16:04:55.872786+0000 mon.a (mon.0) 3080 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:57.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:56 smithi117 bash[21480]: audit 2024-03-04T16:04:55.860498+0000 mon.a (mon.0) 3078 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104"}]': finished 2024-03-04T16:04:57.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:56 smithi117 bash[21480]: cluster 2024-03-04T16:04:55.869964+0000 mon.a (mon.0) 3079 : cluster 0 osdmap e580: 8 total, 8 up, 8 in 2024-03-04T16:04:57.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:56 smithi117 bash[21480]: audit 2024-03-04T16:04:55.871433+0000 mon.c (mon.2) 796 : audit 1 from='client.? 172.21.15.117:0/2461560686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:57.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:56 smithi117 bash[21480]: audit 2024-03-04T16:04:55.872786+0000 mon.a (mon.0) 3080 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:57.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:56 smithi117 bash[28843]: audit 2024-03-04T16:04:55.860498+0000 mon.a (mon.0) 3078 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi117-64061-104"}]': finished 2024-03-04T16:04:57.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:56 smithi117 bash[28843]: cluster 2024-03-04T16:04:55.869964+0000 mon.a (mon.0) 3079 : cluster 0 osdmap e580: 8 total, 8 up, 8 in 2024-03-04T16:04:57.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:56 smithi117 bash[28843]: audit 2024-03-04T16:04:55.871433+0000 mon.c (mon.2) 796 : audit 1 from='client.? 172.21.15.117:0/2461560686' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:57.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:56 smithi117 bash[28843]: audit 2024-03-04T16:04:55.872786+0000 mon.a (mon.0) 3080 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi117-64061-104"}]: dispatch 2024-03-04T16:04:58.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:57 smithi151 bash[23756]: audit 2024-03-04T16:04:56.863960+0000 mon.a (mon.0) 3081 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi117-64061-104"}]': finished 2024-03-04T16:04:58.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:57 smithi151 bash[23756]: cluster 2024-03-04T16:04:56.871647+0000 mon.a (mon.0) 3082 : cluster 0 osdmap e581: 8 total, 8 up, 8 in 2024-03-04T16:04:58.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:57 smithi151 bash[23756]: audit 2024-03-04T16:04:56.914110+0000 mon.a (mon.0) 3083 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:04:58.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:57 smithi151 bash[23756]: audit 2024-03-04T16:04:56.915376+0000 mon.a (mon.0) 3084 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:04:58.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:57 smithi151 bash[23756]: audit 2024-03-04T16:04:56.915775+0000 mon.a (mon.0) 3085 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-64061-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T16:04:58.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:57 smithi151 bash[23756]: audit 2024-03-04T16:04:57.064560+0000 mgr.y (mgr.24428) 589 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:04:58.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:57 smithi151 bash[23756]: cluster 2024-03-04T16:04:57.136257+0000 mgr.y (mgr.24428) 590 : cluster 0 pgmap v941: 164 pgs: 164 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-04T16:04:58.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:57 smithi117 bash[21480]: audit 2024-03-04T16:04:56.863960+0000 mon.a (mon.0) 3081 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi117-64061-104"}]': finished 2024-03-04T16:04:58.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:57 smithi117 bash[21480]: cluster 2024-03-04T16:04:56.871647+0000 mon.a (mon.0) 3082 : cluster 0 osdmap e581: 8 total, 8 up, 8 in 2024-03-04T16:04:58.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:57 smithi117 bash[21480]: audit 2024-03-04T16:04:56.914110+0000 mon.a (mon.0) 3083 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:04:58.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:57 smithi117 bash[21480]: audit 2024-03-04T16:04:56.915376+0000 mon.a (mon.0) 3084 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:04:58.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:57 smithi117 bash[21480]: audit 2024-03-04T16:04:56.915775+0000 mon.a (mon.0) 3085 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-64061-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T16:04:58.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:57 smithi117 bash[21480]: audit 2024-03-04T16:04:57.064560+0000 mgr.y (mgr.24428) 589 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:04:58.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:57 smithi117 bash[21480]: cluster 2024-03-04T16:04:57.136257+0000 mgr.y (mgr.24428) 590 : cluster 0 pgmap v941: 164 pgs: 164 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-04T16:04:58.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:57 smithi117 bash[28843]: audit 2024-03-04T16:04:56.863960+0000 mon.a (mon.0) 3081 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi117-64061-104"}]': finished 2024-03-04T16:04:58.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:57 smithi117 bash[28843]: cluster 2024-03-04T16:04:56.871647+0000 mon.a (mon.0) 3082 : cluster 0 osdmap e581: 8 total, 8 up, 8 in 2024-03-04T16:04:58.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:57 smithi117 bash[28843]: audit 2024-03-04T16:04:56.914110+0000 mon.a (mon.0) 3083 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:04:58.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:57 smithi117 bash[28843]: audit 2024-03-04T16:04:56.915376+0000 mon.a (mon.0) 3084 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:04:58.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:57 smithi117 bash[28843]: audit 2024-03-04T16:04:56.915775+0000 mon.a (mon.0) 3085 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-64061-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-03-04T16:04:58.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:57 smithi117 bash[28843]: audit 2024-03-04T16:04:57.064560+0000 mgr.y (mgr.24428) 589 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:04:58.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:57 smithi117 bash[28843]: cluster 2024-03-04T16:04:57.136257+0000 mgr.y (mgr.24428) 590 : cluster 0 pgmap v941: 164 pgs: 164 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-03-04T16:04:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:59 smithi151 bash[23756]: audit 2024-03-04T16:04:57.879801+0000 mon.a (mon.0) 3086 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-64061-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T16:04:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:59 smithi151 bash[23756]: cluster 2024-03-04T16:04:57.884557+0000 mon.a (mon.0) 3087 : cluster 0 osdmap e582: 8 total, 8 up, 8 in 2024-03-04T16:04:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:04:59 smithi151 bash[23756]: audit 2024-03-04T16:04:57.885469+0000 mon.a (mon.0) 3088 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64061-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:04:59.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:59 smithi117 bash[28843]: audit 2024-03-04T16:04:57.879801+0000 mon.a (mon.0) 3086 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-64061-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T16:04:59.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:59 smithi117 bash[28843]: cluster 2024-03-04T16:04:57.884557+0000 mon.a (mon.0) 3087 : cluster 0 osdmap e582: 8 total, 8 up, 8 in 2024-03-04T16:04:59.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:04:59 smithi117 bash[28843]: audit 2024-03-04T16:04:57.885469+0000 mon.a (mon.0) 3088 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64061-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:04:59.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:59 smithi117 bash[21480]: audit 2024-03-04T16:04:57.879801+0000 mon.a (mon.0) 3086 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi117-64061-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-03-04T16:04:59.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:59 smithi117 bash[21480]: cluster 2024-03-04T16:04:57.884557+0000 mon.a (mon.0) 3087 : cluster 0 osdmap e582: 8 total, 8 up, 8 in 2024-03-04T16:04:59.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:04:59 smithi117 bash[21480]: audit 2024-03-04T16:04:57.885469+0000 mon.a (mon.0) 3088 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64061-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:05:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:00 smithi151 bash[23756]: cluster 2024-03-04T16:04:59.136939+0000 mgr.y (mgr.24428) 591 : cluster 0 pgmap v944: 164 pgs: 164 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:05:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:00 smithi151 bash[23756]: cluster 2024-03-04T16:04:59.332301+0000 mon.a (mon.0) 3089 : cluster 0 osdmap e583: 8 total, 8 up, 8 in 2024-03-04T16:05:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:00 smithi151 bash[23756]: cluster 2024-03-04T16:04:59.332936+0000 mon.a (mon.0) 3090 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:05:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:00 smithi151 bash[23756]: audit 2024-03-04T16:04:59.788930+0000 mon.a (mon.0) 3091 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:05:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:00 smithi151 bash[23756]: audit 2024-03-04T16:04:59.791956+0000 mon.c (mon.2) 797 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:05:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:00 smithi151 bash[23756]: audit 2024-03-04T16:04:59.895504+0000 mon.a (mon.0) 3092 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64061-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:05:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:00 smithi151 bash[23756]: cluster 2024-03-04T16:04:59.903224+0000 mon.a (mon.0) 3093 : cluster 0 osdmap e584: 8 total, 8 up, 8 in 2024-03-04T16:05:00.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:00 smithi117 bash[28843]: cluster 2024-03-04T16:04:59.136939+0000 mgr.y (mgr.24428) 591 : cluster 0 pgmap v944: 164 pgs: 164 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:05:00.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:00 smithi117 bash[28843]: cluster 2024-03-04T16:04:59.332301+0000 mon.a (mon.0) 3089 : cluster 0 osdmap e583: 8 total, 8 up, 8 in 2024-03-04T16:05:00.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:00 smithi117 bash[28843]: cluster 2024-03-04T16:04:59.332936+0000 mon.a (mon.0) 3090 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:05:00.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:00 smithi117 bash[28843]: audit 2024-03-04T16:04:59.788930+0000 mon.a (mon.0) 3091 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:05:00.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:00 smithi117 bash[28843]: audit 2024-03-04T16:04:59.791956+0000 mon.c (mon.2) 797 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:05:00.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:00 smithi117 bash[28843]: audit 2024-03-04T16:04:59.895504+0000 mon.a (mon.0) 3092 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64061-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:05:00.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:00 smithi117 bash[28843]: cluster 2024-03-04T16:04:59.903224+0000 mon.a (mon.0) 3093 : cluster 0 osdmap e584: 8 total, 8 up, 8 in 2024-03-04T16:05:00.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:00 smithi117 bash[21480]: cluster 2024-03-04T16:04:59.136939+0000 mgr.y (mgr.24428) 591 : cluster 0 pgmap v944: 164 pgs: 164 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:05:00.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:00 smithi117 bash[21480]: cluster 2024-03-04T16:04:59.332301+0000 mon.a (mon.0) 3089 : cluster 0 osdmap e583: 8 total, 8 up, 8 in 2024-03-04T16:05:00.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:00 smithi117 bash[21480]: cluster 2024-03-04T16:04:59.332936+0000 mon.a (mon.0) 3090 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:05:00.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:00 smithi117 bash[21480]: audit 2024-03-04T16:04:59.788930+0000 mon.a (mon.0) 3091 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:05:00.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:00 smithi117 bash[21480]: audit 2024-03-04T16:04:59.791956+0000 mon.c (mon.2) 797 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:05:00.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:00 smithi117 bash[21480]: audit 2024-03-04T16:04:59.895504+0000 mon.a (mon.0) 3092 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi117-64061-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:05:00.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:00 smithi117 bash[21480]: cluster 2024-03-04T16:04:59.903224+0000 mon.a (mon.0) 3093 : cluster 0 osdmap e584: 8 total, 8 up, 8 in 2024-03-04T16:05:02.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:01 smithi151 bash[23756]: cluster 2024-03-04T16:05:00.908821+0000 mon.a (mon.0) 3094 : cluster 0 osdmap e585: 8 total, 8 up, 8 in 2024-03-04T16:05:02.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:01 smithi151 bash[23756]: audit 2024-03-04T16:05:00.918787+0000 mon.a (mon.0) 3095 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:05:02.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:01 smithi151 bash[23756]: cluster 2024-03-04T16:05:01.137769+0000 mgr.y (mgr.24428) 592 : cluster 0 pgmap v947: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:05:02.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:01 smithi117 bash[21480]: cluster 2024-03-04T16:05:00.908821+0000 mon.a (mon.0) 3094 : cluster 0 osdmap e585: 8 total, 8 up, 8 in 2024-03-04T16:05:02.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:01 smithi117 bash[21480]: audit 2024-03-04T16:05:00.918787+0000 mon.a (mon.0) 3095 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:05:02.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:01 smithi117 bash[21480]: cluster 2024-03-04T16:05:01.137769+0000 mgr.y (mgr.24428) 592 : cluster 0 pgmap v947: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:05:02.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:01 smithi117 bash[28843]: cluster 2024-03-04T16:05:00.908821+0000 mon.a (mon.0) 3094 : cluster 0 osdmap e585: 8 total, 8 up, 8 in 2024-03-04T16:05:02.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:01 smithi117 bash[28843]: audit 2024-03-04T16:05:00.918787+0000 mon.a (mon.0) 3095 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:05:02.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:01 smithi117 bash[28843]: cluster 2024-03-04T16:05:01.137769+0000 mgr.y (mgr.24428) 592 : cluster 0 pgmap v947: 204 pgs: 40 unknown, 164 active+clean; 582 KiB data, 651 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:05:03.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:02 smithi151 bash[23756]: audit 2024-03-04T16:05:01.908580+0000 mon.a (mon.0) 3096 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:05:03.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:02 smithi151 bash[23756]: cluster 2024-03-04T16:05:01.922023+0000 mon.a (mon.0) 3097 : cluster 0 osdmap e586: 8 total, 8 up, 8 in 2024-03-04T16:05:03.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:02 smithi151 bash[23756]: audit 2024-03-04T16:05:01.976602+0000 mon.a (mon.0) 3098 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:05:03.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:02 smithi117 bash[21480]: audit 2024-03-04T16:05:01.908580+0000 mon.a (mon.0) 3096 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:05:03.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:02 smithi117 bash[21480]: cluster 2024-03-04T16:05:01.922023+0000 mon.a (mon.0) 3097 : cluster 0 osdmap e586: 8 total, 8 up, 8 in 2024-03-04T16:05:03.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:02 smithi117 bash[21480]: audit 2024-03-04T16:05:01.976602+0000 mon.a (mon.0) 3098 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:05:03.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:02 smithi117 bash[28843]: audit 2024-03-04T16:05:01.908580+0000 mon.a (mon.0) 3096 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:05:03.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:02 smithi117 bash[28843]: cluster 2024-03-04T16:05:01.922023+0000 mon.a (mon.0) 3097 : cluster 0 osdmap e586: 8 total, 8 up, 8 in 2024-03-04T16:05:03.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:02 smithi117 bash[28843]: audit 2024-03-04T16:05:01.976602+0000 mon.a (mon.0) 3098 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:05:04.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:03 smithi151 bash[23756]: audit 2024-03-04T16:05:02.911356+0000 mon.a (mon.0) 3099 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-112", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:05:04.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:03 smithi151 bash[23756]: cluster 2024-03-04T16:05:02.919791+0000 mon.a (mon.0) 3100 : cluster 0 osdmap e587: 8 total, 8 up, 8 in 2024-03-04T16:05:04.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:03 smithi151 bash[23756]: audit 2024-03-04T16:05:02.921659+0000 mon.a (mon.0) 3101 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-112"}]: dispatch 2024-03-04T16:05:04.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:03 smithi151 bash[23756]: cluster 2024-03-04T16:05:03.138535+0000 mgr.y (mgr.24428) 593 : cluster 0 pgmap v950: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:05:04.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:03 smithi151 bash[23756]: audit 2024-03-04T16:05:03.915505+0000 mon.a (mon.0) 3102 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-112"}]': finished 2024-03-04T16:05:04.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:03 smithi117 bash[21480]: audit 2024-03-04T16:05:02.911356+0000 mon.a (mon.0) 3099 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-112", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:05:04.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:03 smithi117 bash[21480]: cluster 2024-03-04T16:05:02.919791+0000 mon.a (mon.0) 3100 : cluster 0 osdmap e587: 8 total, 8 up, 8 in 2024-03-04T16:05:04.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:03 smithi117 bash[21480]: audit 2024-03-04T16:05:02.921659+0000 mon.a (mon.0) 3101 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-112"}]: dispatch 2024-03-04T16:05:04.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:03 smithi117 bash[21480]: cluster 2024-03-04T16:05:03.138535+0000 mgr.y (mgr.24428) 593 : cluster 0 pgmap v950: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:05:04.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:03 smithi117 bash[21480]: audit 2024-03-04T16:05:03.915505+0000 mon.a (mon.0) 3102 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-112"}]': finished 2024-03-04T16:05:04.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:03 smithi117 bash[28843]: audit 2024-03-04T16:05:02.911356+0000 mon.a (mon.0) 3099 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-112", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:05:04.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:03 smithi117 bash[28843]: cluster 2024-03-04T16:05:02.919791+0000 mon.a (mon.0) 3100 : cluster 0 osdmap e587: 8 total, 8 up, 8 in 2024-03-04T16:05:04.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:03 smithi117 bash[28843]: audit 2024-03-04T16:05:02.921659+0000 mon.a (mon.0) 3101 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-112"}]: dispatch 2024-03-04T16:05:04.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:03 smithi117 bash[28843]: cluster 2024-03-04T16:05:03.138535+0000 mgr.y (mgr.24428) 593 : cluster 0 pgmap v950: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:05:04.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:03 smithi117 bash[28843]: audit 2024-03-04T16:05:03.915505+0000 mon.a (mon.0) 3102 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-112"}]': finished 2024-03-04T16:05:05.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:04 smithi151 bash[23756]: cluster 2024-03-04T16:05:03.924697+0000 mon.a (mon.0) 3103 : cluster 0 osdmap e588: 8 total, 8 up, 8 in 2024-03-04T16:05:05.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:04 smithi151 bash[23756]: audit 2024-03-04T16:05:04.007664+0000 mon.a (mon.0) 3104 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:05:05.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:04 smithi151 bash[23756]: cluster 2024-03-04T16:05:04.334701+0000 mon.a (mon.0) 3105 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:05:05.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:04 smithi117 bash[21480]: cluster 2024-03-04T16:05:03.924697+0000 mon.a (mon.0) 3103 : cluster 0 osdmap e588: 8 total, 8 up, 8 in 2024-03-04T16:05:05.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:04 smithi117 bash[21480]: audit 2024-03-04T16:05:04.007664+0000 mon.a (mon.0) 3104 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:05:05.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:04 smithi117 bash[21480]: cluster 2024-03-04T16:05:04.334701+0000 mon.a (mon.0) 3105 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:05:05.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:04 smithi117 bash[28843]: cluster 2024-03-04T16:05:03.924697+0000 mon.a (mon.0) 3103 : cluster 0 osdmap e588: 8 total, 8 up, 8 in 2024-03-04T16:05:05.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:04 smithi117 bash[28843]: audit 2024-03-04T16:05:04.007664+0000 mon.a (mon.0) 3104 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:05:05.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:04 smithi117 bash[28843]: cluster 2024-03-04T16:05:04.334701+0000 mon.a (mon.0) 3105 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:05:06.316 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:05:06 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:05:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:05:06.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:05 smithi117 bash[28843]: audit 2024-03-04T16:05:04.944062+0000 mon.a (mon.0) 3106 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:05:06.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:05 smithi117 bash[28843]: cluster 2024-03-04T16:05:04.951860+0000 mon.a (mon.0) 3107 : cluster 0 osdmap e589: 8 total, 8 up, 8 in 2024-03-04T16:05:06.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:05 smithi117 bash[28843]: audit 2024-03-04T16:05:04.954079+0000 mon.a (mon.0) 3108 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-112"}]: dispatch 2024-03-04T16:05:06.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:05 smithi117 bash[28843]: cluster 2024-03-04T16:05:05.139357+0000 mgr.y (mgr.24428) 594 : cluster 0 pgmap v953: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:05:06.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:05 smithi117 bash[21480]: audit 2024-03-04T16:05:04.944062+0000 mon.a (mon.0) 3106 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:05:06.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:05 smithi117 bash[21480]: cluster 2024-03-04T16:05:04.951860+0000 mon.a (mon.0) 3107 : cluster 0 osdmap e589: 8 total, 8 up, 8 in 2024-03-04T16:05:06.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:05 smithi117 bash[21480]: audit 2024-03-04T16:05:04.954079+0000 mon.a (mon.0) 3108 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-112"}]: dispatch 2024-03-04T16:05:06.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:05 smithi117 bash[21480]: cluster 2024-03-04T16:05:05.139357+0000 mgr.y (mgr.24428) 594 : cluster 0 pgmap v953: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:05:06.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:05 smithi151 bash[23756]: audit 2024-03-04T16:05:04.944062+0000 mon.a (mon.0) 3106 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:05:06.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:05 smithi151 bash[23756]: cluster 2024-03-04T16:05:04.951860+0000 mon.a (mon.0) 3107 : cluster 0 osdmap e589: 8 total, 8 up, 8 in 2024-03-04T16:05:06.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:05 smithi151 bash[23756]: audit 2024-03-04T16:05:04.954079+0000 mon.a (mon.0) 3108 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-112"}]: dispatch 2024-03-04T16:05:06.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:05 smithi151 bash[23756]: cluster 2024-03-04T16:05:05.139357+0000 mgr.y (mgr.24428) 594 : cluster 0 pgmap v953: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:05:07.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:06 smithi117 bash[21480]: audit 2024-03-04T16:05:05.953039+0000 mon.a (mon.0) 3109 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-112"}]': finished 2024-03-04T16:05:07.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:06 smithi117 bash[21480]: cluster 2024-03-04T16:05:05.962625+0000 mon.a (mon.0) 3110 : cluster 0 osdmap e590: 8 total, 8 up, 8 in 2024-03-04T16:05:07.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:06 smithi117 bash[28843]: audit 2024-03-04T16:05:05.953039+0000 mon.a (mon.0) 3109 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-112"}]': finished 2024-03-04T16:05:07.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:06 smithi117 bash[28843]: cluster 2024-03-04T16:05:05.962625+0000 mon.a (mon.0) 3110 : cluster 0 osdmap e590: 8 total, 8 up, 8 in 2024-03-04T16:05:07.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:05:07 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:05:07.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:06 smithi151 bash[23756]: audit 2024-03-04T16:05:05.953039+0000 mon.a (mon.0) 3109 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-112"}]': finished 2024-03-04T16:05:07.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:06 smithi151 bash[23756]: cluster 2024-03-04T16:05:05.962625+0000 mon.a (mon.0) 3110 : cluster 0 osdmap e590: 8 total, 8 up, 8 in 2024-03-04T16:05:08.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:07 smithi117 bash[21480]: cluster 2024-03-04T16:05:06.969326+0000 mon.a (mon.0) 3111 : cluster 0 osdmap e591: 8 total, 8 up, 8 in 2024-03-04T16:05:08.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:07 smithi117 bash[21480]: audit 2024-03-04T16:05:07.071409+0000 mgr.y (mgr.24428) 595 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:05:08.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:07 smithi117 bash[21480]: cluster 2024-03-04T16:05:07.140079+0000 mgr.y (mgr.24428) 596 : cluster 0 pgmap v956: 172 pgs: 172 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:08.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:07 smithi117 bash[28843]: cluster 2024-03-04T16:05:06.969326+0000 mon.a (mon.0) 3111 : cluster 0 osdmap e591: 8 total, 8 up, 8 in 2024-03-04T16:05:08.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:07 smithi117 bash[28843]: audit 2024-03-04T16:05:07.071409+0000 mgr.y (mgr.24428) 595 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:05:08.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:07 smithi117 bash[28843]: cluster 2024-03-04T16:05:07.140079+0000 mgr.y (mgr.24428) 596 : cluster 0 pgmap v956: 172 pgs: 172 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:08.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:07 smithi151 bash[23756]: cluster 2024-03-04T16:05:06.969326+0000 mon.a (mon.0) 3111 : cluster 0 osdmap e591: 8 total, 8 up, 8 in 2024-03-04T16:05:08.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:07 smithi151 bash[23756]: audit 2024-03-04T16:05:07.071409+0000 mgr.y (mgr.24428) 595 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:05:08.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:07 smithi151 bash[23756]: cluster 2024-03-04T16:05:07.140079+0000 mgr.y (mgr.24428) 596 : cluster 0 pgmap v956: 172 pgs: 172 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:09.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:09 smithi117 bash[28843]: cluster 2024-03-04T16:05:07.993589+0000 mon.a (mon.0) 3112 : cluster 0 osdmap e592: 8 total, 8 up, 8 in 2024-03-04T16:05:09.316 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:09 smithi117 bash[28843]: audit 2024-03-04T16:05:08.008188+0000 mon.a (mon.0) 3113 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:05:09.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:09 smithi117 bash[21480]: cluster 2024-03-04T16:05:07.993589+0000 mon.a (mon.0) 3112 : cluster 0 osdmap e592: 8 total, 8 up, 8 in 2024-03-04T16:05:09.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:09 smithi117 bash[21480]: audit 2024-03-04T16:05:08.008188+0000 mon.a (mon.0) 3113 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:05:09.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:09 smithi151 bash[23756]: cluster 2024-03-04T16:05:07.993589+0000 mon.a (mon.0) 3112 : cluster 0 osdmap e592: 8 total, 8 up, 8 in 2024-03-04T16:05:09.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:09 smithi151 bash[23756]: audit 2024-03-04T16:05:08.008188+0000 mon.a (mon.0) 3113 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:05:10.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:10 smithi117 bash[21480]: audit 2024-03-04T16:05:08.982151+0000 mon.a (mon.0) 3114 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:05:10.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:10 smithi117 bash[21480]: cluster 2024-03-04T16:05:08.995983+0000 mon.a (mon.0) 3115 : cluster 0 osdmap e593: 8 total, 8 up, 8 in 2024-03-04T16:05:10.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:10 smithi117 bash[21480]: audit 2024-03-04T16:05:09.044594+0000 mon.a (mon.0) 3116 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:05:10.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:10 smithi117 bash[21480]: cluster 2024-03-04T16:05:09.140852+0000 mgr.y (mgr.24428) 597 : cluster 0 pgmap v959: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:10.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:10 smithi117 bash[28843]: audit 2024-03-04T16:05:08.982151+0000 mon.a (mon.0) 3114 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:05:10.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:10 smithi117 bash[28843]: cluster 2024-03-04T16:05:08.995983+0000 mon.a (mon.0) 3115 : cluster 0 osdmap e593: 8 total, 8 up, 8 in 2024-03-04T16:05:10.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:10 smithi117 bash[28843]: audit 2024-03-04T16:05:09.044594+0000 mon.a (mon.0) 3116 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:05:10.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:10 smithi117 bash[28843]: cluster 2024-03-04T16:05:09.140852+0000 mgr.y (mgr.24428) 597 : cluster 0 pgmap v959: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:10.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:10 smithi151 bash[23756]: audit 2024-03-04T16:05:08.982151+0000 mon.a (mon.0) 3114 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:05:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:10 smithi151 bash[23756]: cluster 2024-03-04T16:05:08.995983+0000 mon.a (mon.0) 3115 : cluster 0 osdmap e593: 8 total, 8 up, 8 in 2024-03-04T16:05:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:10 smithi151 bash[23756]: audit 2024-03-04T16:05:09.044594+0000 mon.a (mon.0) 3116 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:05:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:10 smithi151 bash[23756]: cluster 2024-03-04T16:05:09.140852+0000 mgr.y (mgr.24428) 597 : cluster 0 pgmap v959: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:11.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:11 smithi117 bash[21480]: audit 2024-03-04T16:05:10.016083+0000 mon.a (mon.0) 3117 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-114", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:05:11.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:11 smithi117 bash[21480]: cluster 2024-03-04T16:05:10.023064+0000 mon.a (mon.0) 3118 : cluster 0 osdmap e594: 8 total, 8 up, 8 in 2024-03-04T16:05:11.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:11 smithi117 bash[21480]: audit 2024-03-04T16:05:10.024092+0000 mon.a (mon.0) 3119 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-114"}]: dispatch 2024-03-04T16:05:11.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:11 smithi117 bash[28843]: audit 2024-03-04T16:05:10.016083+0000 mon.a (mon.0) 3117 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-114", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:05:11.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:11 smithi117 bash[28843]: cluster 2024-03-04T16:05:10.023064+0000 mon.a (mon.0) 3118 : cluster 0 osdmap e594: 8 total, 8 up, 8 in 2024-03-04T16:05:11.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:11 smithi117 bash[28843]: audit 2024-03-04T16:05:10.024092+0000 mon.a (mon.0) 3119 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-114"}]: dispatch 2024-03-04T16:05:11.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:11 smithi151 bash[23756]: audit 2024-03-04T16:05:10.016083+0000 mon.a (mon.0) 3117 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-114", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:05:11.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:11 smithi151 bash[23756]: cluster 2024-03-04T16:05:10.023064+0000 mon.a (mon.0) 3118 : cluster 0 osdmap e594: 8 total, 8 up, 8 in 2024-03-04T16:05:11.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:11 smithi151 bash[23756]: audit 2024-03-04T16:05:10.024092+0000 mon.a (mon.0) 3119 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-114"}]: dispatch 2024-03-04T16:05:12.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:12 smithi117 bash[21480]: audit 2024-03-04T16:05:11.019050+0000 mon.a (mon.0) 3120 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-114"}]': finished 2024-03-04T16:05:12.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:12 smithi117 bash[21480]: cluster 2024-03-04T16:05:11.027269+0000 mon.a (mon.0) 3121 : cluster 0 osdmap e595: 8 total, 8 up, 8 in 2024-03-04T16:05:12.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:12 smithi117 bash[21480]: audit 2024-03-04T16:05:11.031772+0000 mon.a (mon.0) 3122 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-114", "mode": "writeback"}]: dispatch 2024-03-04T16:05:12.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:12 smithi117 bash[21480]: cluster 2024-03-04T16:05:11.141720+0000 mgr.y (mgr.24428) 598 : cluster 0 pgmap v962: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:05:12.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:12 smithi117 bash[21480]: cluster 2024-03-04T16:05:12.019438+0000 mon.a (mon.0) 3123 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:05:12.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:12 smithi117 bash[21480]: audit 2024-03-04T16:05:12.021930+0000 mon.a (mon.0) 3124 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-114", "mode": "writeback"}]': finished 2024-03-04T16:05:12.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:12 smithi117 bash[28843]: audit 2024-03-04T16:05:11.019050+0000 mon.a (mon.0) 3120 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-114"}]': finished 2024-03-04T16:05:12.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:12 smithi117 bash[28843]: cluster 2024-03-04T16:05:11.027269+0000 mon.a (mon.0) 3121 : cluster 0 osdmap e595: 8 total, 8 up, 8 in 2024-03-04T16:05:12.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:12 smithi117 bash[28843]: audit 2024-03-04T16:05:11.031772+0000 mon.a (mon.0) 3122 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-114", "mode": "writeback"}]: dispatch 2024-03-04T16:05:12.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:12 smithi117 bash[28843]: cluster 2024-03-04T16:05:11.141720+0000 mgr.y (mgr.24428) 598 : cluster 0 pgmap v962: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:05:12.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:12 smithi117 bash[28843]: cluster 2024-03-04T16:05:12.019438+0000 mon.a (mon.0) 3123 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:05:12.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:12 smithi117 bash[28843]: audit 2024-03-04T16:05:12.021930+0000 mon.a (mon.0) 3124 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-114", "mode": "writeback"}]': finished 2024-03-04T16:05:12.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:12 smithi151 bash[23756]: audit 2024-03-04T16:05:11.019050+0000 mon.a (mon.0) 3120 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-114"}]': finished 2024-03-04T16:05:12.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:12 smithi151 bash[23756]: cluster 2024-03-04T16:05:11.027269+0000 mon.a (mon.0) 3121 : cluster 0 osdmap e595: 8 total, 8 up, 8 in 2024-03-04T16:05:12.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:12 smithi151 bash[23756]: audit 2024-03-04T16:05:11.031772+0000 mon.a (mon.0) 3122 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-114", "mode": "writeback"}]: dispatch 2024-03-04T16:05:12.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:12 smithi151 bash[23756]: cluster 2024-03-04T16:05:11.141720+0000 mgr.y (mgr.24428) 598 : cluster 0 pgmap v962: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 660 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:05:12.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:12 smithi151 bash[23756]: cluster 2024-03-04T16:05:12.019438+0000 mon.a (mon.0) 3123 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:05:12.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:12 smithi151 bash[23756]: audit 2024-03-04T16:05:12.021930+0000 mon.a (mon.0) 3124 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-114", "mode": "writeback"}]': finished 2024-03-04T16:05:13.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:13 smithi151 bash[23756]: cluster 2024-03-04T16:05:12.034045+0000 mon.a (mon.0) 3125 : cluster 0 osdmap e596: 8 total, 8 up, 8 in 2024-03-04T16:05:13.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:13 smithi151 bash[23756]: audit 2024-03-04T16:05:12.135221+0000 mon.a (mon.0) 3126 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:05:13.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:13 smithi117 bash[21480]: cluster 2024-03-04T16:05:12.034045+0000 mon.a (mon.0) 3125 : cluster 0 osdmap e596: 8 total, 8 up, 8 in 2024-03-04T16:05:13.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:13 smithi117 bash[21480]: audit 2024-03-04T16:05:12.135221+0000 mon.a (mon.0) 3126 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:05:13.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:13 smithi117 bash[28843]: cluster 2024-03-04T16:05:12.034045+0000 mon.a (mon.0) 3125 : cluster 0 osdmap e596: 8 total, 8 up, 8 in 2024-03-04T16:05:13.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:13 smithi117 bash[28843]: audit 2024-03-04T16:05:12.135221+0000 mon.a (mon.0) 3126 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:05:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:14 smithi151 bash[23756]: audit 2024-03-04T16:05:13.056363+0000 mon.a (mon.0) 3127 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:05:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:14 smithi151 bash[23756]: cluster 2024-03-04T16:05:13.064797+0000 mon.a (mon.0) 3128 : cluster 0 osdmap e597: 8 total, 8 up, 8 in 2024-03-04T16:05:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:14 smithi151 bash[23756]: audit 2024-03-04T16:05:13.068310+0000 mon.a (mon.0) 3129 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-114"}]: dispatch 2024-03-04T16:05:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:14 smithi151 bash[23756]: cluster 2024-03-04T16:05:13.142488+0000 mgr.y (mgr.24428) 599 : cluster 0 pgmap v965: 204 pgs: 204 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:14 smithi151 bash[23756]: cluster 2024-03-04T16:05:14.056627+0000 mon.a (mon.0) 3130 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:05:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:14 smithi151 bash[23756]: audit 2024-03-04T16:05:14.058886+0000 mon.a (mon.0) 3131 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-114"}]': finished 2024-03-04T16:05:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:14 smithi151 bash[23756]: cluster 2024-03-04T16:05:14.066589+0000 mon.a (mon.0) 3132 : cluster 0 osdmap e598: 8 total, 8 up, 8 in 2024-03-04T16:05:14.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:14 smithi117 bash[21480]: audit 2024-03-04T16:05:13.056363+0000 mon.a (mon.0) 3127 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:05:14.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:14 smithi117 bash[21480]: cluster 2024-03-04T16:05:13.064797+0000 mon.a (mon.0) 3128 : cluster 0 osdmap e597: 8 total, 8 up, 8 in 2024-03-04T16:05:14.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:14 smithi117 bash[21480]: audit 2024-03-04T16:05:13.068310+0000 mon.a (mon.0) 3129 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-114"}]: dispatch 2024-03-04T16:05:14.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:14 smithi117 bash[21480]: cluster 2024-03-04T16:05:13.142488+0000 mgr.y (mgr.24428) 599 : cluster 0 pgmap v965: 204 pgs: 204 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:14.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:14 smithi117 bash[21480]: cluster 2024-03-04T16:05:14.056627+0000 mon.a (mon.0) 3130 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:05:14.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:14 smithi117 bash[21480]: audit 2024-03-04T16:05:14.058886+0000 mon.a (mon.0) 3131 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-114"}]': finished 2024-03-04T16:05:14.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:14 smithi117 bash[21480]: cluster 2024-03-04T16:05:14.066589+0000 mon.a (mon.0) 3132 : cluster 0 osdmap e598: 8 total, 8 up, 8 in 2024-03-04T16:05:14.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:14 smithi117 bash[28843]: audit 2024-03-04T16:05:13.056363+0000 mon.a (mon.0) 3127 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:05:14.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:14 smithi117 bash[28843]: cluster 2024-03-04T16:05:13.064797+0000 mon.a (mon.0) 3128 : cluster 0 osdmap e597: 8 total, 8 up, 8 in 2024-03-04T16:05:14.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:14 smithi117 bash[28843]: audit 2024-03-04T16:05:13.068310+0000 mon.a (mon.0) 3129 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-114"}]: dispatch 2024-03-04T16:05:14.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:14 smithi117 bash[28843]: cluster 2024-03-04T16:05:13.142488+0000 mgr.y (mgr.24428) 599 : cluster 0 pgmap v965: 204 pgs: 204 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:14.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:14 smithi117 bash[28843]: cluster 2024-03-04T16:05:14.056627+0000 mon.a (mon.0) 3130 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:05:14.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:14 smithi117 bash[28843]: audit 2024-03-04T16:05:14.058886+0000 mon.a (mon.0) 3131 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-114"}]': finished 2024-03-04T16:05:14.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:14 smithi117 bash[28843]: cluster 2024-03-04T16:05:14.066589+0000 mon.a (mon.0) 3132 : cluster 0 osdmap e598: 8 total, 8 up, 8 in 2024-03-04T16:05:15.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:15 smithi117 bash[21480]: audit 2024-03-04T16:05:14.824811+0000 mon.c (mon.2) 798 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:05:15.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:15 smithi117 bash[28843]: audit 2024-03-04T16:05:14.824811+0000 mon.c (mon.2) 798 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:05:15.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:15 smithi151 bash[23756]: audit 2024-03-04T16:05:14.824811+0000 mon.c (mon.2) 798 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:05:16.549 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:05:16 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:05:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:05:16.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:16 smithi117 bash[28843]: cluster 2024-03-04T16:05:15.095979+0000 mon.a (mon.0) 3133 : cluster 0 osdmap e599: 8 total, 8 up, 8 in 2024-03-04T16:05:16.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:16 smithi117 bash[28843]: cluster 2024-03-04T16:05:15.143289+0000 mgr.y (mgr.24428) 600 : cluster 0 pgmap v968: 172 pgs: 172 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:16.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:16 smithi117 bash[28843]: cluster 2024-03-04T16:05:16.105122+0000 mon.a (mon.0) 3134 : cluster 0 osdmap e600: 8 total, 8 up, 8 in 2024-03-04T16:05:16.816 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:16 smithi117 bash[28843]: audit 2024-03-04T16:05:16.110136+0000 mon.a (mon.0) 3135 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:05:16.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:16 smithi117 bash[21480]: cluster 2024-03-04T16:05:15.095979+0000 mon.a (mon.0) 3133 : cluster 0 osdmap e599: 8 total, 8 up, 8 in 2024-03-04T16:05:16.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:16 smithi117 bash[21480]: cluster 2024-03-04T16:05:15.143289+0000 mgr.y (mgr.24428) 600 : cluster 0 pgmap v968: 172 pgs: 172 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:16.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:16 smithi117 bash[21480]: cluster 2024-03-04T16:05:16.105122+0000 mon.a (mon.0) 3134 : cluster 0 osdmap e600: 8 total, 8 up, 8 in 2024-03-04T16:05:16.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:16 smithi117 bash[21480]: audit 2024-03-04T16:05:16.110136+0000 mon.a (mon.0) 3135 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:05:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:16 smithi151 bash[23756]: cluster 2024-03-04T16:05:15.095979+0000 mon.a (mon.0) 3133 : cluster 0 osdmap e599: 8 total, 8 up, 8 in 2024-03-04T16:05:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:16 smithi151 bash[23756]: cluster 2024-03-04T16:05:15.143289+0000 mgr.y (mgr.24428) 600 : cluster 0 pgmap v968: 172 pgs: 172 active+clean; 582 KiB data, 665 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:16 smithi151 bash[23756]: cluster 2024-03-04T16:05:16.105122+0000 mon.a (mon.0) 3134 : cluster 0 osdmap e600: 8 total, 8 up, 8 in 2024-03-04T16:05:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:16 smithi151 bash[23756]: audit 2024-03-04T16:05:16.110136+0000 mon.a (mon.0) 3135 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:05:17.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:05:17 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:05:18.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:18 smithi151 bash[23756]: audit 2024-03-04T16:05:17.082937+0000 mgr.y (mgr.24428) 601 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:05:18.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:18 smithi151 bash[23756]: audit 2024-03-04T16:05:17.094849+0000 mon.a (mon.0) 3136 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:05:18.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:18 smithi151 bash[23756]: cluster 2024-03-04T16:05:17.104378+0000 mon.a (mon.0) 3137 : cluster 0 osdmap e601: 8 total, 8 up, 8 in 2024-03-04T16:05:18.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:18 smithi151 bash[23756]: cluster 2024-03-04T16:05:17.144300+0000 mgr.y (mgr.24428) 602 : cluster 0 pgmap v971: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:05:18.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:18 smithi117 bash[28843]: audit 2024-03-04T16:05:17.082937+0000 mgr.y (mgr.24428) 601 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:05:18.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:18 smithi117 bash[28843]: audit 2024-03-04T16:05:17.094849+0000 mon.a (mon.0) 3136 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:05:18.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:18 smithi117 bash[28843]: cluster 2024-03-04T16:05:17.104378+0000 mon.a (mon.0) 3137 : cluster 0 osdmap e601: 8 total, 8 up, 8 in 2024-03-04T16:05:18.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:18 smithi117 bash[28843]: cluster 2024-03-04T16:05:17.144300+0000 mgr.y (mgr.24428) 602 : cluster 0 pgmap v971: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:05:18.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:18 smithi117 bash[21480]: audit 2024-03-04T16:05:17.082937+0000 mgr.y (mgr.24428) 601 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:05:18.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:18 smithi117 bash[21480]: audit 2024-03-04T16:05:17.094849+0000 mon.a (mon.0) 3136 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:05:18.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:18 smithi117 bash[21480]: cluster 2024-03-04T16:05:17.104378+0000 mon.a (mon.0) 3137 : cluster 0 osdmap e601: 8 total, 8 up, 8 in 2024-03-04T16:05:18.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:18 smithi117 bash[21480]: cluster 2024-03-04T16:05:17.144300+0000 mgr.y (mgr.24428) 602 : cluster 0 pgmap v971: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:05:19.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:19 smithi151 bash[23756]: cluster 2024-03-04T16:05:18.112841+0000 mon.a (mon.0) 3138 : cluster 0 osdmap e602: 8 total, 8 up, 8 in 2024-03-04T16:05:19.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:19 smithi151 bash[23756]: audit 2024-03-04T16:05:18.148775+0000 mon.a (mon.0) 3139 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:05:19.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:19 smithi117 bash[28843]: cluster 2024-03-04T16:05:18.112841+0000 mon.a (mon.0) 3138 : cluster 0 osdmap e602: 8 total, 8 up, 8 in 2024-03-04T16:05:19.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:19 smithi117 bash[28843]: audit 2024-03-04T16:05:18.148775+0000 mon.a (mon.0) 3139 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:05:19.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:19 smithi117 bash[21480]: cluster 2024-03-04T16:05:18.112841+0000 mon.a (mon.0) 3138 : cluster 0 osdmap e602: 8 total, 8 up, 8 in 2024-03-04T16:05:19.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:19 smithi117 bash[21480]: audit 2024-03-04T16:05:18.148775+0000 mon.a (mon.0) 3139 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:05:20.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:20 smithi151 bash[23756]: audit 2024-03-04T16:05:19.116980+0000 mon.a (mon.0) 3140 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-116", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:05:20.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:20 smithi151 bash[23756]: cluster 2024-03-04T16:05:19.124352+0000 mon.a (mon.0) 3141 : cluster 0 osdmap e603: 8 total, 8 up, 8 in 2024-03-04T16:05:20.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:20 smithi151 bash[23756]: audit 2024-03-04T16:05:19.125118+0000 mon.a (mon.0) 3142 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-116"}]: dispatch 2024-03-04T16:05:20.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:20 smithi151 bash[23756]: cluster 2024-03-04T16:05:19.145193+0000 mgr.y (mgr.24428) 603 : cluster 0 pgmap v974: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:05:20.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:20 smithi117 bash[21480]: audit 2024-03-04T16:05:19.116980+0000 mon.a (mon.0) 3140 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-116", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:05:20.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:20 smithi117 bash[21480]: cluster 2024-03-04T16:05:19.124352+0000 mon.a (mon.0) 3141 : cluster 0 osdmap e603: 8 total, 8 up, 8 in 2024-03-04T16:05:20.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:20 smithi117 bash[21480]: audit 2024-03-04T16:05:19.125118+0000 mon.a (mon.0) 3142 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-116"}]: dispatch 2024-03-04T16:05:20.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:20 smithi117 bash[21480]: cluster 2024-03-04T16:05:19.145193+0000 mgr.y (mgr.24428) 603 : cluster 0 pgmap v974: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:05:20.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:20 smithi117 bash[28843]: audit 2024-03-04T16:05:19.116980+0000 mon.a (mon.0) 3140 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-116", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:05:20.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:20 smithi117 bash[28843]: cluster 2024-03-04T16:05:19.124352+0000 mon.a (mon.0) 3141 : cluster 0 osdmap e603: 8 total, 8 up, 8 in 2024-03-04T16:05:20.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:20 smithi117 bash[28843]: audit 2024-03-04T16:05:19.125118+0000 mon.a (mon.0) 3142 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-116"}]: dispatch 2024-03-04T16:05:20.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:20 smithi117 bash[28843]: cluster 2024-03-04T16:05:19.145193+0000 mgr.y (mgr.24428) 603 : cluster 0 pgmap v974: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 653 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:05:21.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:21 smithi151 bash[23756]: audit 2024-03-04T16:05:20.126382+0000 mon.a (mon.0) 3143 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-116"}]': finished 2024-03-04T16:05:21.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:21 smithi151 bash[23756]: cluster 2024-03-04T16:05:20.136458+0000 mon.a (mon.0) 3144 : cluster 0 osdmap e604: 8 total, 8 up, 8 in 2024-03-04T16:05:21.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:21 smithi151 bash[23756]: audit 2024-03-04T16:05:20.137686+0000 mon.a (mon.0) 3145 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-116", "mode": "writeback"}]: dispatch 2024-03-04T16:05:21.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:21 smithi117 bash[21480]: audit 2024-03-04T16:05:20.126382+0000 mon.a (mon.0) 3143 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-116"}]': finished 2024-03-04T16:05:21.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:21 smithi117 bash[21480]: cluster 2024-03-04T16:05:20.136458+0000 mon.a (mon.0) 3144 : cluster 0 osdmap e604: 8 total, 8 up, 8 in 2024-03-04T16:05:21.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:21 smithi117 bash[21480]: audit 2024-03-04T16:05:20.137686+0000 mon.a (mon.0) 3145 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-116", "mode": "writeback"}]: dispatch 2024-03-04T16:05:21.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:21 smithi117 bash[28843]: audit 2024-03-04T16:05:20.126382+0000 mon.a (mon.0) 3143 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-116"}]': finished 2024-03-04T16:05:21.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:21 smithi117 bash[28843]: cluster 2024-03-04T16:05:20.136458+0000 mon.a (mon.0) 3144 : cluster 0 osdmap e604: 8 total, 8 up, 8 in 2024-03-04T16:05:21.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:21 smithi117 bash[28843]: audit 2024-03-04T16:05:20.137686+0000 mon.a (mon.0) 3145 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-116", "mode": "writeback"}]: dispatch 2024-03-04T16:05:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:22 smithi151 bash[23756]: cluster 2024-03-04T16:05:21.127590+0000 mon.a (mon.0) 3146 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:05:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:22 smithi151 bash[23756]: audit 2024-03-04T16:05:21.137521+0000 mon.a (mon.0) 3147 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-116", "mode": "writeback"}]': finished 2024-03-04T16:05:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:22 smithi151 bash[23756]: cluster 2024-03-04T16:05:21.145295+0000 mon.a (mon.0) 3148 : cluster 0 osdmap e605: 8 total, 8 up, 8 in 2024-03-04T16:05:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:22 smithi151 bash[23756]: cluster 2024-03-04T16:05:21.146067+0000 mgr.y (mgr.24428) 604 : cluster 0 pgmap v977: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 653 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:05:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:22 smithi151 bash[23756]: audit 2024-03-04T16:05:21.177029+0000 mon.a (mon.0) 3149 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "326.2"}]: dispatch 2024-03-04T16:05:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:22 smithi151 bash[23756]: audit 2024-03-04T16:05:21.177478+0000 mgr.y (mgr.24428) 605 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "326.2"}]: dispatch 2024-03-04T16:05:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:22 smithi151 bash[23756]: cluster 2024-03-04T16:05:21.494505+0000 osd.7 (osd.7) 3 : cluster 0 326.2 scrub starts 2024-03-04T16:05:22.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:22 smithi151 bash[23756]: cluster 2024-03-04T16:05:21.497127+0000 osd.7 (osd.7) 4 : cluster 0 326.2 scrub ok 2024-03-04T16:05:22.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:22 smithi117 bash[21480]: cluster 2024-03-04T16:05:21.127590+0000 mon.a (mon.0) 3146 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:05:22.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:22 smithi117 bash[21480]: audit 2024-03-04T16:05:21.137521+0000 mon.a (mon.0) 3147 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-116", "mode": "writeback"}]': finished 2024-03-04T16:05:22.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:22 smithi117 bash[21480]: cluster 2024-03-04T16:05:21.145295+0000 mon.a (mon.0) 3148 : cluster 0 osdmap e605: 8 total, 8 up, 8 in 2024-03-04T16:05:22.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:22 smithi117 bash[21480]: cluster 2024-03-04T16:05:21.146067+0000 mgr.y (mgr.24428) 604 : cluster 0 pgmap v977: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 653 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:05:22.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:22 smithi117 bash[21480]: audit 2024-03-04T16:05:21.177029+0000 mon.a (mon.0) 3149 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "326.2"}]: dispatch 2024-03-04T16:05:22.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:22 smithi117 bash[21480]: audit 2024-03-04T16:05:21.177478+0000 mgr.y (mgr.24428) 605 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "326.2"}]: dispatch 2024-03-04T16:05:22.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:22 smithi117 bash[21480]: cluster 2024-03-04T16:05:21.494505+0000 osd.7 (osd.7) 3 : cluster 0 326.2 scrub starts 2024-03-04T16:05:22.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:22 smithi117 bash[21480]: cluster 2024-03-04T16:05:21.497127+0000 osd.7 (osd.7) 4 : cluster 0 326.2 scrub ok 2024-03-04T16:05:22.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:22 smithi117 bash[28843]: cluster 2024-03-04T16:05:21.127590+0000 mon.a (mon.0) 3146 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:05:22.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:22 smithi117 bash[28843]: audit 2024-03-04T16:05:21.137521+0000 mon.a (mon.0) 3147 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-116", "mode": "writeback"}]': finished 2024-03-04T16:05:22.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:22 smithi117 bash[28843]: cluster 2024-03-04T16:05:21.145295+0000 mon.a (mon.0) 3148 : cluster 0 osdmap e605: 8 total, 8 up, 8 in 2024-03-04T16:05:22.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:22 smithi117 bash[28843]: cluster 2024-03-04T16:05:21.146067+0000 mgr.y (mgr.24428) 604 : cluster 0 pgmap v977: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 653 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:05:22.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:22 smithi117 bash[28843]: audit 2024-03-04T16:05:21.177029+0000 mon.a (mon.0) 3149 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "326.2"}]: dispatch 2024-03-04T16:05:22.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:22 smithi117 bash[28843]: audit 2024-03-04T16:05:21.177478+0000 mgr.y (mgr.24428) 605 : audit 0 from='mon.0 -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "326.2"}]: dispatch 2024-03-04T16:05:22.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:22 smithi117 bash[28843]: cluster 2024-03-04T16:05:21.494505+0000 osd.7 (osd.7) 3 : cluster 0 326.2 scrub starts 2024-03-04T16:05:22.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:22 smithi117 bash[28843]: cluster 2024-03-04T16:05:21.497127+0000 osd.7 (osd.7) 4 : cluster 0 326.2 scrub ok 2024-03-04T16:05:24.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:24 smithi117 bash[21480]: cluster 2024-03-04T16:05:23.148216+0000 mgr.y (mgr.24428) 606 : cluster 0 pgmap v978: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-03-04T16:05:24.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:24 smithi117 bash[28843]: cluster 2024-03-04T16:05:23.148216+0000 mgr.y (mgr.24428) 606 : cluster 0 pgmap v978: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-03-04T16:05:24.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:24 smithi151 bash[23756]: cluster 2024-03-04T16:05:23.148216+0000 mgr.y (mgr.24428) 606 : cluster 0 pgmap v978: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.4 KiB/s wr, 3 op/s 2024-03-04T16:05:26.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:26 smithi117 bash[21480]: cluster 2024-03-04T16:05:25.149123+0000 mgr.y (mgr.24428) 607 : cluster 0 pgmap v979: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T16:05:26.566 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:05:26 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:05:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:05:26.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:26 smithi117 bash[28843]: cluster 2024-03-04T16:05:25.149123+0000 mgr.y (mgr.24428) 607 : cluster 0 pgmap v979: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T16:05:26.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:26 smithi151 bash[23756]: cluster 2024-03-04T16:05:25.149123+0000 mgr.y (mgr.24428) 607 : cluster 0 pgmap v979: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T16:05:27.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:05:27 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:05:28.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:28 smithi117 bash[21480]: audit 2024-03-04T16:05:27.094248+0000 mgr.y (mgr.24428) 608 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:05:28.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:28 smithi117 bash[21480]: cluster 2024-03-04T16:05:27.151127+0000 mgr.y (mgr.24428) 609 : cluster 0 pgmap v980: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-03-04T16:05:28.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:28 smithi117 bash[28843]: audit 2024-03-04T16:05:27.094248+0000 mgr.y (mgr.24428) 608 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:05:28.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:28 smithi117 bash[28843]: cluster 2024-03-04T16:05:27.151127+0000 mgr.y (mgr.24428) 609 : cluster 0 pgmap v980: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-03-04T16:05:28.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:28 smithi151 bash[23756]: audit 2024-03-04T16:05:27.094248+0000 mgr.y (mgr.24428) 608 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:05:28.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:28 smithi151 bash[23756]: cluster 2024-03-04T16:05:27.151127+0000 mgr.y (mgr.24428) 609 : cluster 0 pgmap v980: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 895 B/s wr, 2 op/s 2024-03-04T16:05:30.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:30 smithi117 bash[21480]: cluster 2024-03-04T16:05:29.151911+0000 mgr.y (mgr.24428) 610 : cluster 0 pgmap v981: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 794 B/s wr, 2 op/s 2024-03-04T16:05:30.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:30 smithi117 bash[21480]: audit 2024-03-04T16:05:29.865214+0000 mon.a (mon.0) 3150 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:05:30.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:30 smithi117 bash[21480]: audit 2024-03-04T16:05:29.870280+0000 mon.c (mon.2) 799 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:05:30.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:30 smithi117 bash[28843]: cluster 2024-03-04T16:05:29.151911+0000 mgr.y (mgr.24428) 610 : cluster 0 pgmap v981: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 794 B/s wr, 2 op/s 2024-03-04T16:05:30.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:30 smithi117 bash[28843]: audit 2024-03-04T16:05:29.865214+0000 mon.a (mon.0) 3150 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:05:30.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:30 smithi117 bash[28843]: audit 2024-03-04T16:05:29.870280+0000 mon.c (mon.2) 799 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:05:30.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:30 smithi151 bash[23756]: cluster 2024-03-04T16:05:29.151911+0000 mgr.y (mgr.24428) 610 : cluster 0 pgmap v981: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 794 B/s wr, 2 op/s 2024-03-04T16:05:30.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:30 smithi151 bash[23756]: audit 2024-03-04T16:05:29.865214+0000 mon.a (mon.0) 3150 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:05:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:30 smithi151 bash[23756]: audit 2024-03-04T16:05:29.870280+0000 mon.c (mon.2) 799 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:05:32.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:32 smithi117 bash[21480]: cluster 2024-03-04T16:05:31.152694+0000 mgr.y (mgr.24428) 611 : cluster 0 pgmap v982: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 715 B/s wr, 2 op/s 2024-03-04T16:05:32.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:32 smithi117 bash[28843]: cluster 2024-03-04T16:05:31.152694+0000 mgr.y (mgr.24428) 611 : cluster 0 pgmap v982: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 715 B/s wr, 2 op/s 2024-03-04T16:05:32.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:32 smithi151 bash[23756]: cluster 2024-03-04T16:05:31.152694+0000 mgr.y (mgr.24428) 611 : cluster 0 pgmap v982: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 715 B/s wr, 2 op/s 2024-03-04T16:05:34.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:34 smithi117 bash[28843]: cluster 2024-03-04T16:05:33.154946+0000 mgr.y (mgr.24428) 612 : cluster 0 pgmap v983: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 596 B/s wr, 2 op/s 2024-03-04T16:05:34.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:34 smithi117 bash[21480]: cluster 2024-03-04T16:05:33.154946+0000 mgr.y (mgr.24428) 612 : cluster 0 pgmap v983: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 596 B/s wr, 2 op/s 2024-03-04T16:05:34.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:34 smithi151 bash[23756]: cluster 2024-03-04T16:05:33.154946+0000 mgr.y (mgr.24428) 612 : cluster 0 pgmap v983: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.4 KiB/s rd, 596 B/s wr, 2 op/s 2024-03-04T16:05:36.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:36 smithi117 bash[21480]: cluster 2024-03-04T16:05:35.155727+0000 mgr.y (mgr.24428) 613 : cluster 0 pgmap v984: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:05:36.567 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:05:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:05:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:05:36.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:36 smithi117 bash[28843]: cluster 2024-03-04T16:05:35.155727+0000 mgr.y (mgr.24428) 613 : cluster 0 pgmap v984: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:05:36.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:36 smithi151 bash[23756]: cluster 2024-03-04T16:05:35.155727+0000 mgr.y (mgr.24428) 613 : cluster 0 pgmap v984: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:05:37.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:05:37 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:05:37.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:37 smithi151 bash[23756]: cluster 2024-03-04T16:05:36.253395+0000 mon.a (mon.0) 3151 : cluster 0 osdmap e606: 8 total, 8 up, 8 in 2024-03-04T16:05:37.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:37 smithi151 bash[23756]: audit 2024-03-04T16:05:36.326199+0000 mon.a (mon.0) 3152 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:05:37.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:37 smithi117 bash[21480]: cluster 2024-03-04T16:05:36.253395+0000 mon.a (mon.0) 3151 : cluster 0 osdmap e606: 8 total, 8 up, 8 in 2024-03-04T16:05:37.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:37 smithi117 bash[21480]: audit 2024-03-04T16:05:36.326199+0000 mon.a (mon.0) 3152 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:05:37.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:37 smithi117 bash[28843]: cluster 2024-03-04T16:05:36.253395+0000 mon.a (mon.0) 3151 : cluster 0 osdmap e606: 8 total, 8 up, 8 in 2024-03-04T16:05:37.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:37 smithi117 bash[28843]: audit 2024-03-04T16:05:36.326199+0000 mon.a (mon.0) 3152 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:05:38.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:38 smithi117 bash[21480]: audit 2024-03-04T16:05:37.105670+0000 mgr.y (mgr.24428) 614 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:05:38.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:38 smithi117 bash[21480]: cluster 2024-03-04T16:05:37.157822+0000 mgr.y (mgr.24428) 615 : cluster 0 pgmap v986: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:05:38.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:38 smithi117 bash[21480]: audit 2024-03-04T16:05:37.255282+0000 mon.a (mon.0) 3153 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:05:38.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:38 smithi117 bash[21480]: cluster 2024-03-04T16:05:37.265413+0000 mon.a (mon.0) 3154 : cluster 0 osdmap e607: 8 total, 8 up, 8 in 2024-03-04T16:05:38.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:38 smithi117 bash[21480]: audit 2024-03-04T16:05:37.266327+0000 mon.a (mon.0) 3155 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-116"}]: dispatch 2024-03-04T16:05:38.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:38 smithi117 bash[28843]: audit 2024-03-04T16:05:37.105670+0000 mgr.y (mgr.24428) 614 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:05:38.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:38 smithi117 bash[28843]: cluster 2024-03-04T16:05:37.157822+0000 mgr.y (mgr.24428) 615 : cluster 0 pgmap v986: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:05:38.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:38 smithi117 bash[28843]: audit 2024-03-04T16:05:37.255282+0000 mon.a (mon.0) 3153 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:05:38.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:38 smithi117 bash[28843]: cluster 2024-03-04T16:05:37.265413+0000 mon.a (mon.0) 3154 : cluster 0 osdmap e607: 8 total, 8 up, 8 in 2024-03-04T16:05:38.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:38 smithi117 bash[28843]: audit 2024-03-04T16:05:37.266327+0000 mon.a (mon.0) 3155 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-116"}]: dispatch 2024-03-04T16:05:38.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:38 smithi151 bash[23756]: audit 2024-03-04T16:05:37.105670+0000 mgr.y (mgr.24428) 614 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:05:38.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:38 smithi151 bash[23756]: cluster 2024-03-04T16:05:37.157822+0000 mgr.y (mgr.24428) 615 : cluster 0 pgmap v986: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:05:38.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:38 smithi151 bash[23756]: audit 2024-03-04T16:05:37.255282+0000 mon.a (mon.0) 3153 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:05:38.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:38 smithi151 bash[23756]: cluster 2024-03-04T16:05:37.265413+0000 mon.a (mon.0) 3154 : cluster 0 osdmap e607: 8 total, 8 up, 8 in 2024-03-04T16:05:38.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:38 smithi151 bash[23756]: audit 2024-03-04T16:05:37.266327+0000 mon.a (mon.0) 3155 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-116"}]: dispatch 2024-03-04T16:05:39.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:39 smithi117 bash[21480]: cluster 2024-03-04T16:05:38.255751+0000 mon.a (mon.0) 3156 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:05:39.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:39 smithi117 bash[21480]: audit 2024-03-04T16:05:38.258161+0000 mon.a (mon.0) 3157 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-116"}]': finished 2024-03-04T16:05:39.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:39 smithi117 bash[21480]: cluster 2024-03-04T16:05:38.266705+0000 mon.a (mon.0) 3158 : cluster 0 osdmap e608: 8 total, 8 up, 8 in 2024-03-04T16:05:39.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:39 smithi117 bash[28843]: cluster 2024-03-04T16:05:38.255751+0000 mon.a (mon.0) 3156 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:05:39.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:39 smithi117 bash[28843]: audit 2024-03-04T16:05:38.258161+0000 mon.a (mon.0) 3157 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-116"}]': finished 2024-03-04T16:05:39.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:39 smithi117 bash[28843]: cluster 2024-03-04T16:05:38.266705+0000 mon.a (mon.0) 3158 : cluster 0 osdmap e608: 8 total, 8 up, 8 in 2024-03-04T16:05:39.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:39 smithi151 bash[23756]: cluster 2024-03-04T16:05:38.255751+0000 mon.a (mon.0) 3156 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:05:39.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:39 smithi151 bash[23756]: audit 2024-03-04T16:05:38.258161+0000 mon.a (mon.0) 3157 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-116"}]': finished 2024-03-04T16:05:39.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:39 smithi151 bash[23756]: cluster 2024-03-04T16:05:38.266705+0000 mon.a (mon.0) 3158 : cluster 0 osdmap e608: 8 total, 8 up, 8 in 2024-03-04T16:05:40.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:40 smithi117 bash[28843]: cluster 2024-03-04T16:05:39.158653+0000 mgr.y (mgr.24428) 616 : cluster 0 pgmap v989: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-04T16:05:40.566 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:40 smithi117 bash[28843]: cluster 2024-03-04T16:05:39.272234+0000 mon.a (mon.0) 3159 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:05:40.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:40 smithi117 bash[28843]: cluster 2024-03-04T16:05:39.294972+0000 mon.a (mon.0) 3160 : cluster 0 osdmap e609: 8 total, 8 up, 8 in 2024-03-04T16:05:40.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:40 smithi117 bash[28843]: cluster 2024-03-04T16:05:39.767220+0000 mon.a (mon.0) 3161 : cluster 0 osdmap e610: 8 total, 8 up, 8 in 2024-03-04T16:05:40.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:40 smithi117 bash[28843]: audit 2024-03-04T16:05:39.772709+0000 mon.a (mon.0) 3162 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:05:40.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:40 smithi117 bash[21480]: cluster 2024-03-04T16:05:39.158653+0000 mgr.y (mgr.24428) 616 : cluster 0 pgmap v989: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-04T16:05:40.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:40 smithi117 bash[21480]: cluster 2024-03-04T16:05:39.272234+0000 mon.a (mon.0) 3159 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:05:40.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:40 smithi117 bash[21480]: cluster 2024-03-04T16:05:39.294972+0000 mon.a (mon.0) 3160 : cluster 0 osdmap e609: 8 total, 8 up, 8 in 2024-03-04T16:05:40.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:40 smithi117 bash[21480]: cluster 2024-03-04T16:05:39.767220+0000 mon.a (mon.0) 3161 : cluster 0 osdmap e610: 8 total, 8 up, 8 in 2024-03-04T16:05:40.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:40 smithi117 bash[21480]: audit 2024-03-04T16:05:39.772709+0000 mon.a (mon.0) 3162 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:05:40.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:40 smithi151 bash[23756]: cluster 2024-03-04T16:05:39.158653+0000 mgr.y (mgr.24428) 616 : cluster 0 pgmap v989: 204 pgs: 204 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-04T16:05:40.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:40 smithi151 bash[23756]: cluster 2024-03-04T16:05:39.272234+0000 mon.a (mon.0) 3159 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:05:40.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:40 smithi151 bash[23756]: cluster 2024-03-04T16:05:39.294972+0000 mon.a (mon.0) 3160 : cluster 0 osdmap e609: 8 total, 8 up, 8 in 2024-03-04T16:05:40.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:40 smithi151 bash[23756]: cluster 2024-03-04T16:05:39.767220+0000 mon.a (mon.0) 3161 : cluster 0 osdmap e610: 8 total, 8 up, 8 in 2024-03-04T16:05:40.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:40 smithi151 bash[23756]: audit 2024-03-04T16:05:39.772709+0000 mon.a (mon.0) 3162 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:05:42.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:41 smithi117 bash[28843]: audit 2024-03-04T16:05:40.761305+0000 mon.a (mon.0) 3163 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:05:42.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:41 smithi117 bash[28843]: cluster 2024-03-04T16:05:40.770764+0000 mon.a (mon.0) 3164 : cluster 0 osdmap e611: 8 total, 8 up, 8 in 2024-03-04T16:05:42.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:41 smithi117 bash[28843]: cluster 2024-03-04T16:05:41.159431+0000 mgr.y (mgr.24428) 617 : cluster 0 pgmap v993: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:05:42.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:41 smithi117 bash[21480]: audit 2024-03-04T16:05:40.761305+0000 mon.a (mon.0) 3163 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:05:42.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:41 smithi117 bash[21480]: cluster 2024-03-04T16:05:40.770764+0000 mon.a (mon.0) 3164 : cluster 0 osdmap e611: 8 total, 8 up, 8 in 2024-03-04T16:05:42.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:41 smithi117 bash[21480]: cluster 2024-03-04T16:05:41.159431+0000 mgr.y (mgr.24428) 617 : cluster 0 pgmap v993: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:05:42.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:41 smithi151 bash[23756]: audit 2024-03-04T16:05:40.761305+0000 mon.a (mon.0) 3163 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:05:42.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:41 smithi151 bash[23756]: cluster 2024-03-04T16:05:40.770764+0000 mon.a (mon.0) 3164 : cluster 0 osdmap e611: 8 total, 8 up, 8 in 2024-03-04T16:05:42.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:41 smithi151 bash[23756]: cluster 2024-03-04T16:05:41.159431+0000 mgr.y (mgr.24428) 617 : cluster 0 pgmap v993: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 654 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:05:43.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:42 smithi117 bash[28843]: cluster 2024-03-04T16:05:41.779121+0000 mon.a (mon.0) 3165 : cluster 0 osdmap e612: 8 total, 8 up, 8 in 2024-03-04T16:05:43.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:42 smithi117 bash[28843]: audit 2024-03-04T16:05:41.796088+0000 mon.a (mon.0) 3166 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:05:43.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:42 smithi117 bash[21480]: cluster 2024-03-04T16:05:41.779121+0000 mon.a (mon.0) 3165 : cluster 0 osdmap e612: 8 total, 8 up, 8 in 2024-03-04T16:05:43.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:42 smithi117 bash[21480]: audit 2024-03-04T16:05:41.796088+0000 mon.a (mon.0) 3166 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:05:43.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:42 smithi151 bash[23756]: cluster 2024-03-04T16:05:41.779121+0000 mon.a (mon.0) 3165 : cluster 0 osdmap e612: 8 total, 8 up, 8 in 2024-03-04T16:05:43.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:42 smithi151 bash[23756]: audit 2024-03-04T16:05:41.796088+0000 mon.a (mon.0) 3166 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:05:44.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:43 smithi117 bash[21480]: audit 2024-03-04T16:05:42.773048+0000 mon.a (mon.0) 3167 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-118", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:05:44.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:43 smithi117 bash[21480]: cluster 2024-03-04T16:05:42.780460+0000 mon.a (mon.0) 3168 : cluster 0 osdmap e613: 8 total, 8 up, 8 in 2024-03-04T16:05:44.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:43 smithi117 bash[21480]: audit 2024-03-04T16:05:42.784269+0000 mon.a (mon.0) 3169 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-118"}]: dispatch 2024-03-04T16:05:44.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:43 smithi117 bash[21480]: cluster 2024-03-04T16:05:43.160259+0000 mgr.y (mgr.24428) 618 : cluster 0 pgmap v996: 204 pgs: 204 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 264 B/s wr, 1 op/s 2024-03-04T16:05:44.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:43 smithi117 bash[21480]: audit 2024-03-04T16:05:43.775694+0000 mon.a (mon.0) 3170 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-118"}]': finished 2024-03-04T16:05:44.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:43 smithi117 bash[21480]: cluster 2024-03-04T16:05:43.785749+0000 mon.a (mon.0) 3171 : cluster 0 osdmap e614: 8 total, 8 up, 8 in 2024-03-04T16:05:44.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:43 smithi117 bash[28843]: audit 2024-03-04T16:05:42.773048+0000 mon.a (mon.0) 3167 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-118", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:05:44.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:43 smithi117 bash[28843]: cluster 2024-03-04T16:05:42.780460+0000 mon.a (mon.0) 3168 : cluster 0 osdmap e613: 8 total, 8 up, 8 in 2024-03-04T16:05:44.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:43 smithi117 bash[28843]: audit 2024-03-04T16:05:42.784269+0000 mon.a (mon.0) 3169 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-118"}]: dispatch 2024-03-04T16:05:44.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:43 smithi117 bash[28843]: cluster 2024-03-04T16:05:43.160259+0000 mgr.y (mgr.24428) 618 : cluster 0 pgmap v996: 204 pgs: 204 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 264 B/s wr, 1 op/s 2024-03-04T16:05:44.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:43 smithi117 bash[28843]: audit 2024-03-04T16:05:43.775694+0000 mon.a (mon.0) 3170 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-118"}]': finished 2024-03-04T16:05:44.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:43 smithi117 bash[28843]: cluster 2024-03-04T16:05:43.785749+0000 mon.a (mon.0) 3171 : cluster 0 osdmap e614: 8 total, 8 up, 8 in 2024-03-04T16:05:44.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:43 smithi151 bash[23756]: audit 2024-03-04T16:05:42.773048+0000 mon.a (mon.0) 3167 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-118", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:05:44.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:43 smithi151 bash[23756]: cluster 2024-03-04T16:05:42.780460+0000 mon.a (mon.0) 3168 : cluster 0 osdmap e613: 8 total, 8 up, 8 in 2024-03-04T16:05:44.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:43 smithi151 bash[23756]: audit 2024-03-04T16:05:42.784269+0000 mon.a (mon.0) 3169 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-118"}]: dispatch 2024-03-04T16:05:44.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:43 smithi151 bash[23756]: cluster 2024-03-04T16:05:43.160259+0000 mgr.y (mgr.24428) 618 : cluster 0 pgmap v996: 204 pgs: 204 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 264 B/s wr, 1 op/s 2024-03-04T16:05:44.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:43 smithi151 bash[23756]: audit 2024-03-04T16:05:43.775694+0000 mon.a (mon.0) 3170 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-118"}]': finished 2024-03-04T16:05:44.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:43 smithi151 bash[23756]: cluster 2024-03-04T16:05:43.785749+0000 mon.a (mon.0) 3171 : cluster 0 osdmap e614: 8 total, 8 up, 8 in 2024-03-04T16:05:45.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:44 smithi117 bash[28843]: audit 2024-03-04T16:05:43.789529+0000 mon.a (mon.0) 3172 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-118", "mode": "writeback"}]: dispatch 2024-03-04T16:05:45.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:44 smithi117 bash[28843]: cluster 2024-03-04T16:05:44.756299+0000 mon.a (mon.0) 3173 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:05:45.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:44 smithi117 bash[28843]: cluster 2024-03-04T16:05:44.776329+0000 mon.a (mon.0) 3174 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:05:45.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:44 smithi117 bash[28843]: audit 2024-03-04T16:05:44.778609+0000 mon.a (mon.0) 3175 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-118", "mode": "writeback"}]': finished 2024-03-04T16:05:45.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:44 smithi117 bash[28843]: cluster 2024-03-04T16:05:44.787200+0000 mon.a (mon.0) 3176 : cluster 0 osdmap e615: 8 total, 8 up, 8 in 2024-03-04T16:05:45.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:44 smithi117 bash[21480]: audit 2024-03-04T16:05:43.789529+0000 mon.a (mon.0) 3172 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-118", "mode": "writeback"}]: dispatch 2024-03-04T16:05:45.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:44 smithi117 bash[21480]: cluster 2024-03-04T16:05:44.756299+0000 mon.a (mon.0) 3173 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:05:45.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:44 smithi117 bash[21480]: cluster 2024-03-04T16:05:44.776329+0000 mon.a (mon.0) 3174 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:05:45.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:44 smithi117 bash[21480]: audit 2024-03-04T16:05:44.778609+0000 mon.a (mon.0) 3175 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-118", "mode": "writeback"}]': finished 2024-03-04T16:05:45.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:44 smithi117 bash[21480]: cluster 2024-03-04T16:05:44.787200+0000 mon.a (mon.0) 3176 : cluster 0 osdmap e615: 8 total, 8 up, 8 in 2024-03-04T16:05:45.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:44 smithi151 bash[23756]: audit 2024-03-04T16:05:43.789529+0000 mon.a (mon.0) 3172 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-118", "mode": "writeback"}]: dispatch 2024-03-04T16:05:45.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:44 smithi151 bash[23756]: cluster 2024-03-04T16:05:44.756299+0000 mon.a (mon.0) 3173 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:05:45.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:44 smithi151 bash[23756]: cluster 2024-03-04T16:05:44.776329+0000 mon.a (mon.0) 3174 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:05:45.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:44 smithi151 bash[23756]: audit 2024-03-04T16:05:44.778609+0000 mon.a (mon.0) 3175 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-118", "mode": "writeback"}]': finished 2024-03-04T16:05:45.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:44 smithi151 bash[23756]: cluster 2024-03-04T16:05:44.787200+0000 mon.a (mon.0) 3176 : cluster 0 osdmap e615: 8 total, 8 up, 8 in 2024-03-04T16:05:46.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:45 smithi117 bash[21480]: audit 2024-03-04T16:05:44.903507+0000 mon.c (mon.2) 800 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:05:46.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:45 smithi117 bash[21480]: cluster 2024-03-04T16:05:45.161046+0000 mgr.y (mgr.24428) 619 : cluster 0 pgmap v999: 204 pgs: 204 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:46.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:45 smithi117 bash[21480]: cluster 2024-03-04T16:05:45.788969+0000 mon.a (mon.0) 3177 : cluster 0 osdmap e616: 8 total, 8 up, 8 in 2024-03-04T16:05:46.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:45 smithi117 bash[28843]: audit 2024-03-04T16:05:44.903507+0000 mon.c (mon.2) 800 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:05:46.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:45 smithi117 bash[28843]: cluster 2024-03-04T16:05:45.161046+0000 mgr.y (mgr.24428) 619 : cluster 0 pgmap v999: 204 pgs: 204 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:46.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:45 smithi117 bash[28843]: cluster 2024-03-04T16:05:45.788969+0000 mon.a (mon.0) 3177 : cluster 0 osdmap e616: 8 total, 8 up, 8 in 2024-03-04T16:05:46.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:45 smithi151 bash[23756]: audit 2024-03-04T16:05:44.903507+0000 mon.c (mon.2) 800 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:05:46.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:45 smithi151 bash[23756]: cluster 2024-03-04T16:05:45.161046+0000 mgr.y (mgr.24428) 619 : cluster 0 pgmap v999: 204 pgs: 204 active+clean; 582 KiB data, 655 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:46.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:45 smithi151 bash[23756]: cluster 2024-03-04T16:05:45.788969+0000 mon.a (mon.0) 3177 : cluster 0 osdmap e616: 8 total, 8 up, 8 in 2024-03-04T16:05:46.566 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:05:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:05:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:05:47.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:46 smithi117 bash[21480]: audit 2024-03-04T16:05:45.875615+0000 mon.a (mon.0) 3178 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:05:47.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:46 smithi117 bash[28843]: audit 2024-03-04T16:05:45.875615+0000 mon.a (mon.0) 3178 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:05:47.104 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:46 smithi151 bash[23756]: audit 2024-03-04T16:05:45.875615+0000 mon.a (mon.0) 3178 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:05:47.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:05:47 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:05:48.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:47 smithi151 bash[23756]: audit 2024-03-04T16:05:46.817978+0000 mon.a (mon.0) 3179 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:05:48.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:47 smithi151 bash[23756]: cluster 2024-03-04T16:05:46.824891+0000 mon.a (mon.0) 3180 : cluster 0 osdmap e617: 8 total, 8 up, 8 in 2024-03-04T16:05:48.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:47 smithi151 bash[23756]: audit 2024-03-04T16:05:46.827856+0000 mon.a (mon.0) 3181 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-118"}]: dispatch 2024-03-04T16:05:48.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:47 smithi151 bash[23756]: audit 2024-03-04T16:05:47.108394+0000 mgr.y (mgr.24428) 620 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:05:48.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:47 smithi151 bash[23756]: cluster 2024-03-04T16:05:47.161847+0000 mgr.y (mgr.24428) 621 : cluster 0 pgmap v1002: 204 pgs: 204 active+clean; 582 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:48.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:47 smithi117 bash[21480]: audit 2024-03-04T16:05:46.817978+0000 mon.a (mon.0) 3179 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:05:48.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:47 smithi117 bash[21480]: cluster 2024-03-04T16:05:46.824891+0000 mon.a (mon.0) 3180 : cluster 0 osdmap e617: 8 total, 8 up, 8 in 2024-03-04T16:05:48.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:47 smithi117 bash[21480]: audit 2024-03-04T16:05:46.827856+0000 mon.a (mon.0) 3181 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-118"}]: dispatch 2024-03-04T16:05:48.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:47 smithi117 bash[21480]: audit 2024-03-04T16:05:47.108394+0000 mgr.y (mgr.24428) 620 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:05:48.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:47 smithi117 bash[21480]: cluster 2024-03-04T16:05:47.161847+0000 mgr.y (mgr.24428) 621 : cluster 0 pgmap v1002: 204 pgs: 204 active+clean; 582 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:48.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:47 smithi117 bash[28843]: audit 2024-03-04T16:05:46.817978+0000 mon.a (mon.0) 3179 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:05:48.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:47 smithi117 bash[28843]: cluster 2024-03-04T16:05:46.824891+0000 mon.a (mon.0) 3180 : cluster 0 osdmap e617: 8 total, 8 up, 8 in 2024-03-04T16:05:48.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:47 smithi117 bash[28843]: audit 2024-03-04T16:05:46.827856+0000 mon.a (mon.0) 3181 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-118"}]: dispatch 2024-03-04T16:05:48.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:47 smithi117 bash[28843]: audit 2024-03-04T16:05:47.108394+0000 mgr.y (mgr.24428) 620 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:05:48.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:47 smithi117 bash[28843]: cluster 2024-03-04T16:05:47.161847+0000 mgr.y (mgr.24428) 621 : cluster 0 pgmap v1002: 204 pgs: 204 active+clean; 582 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:49.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:49 smithi117 bash[21480]: cluster 2024-03-04T16:05:47.819127+0000 mon.a (mon.0) 3182 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:05:49.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:49 smithi117 bash[21480]: audit 2024-03-04T16:05:47.828911+0000 mon.a (mon.0) 3183 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-118"}]': finished 2024-03-04T16:05:49.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:49 smithi117 bash[21480]: cluster 2024-03-04T16:05:47.839428+0000 mon.a (mon.0) 3184 : cluster 0 osdmap e618: 8 total, 8 up, 8 in 2024-03-04T16:05:49.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:49 smithi117 bash[28843]: cluster 2024-03-04T16:05:47.819127+0000 mon.a (mon.0) 3182 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:05:49.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:49 smithi117 bash[28843]: audit 2024-03-04T16:05:47.828911+0000 mon.a (mon.0) 3183 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-118"}]': finished 2024-03-04T16:05:49.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:49 smithi117 bash[28843]: cluster 2024-03-04T16:05:47.839428+0000 mon.a (mon.0) 3184 : cluster 0 osdmap e618: 8 total, 8 up, 8 in 2024-03-04T16:05:49.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:49 smithi151 bash[23756]: cluster 2024-03-04T16:05:47.819127+0000 mon.a (mon.0) 3182 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:05:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:49 smithi151 bash[23756]: audit 2024-03-04T16:05:47.828911+0000 mon.a (mon.0) 3183 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-118"}]': finished 2024-03-04T16:05:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:49 smithi151 bash[23756]: cluster 2024-03-04T16:05:47.839428+0000 mon.a (mon.0) 3184 : cluster 0 osdmap e618: 8 total, 8 up, 8 in 2024-03-04T16:05:50.566 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:50 smithi117 bash[21480]: cluster 2024-03-04T16:05:48.846046+0000 mon.a (mon.0) 3185 : cluster 0 osdmap e619: 8 total, 8 up, 8 in 2024-03-04T16:05:50.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:50 smithi117 bash[21480]: cluster 2024-03-04T16:05:49.162570+0000 mgr.y (mgr.24428) 622 : cluster 0 pgmap v1005: 172 pgs: 172 active+clean; 582 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:50.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:50 smithi117 bash[21480]: cluster 2024-03-04T16:05:49.770597+0000 mon.a (mon.0) 3186 : cluster 0 osdmap e620: 8 total, 8 up, 8 in 2024-03-04T16:05:50.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:50 smithi117 bash[21480]: audit 2024-03-04T16:05:49.773707+0000 mon.a (mon.0) 3187 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:05:50.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:50 smithi117 bash[28843]: cluster 2024-03-04T16:05:48.846046+0000 mon.a (mon.0) 3185 : cluster 0 osdmap e619: 8 total, 8 up, 8 in 2024-03-04T16:05:50.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:50 smithi117 bash[28843]: cluster 2024-03-04T16:05:49.162570+0000 mgr.y (mgr.24428) 622 : cluster 0 pgmap v1005: 172 pgs: 172 active+clean; 582 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:50.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:50 smithi117 bash[28843]: cluster 2024-03-04T16:05:49.770597+0000 mon.a (mon.0) 3186 : cluster 0 osdmap e620: 8 total, 8 up, 8 in 2024-03-04T16:05:50.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:50 smithi117 bash[28843]: audit 2024-03-04T16:05:49.773707+0000 mon.a (mon.0) 3187 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:05:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:50 smithi151 bash[23756]: cluster 2024-03-04T16:05:48.846046+0000 mon.a (mon.0) 3185 : cluster 0 osdmap e619: 8 total, 8 up, 8 in 2024-03-04T16:05:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:50 smithi151 bash[23756]: cluster 2024-03-04T16:05:49.162570+0000 mgr.y (mgr.24428) 622 : cluster 0 pgmap v1005: 172 pgs: 172 active+clean; 582 KiB data, 673 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:50 smithi151 bash[23756]: cluster 2024-03-04T16:05:49.770597+0000 mon.a (mon.0) 3186 : cluster 0 osdmap e620: 8 total, 8 up, 8 in 2024-03-04T16:05:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:50 smithi151 bash[23756]: audit 2024-03-04T16:05:49.773707+0000 mon.a (mon.0) 3187 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:05:52.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:51 smithi117 bash[28843]: audit 2024-03-04T16:05:50.768385+0000 mon.a (mon.0) 3188 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:05:52.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:51 smithi117 bash[28843]: cluster 2024-03-04T16:05:50.781861+0000 mon.a (mon.0) 3189 : cluster 0 osdmap e621: 8 total, 8 up, 8 in 2024-03-04T16:05:52.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:51 smithi117 bash[28843]: audit 2024-03-04T16:05:50.823357+0000 mon.a (mon.0) 3190 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:05:52.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:51 smithi117 bash[28843]: cluster 2024-03-04T16:05:51.163377+0000 mgr.y (mgr.24428) 623 : cluster 0 pgmap v1008: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 673 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:05:52.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:51 smithi117 bash[21480]: audit 2024-03-04T16:05:50.768385+0000 mon.a (mon.0) 3188 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:05:52.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:51 smithi117 bash[21480]: cluster 2024-03-04T16:05:50.781861+0000 mon.a (mon.0) 3189 : cluster 0 osdmap e621: 8 total, 8 up, 8 in 2024-03-04T16:05:52.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:51 smithi117 bash[21480]: audit 2024-03-04T16:05:50.823357+0000 mon.a (mon.0) 3190 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:05:52.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:51 smithi117 bash[21480]: cluster 2024-03-04T16:05:51.163377+0000 mgr.y (mgr.24428) 623 : cluster 0 pgmap v1008: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 673 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:05:52.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:51 smithi151 bash[23756]: audit 2024-03-04T16:05:50.768385+0000 mon.a (mon.0) 3188 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:05:52.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:51 smithi151 bash[23756]: cluster 2024-03-04T16:05:50.781861+0000 mon.a (mon.0) 3189 : cluster 0 osdmap e621: 8 total, 8 up, 8 in 2024-03-04T16:05:52.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:51 smithi151 bash[23756]: audit 2024-03-04T16:05:50.823357+0000 mon.a (mon.0) 3190 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:05:52.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:51 smithi151 bash[23756]: cluster 2024-03-04T16:05:51.163377+0000 mgr.y (mgr.24428) 623 : cluster 0 pgmap v1008: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 673 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:05:53.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:52 smithi117 bash[28843]: audit 2024-03-04T16:05:51.777566+0000 mon.a (mon.0) 3191 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-120", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:05:53.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:52 smithi117 bash[28843]: cluster 2024-03-04T16:05:51.788452+0000 mon.a (mon.0) 3192 : cluster 0 osdmap e622: 8 total, 8 up, 8 in 2024-03-04T16:05:53.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:52 smithi117 bash[28843]: audit 2024-03-04T16:05:51.790025+0000 mon.a (mon.0) 3193 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-120"}]: dispatch 2024-03-04T16:05:53.066 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:52 smithi117 bash[28843]: audit 2024-03-04T16:05:52.448641+0000 mon.c (mon.2) 801 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:05:53.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:52 smithi117 bash[21480]: audit 2024-03-04T16:05:51.777566+0000 mon.a (mon.0) 3191 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-120", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:05:53.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:52 smithi117 bash[21480]: cluster 2024-03-04T16:05:51.788452+0000 mon.a (mon.0) 3192 : cluster 0 osdmap e622: 8 total, 8 up, 8 in 2024-03-04T16:05:53.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:52 smithi117 bash[21480]: audit 2024-03-04T16:05:51.790025+0000 mon.a (mon.0) 3193 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-120"}]: dispatch 2024-03-04T16:05:53.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:52 smithi117 bash[21480]: audit 2024-03-04T16:05:52.448641+0000 mon.c (mon.2) 801 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:05:53.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:52 smithi151 bash[23756]: audit 2024-03-04T16:05:51.777566+0000 mon.a (mon.0) 3191 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-120", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:05:53.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:52 smithi151 bash[23756]: cluster 2024-03-04T16:05:51.788452+0000 mon.a (mon.0) 3192 : cluster 0 osdmap e622: 8 total, 8 up, 8 in 2024-03-04T16:05:53.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:52 smithi151 bash[23756]: audit 2024-03-04T16:05:51.790025+0000 mon.a (mon.0) 3193 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-120"}]: dispatch 2024-03-04T16:05:53.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:52 smithi151 bash[23756]: audit 2024-03-04T16:05:52.448641+0000 mon.c (mon.2) 801 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:05:54.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:53 smithi117 bash[21480]: audit 2024-03-04T16:05:52.780222+0000 mon.a (mon.0) 3194 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-120"}]': finished 2024-03-04T16:05:54.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:53 smithi117 bash[21480]: cluster 2024-03-04T16:05:52.786903+0000 mon.a (mon.0) 3195 : cluster 0 osdmap e623: 8 total, 8 up, 8 in 2024-03-04T16:05:54.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:53 smithi117 bash[21480]: audit 2024-03-04T16:05:52.788412+0000 mon.a (mon.0) 3196 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-120", "mode": "writeback"}]: dispatch 2024-03-04T16:05:54.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:53 smithi117 bash[21480]: cluster 2024-03-04T16:05:53.164200+0000 mgr.y (mgr.24428) 624 : cluster 0 pgmap v1011: 204 pgs: 204 active+clean; 582 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:54.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:53 smithi117 bash[21480]: audit 2024-03-04T16:05:53.173546+0000 mon.c (mon.2) 802 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:05:54.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:53 smithi117 bash[21480]: audit 2024-03-04T16:05:53.175320+0000 mon.c (mon.2) 803 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:05:54.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:53 smithi117 bash[21480]: audit 2024-03-04T16:05:53.186860+0000 mon.a (mon.0) 3197 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:05:54.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:53 smithi117 bash[21480]: cluster 2024-03-04T16:05:53.780709+0000 mon.a (mon.0) 3198 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:05:54.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:53 smithi117 bash[21480]: audit 2024-03-04T16:05:53.783113+0000 mon.a (mon.0) 3199 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-120", "mode": "writeback"}]': finished 2024-03-04T16:05:54.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:53 smithi117 bash[28843]: audit 2024-03-04T16:05:52.780222+0000 mon.a (mon.0) 3194 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-120"}]': finished 2024-03-04T16:05:54.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:53 smithi117 bash[28843]: cluster 2024-03-04T16:05:52.786903+0000 mon.a (mon.0) 3195 : cluster 0 osdmap e623: 8 total, 8 up, 8 in 2024-03-04T16:05:54.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:53 smithi117 bash[28843]: audit 2024-03-04T16:05:52.788412+0000 mon.a (mon.0) 3196 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-120", "mode": "writeback"}]: dispatch 2024-03-04T16:05:54.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:53 smithi117 bash[28843]: cluster 2024-03-04T16:05:53.164200+0000 mgr.y (mgr.24428) 624 : cluster 0 pgmap v1011: 204 pgs: 204 active+clean; 582 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:54.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:53 smithi117 bash[28843]: audit 2024-03-04T16:05:53.173546+0000 mon.c (mon.2) 802 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:05:54.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:53 smithi117 bash[28843]: audit 2024-03-04T16:05:53.175320+0000 mon.c (mon.2) 803 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:05:54.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:53 smithi117 bash[28843]: audit 2024-03-04T16:05:53.186860+0000 mon.a (mon.0) 3197 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:05:54.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:53 smithi117 bash[28843]: cluster 2024-03-04T16:05:53.780709+0000 mon.a (mon.0) 3198 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:05:54.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:53 smithi117 bash[28843]: audit 2024-03-04T16:05:53.783113+0000 mon.a (mon.0) 3199 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-120", "mode": "writeback"}]': finished 2024-03-04T16:05:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:53 smithi151 bash[23756]: audit 2024-03-04T16:05:52.780222+0000 mon.a (mon.0) 3194 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-120"}]': finished 2024-03-04T16:05:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:53 smithi151 bash[23756]: cluster 2024-03-04T16:05:52.786903+0000 mon.a (mon.0) 3195 : cluster 0 osdmap e623: 8 total, 8 up, 8 in 2024-03-04T16:05:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:53 smithi151 bash[23756]: audit 2024-03-04T16:05:52.788412+0000 mon.a (mon.0) 3196 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-120", "mode": "writeback"}]: dispatch 2024-03-04T16:05:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:53 smithi151 bash[23756]: cluster 2024-03-04T16:05:53.164200+0000 mgr.y (mgr.24428) 624 : cluster 0 pgmap v1011: 204 pgs: 204 active+clean; 582 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:53 smithi151 bash[23756]: audit 2024-03-04T16:05:53.173546+0000 mon.c (mon.2) 802 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:05:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:53 smithi151 bash[23756]: audit 2024-03-04T16:05:53.175320+0000 mon.c (mon.2) 803 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:05:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:53 smithi151 bash[23756]: audit 2024-03-04T16:05:53.186860+0000 mon.a (mon.0) 3197 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:05:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:53 smithi151 bash[23756]: cluster 2024-03-04T16:05:53.780709+0000 mon.a (mon.0) 3198 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:05:54.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:53 smithi151 bash[23756]: audit 2024-03-04T16:05:53.783113+0000 mon.a (mon.0) 3199 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-120", "mode": "writeback"}]': finished 2024-03-04T16:05:55.066 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:54 smithi117 bash[21480]: cluster 2024-03-04T16:05:53.790084+0000 mon.a (mon.0) 3200 : cluster 0 osdmap e624: 8 total, 8 up, 8 in 2024-03-04T16:05:55.067 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:54 smithi117 bash[21480]: audit 2024-03-04T16:05:53.934897+0000 mon.a (mon.0) 3201 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:05:55.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:54 smithi117 bash[28843]: cluster 2024-03-04T16:05:53.790084+0000 mon.a (mon.0) 3200 : cluster 0 osdmap e624: 8 total, 8 up, 8 in 2024-03-04T16:05:55.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:54 smithi117 bash[28843]: audit 2024-03-04T16:05:53.934897+0000 mon.a (mon.0) 3201 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:05:55.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:54 smithi151 bash[23756]: cluster 2024-03-04T16:05:53.790084+0000 mon.a (mon.0) 3200 : cluster 0 osdmap e624: 8 total, 8 up, 8 in 2024-03-04T16:05:55.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:54 smithi151 bash[23756]: audit 2024-03-04T16:05:53.934897+0000 mon.a (mon.0) 3201 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:05:56.091 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:55 smithi117 bash[28843]: audit 2024-03-04T16:05:54.814351+0000 mon.a (mon.0) 3202 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:05:56.092 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:55 smithi117 bash[28843]: cluster 2024-03-04T16:05:54.820835+0000 mon.a (mon.0) 3203 : cluster 0 osdmap e625: 8 total, 8 up, 8 in 2024-03-04T16:05:56.092 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:55 smithi117 bash[28843]: audit 2024-03-04T16:05:54.821821+0000 mon.a (mon.0) 3204 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-120"}]: dispatch 2024-03-04T16:05:56.092 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:55 smithi117 bash[28843]: cluster 2024-03-04T16:05:55.165017+0000 mgr.y (mgr.24428) 625 : cluster 0 pgmap v1014: 204 pgs: 204 active+clean; 582 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:56.092 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:55 smithi117 bash[21480]: audit 2024-03-04T16:05:54.814351+0000 mon.a (mon.0) 3202 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:05:56.092 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:55 smithi117 bash[21480]: cluster 2024-03-04T16:05:54.820835+0000 mon.a (mon.0) 3203 : cluster 0 osdmap e625: 8 total, 8 up, 8 in 2024-03-04T16:05:56.092 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:55 smithi117 bash[21480]: audit 2024-03-04T16:05:54.821821+0000 mon.a (mon.0) 3204 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-120"}]: dispatch 2024-03-04T16:05:56.092 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:55 smithi117 bash[21480]: cluster 2024-03-04T16:05:55.165017+0000 mgr.y (mgr.24428) 625 : cluster 0 pgmap v1014: 204 pgs: 204 active+clean; 582 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:55 smithi151 bash[23756]: audit 2024-03-04T16:05:54.814351+0000 mon.a (mon.0) 3202 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:05:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:55 smithi151 bash[23756]: cluster 2024-03-04T16:05:54.820835+0000 mon.a (mon.0) 3203 : cluster 0 osdmap e625: 8 total, 8 up, 8 in 2024-03-04T16:05:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:55 smithi151 bash[23756]: audit 2024-03-04T16:05:54.821821+0000 mon.a (mon.0) 3204 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-120"}]: dispatch 2024-03-04T16:05:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:55 smithi151 bash[23756]: cluster 2024-03-04T16:05:55.165017+0000 mgr.y (mgr.24428) 625 : cluster 0 pgmap v1014: 204 pgs: 204 active+clean; 582 KiB data, 691 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:56.566 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:05:56 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:05:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:05:57.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:05:57 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:05:57.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:57 smithi151 bash[23756]: cluster 2024-03-04T16:05:55.815430+0000 mon.a (mon.0) 3205 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:05:57.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:57 smithi151 bash[23756]: audit 2024-03-04T16:05:55.825083+0000 mon.a (mon.0) 3206 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-120"}]': finished 2024-03-04T16:05:57.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:57 smithi151 bash[23756]: cluster 2024-03-04T16:05:55.833426+0000 mon.a (mon.0) 3207 : cluster 0 osdmap e626: 8 total, 8 up, 8 in 2024-03-04T16:05:57.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:57 smithi117 bash[21480]: cluster 2024-03-04T16:05:55.815430+0000 mon.a (mon.0) 3205 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:05:57.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:57 smithi117 bash[21480]: audit 2024-03-04T16:05:55.825083+0000 mon.a (mon.0) 3206 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-120"}]': finished 2024-03-04T16:05:57.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:57 smithi117 bash[21480]: cluster 2024-03-04T16:05:55.833426+0000 mon.a (mon.0) 3207 : cluster 0 osdmap e626: 8 total, 8 up, 8 in 2024-03-04T16:05:57.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:57 smithi117 bash[28843]: cluster 2024-03-04T16:05:55.815430+0000 mon.a (mon.0) 3205 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:05:57.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:57 smithi117 bash[28843]: audit 2024-03-04T16:05:55.825083+0000 mon.a (mon.0) 3206 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-120"}]': finished 2024-03-04T16:05:57.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:57 smithi117 bash[28843]: cluster 2024-03-04T16:05:55.833426+0000 mon.a (mon.0) 3207 : cluster 0 osdmap e626: 8 total, 8 up, 8 in 2024-03-04T16:05:58.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:58 smithi117 bash[21480]: cluster 2024-03-04T16:05:56.838311+0000 mon.a (mon.0) 3208 : cluster 0 osdmap e627: 8 total, 8 up, 8 in 2024-03-04T16:05:58.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:58 smithi117 bash[21480]: audit 2024-03-04T16:05:57.119750+0000 mgr.y (mgr.24428) 626 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:05:58.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:58 smithi117 bash[21480]: cluster 2024-03-04T16:05:57.165737+0000 mgr.y (mgr.24428) 627 : cluster 0 pgmap v1017: 172 pgs: 172 active+clean; 582 KiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:58.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:58 smithi117 bash[21480]: cluster 2024-03-04T16:05:57.839857+0000 mon.a (mon.0) 3209 : cluster 0 osdmap e628: 8 total, 8 up, 8 in 2024-03-04T16:05:58.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:58 smithi117 bash[21480]: audit 2024-03-04T16:05:57.844340+0000 mon.a (mon.0) 3210 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:05:58.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:58 smithi117 bash[28843]: cluster 2024-03-04T16:05:56.838311+0000 mon.a (mon.0) 3208 : cluster 0 osdmap e627: 8 total, 8 up, 8 in 2024-03-04T16:05:58.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:58 smithi117 bash[28843]: audit 2024-03-04T16:05:57.119750+0000 mgr.y (mgr.24428) 626 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:05:58.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:58 smithi117 bash[28843]: cluster 2024-03-04T16:05:57.165737+0000 mgr.y (mgr.24428) 627 : cluster 0 pgmap v1017: 172 pgs: 172 active+clean; 582 KiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:58.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:58 smithi117 bash[28843]: cluster 2024-03-04T16:05:57.839857+0000 mon.a (mon.0) 3209 : cluster 0 osdmap e628: 8 total, 8 up, 8 in 2024-03-04T16:05:58.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:58 smithi117 bash[28843]: audit 2024-03-04T16:05:57.844340+0000 mon.a (mon.0) 3210 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:05:58.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:58 smithi151 bash[23756]: cluster 2024-03-04T16:05:56.838311+0000 mon.a (mon.0) 3208 : cluster 0 osdmap e627: 8 total, 8 up, 8 in 2024-03-04T16:05:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:58 smithi151 bash[23756]: audit 2024-03-04T16:05:57.119750+0000 mgr.y (mgr.24428) 626 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:05:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:58 smithi151 bash[23756]: cluster 2024-03-04T16:05:57.165737+0000 mgr.y (mgr.24428) 627 : cluster 0 pgmap v1017: 172 pgs: 172 active+clean; 582 KiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:05:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:58 smithi151 bash[23756]: cluster 2024-03-04T16:05:57.839857+0000 mon.a (mon.0) 3209 : cluster 0 osdmap e628: 8 total, 8 up, 8 in 2024-03-04T16:05:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:58 smithi151 bash[23756]: audit 2024-03-04T16:05:57.844340+0000 mon.a (mon.0) 3210 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:06:00.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:59 smithi151 bash[23756]: audit 2024-03-04T16:05:58.835369+0000 mon.a (mon.0) 3211 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:00.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:59 smithi151 bash[23756]: cluster 2024-03-04T16:05:58.847458+0000 mon.a (mon.0) 3212 : cluster 0 osdmap e629: 8 total, 8 up, 8 in 2024-03-04T16:06:00.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:59 smithi151 bash[23756]: audit 2024-03-04T16:05:58.899768+0000 mon.a (mon.0) 3213 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:06:00.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:05:59 smithi151 bash[23756]: cluster 2024-03-04T16:05:59.166577+0000 mgr.y (mgr.24428) 628 : cluster 0 pgmap v1020: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:00.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:59 smithi117 bash[21480]: audit 2024-03-04T16:05:58.835369+0000 mon.a (mon.0) 3211 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:00.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:59 smithi117 bash[21480]: cluster 2024-03-04T16:05:58.847458+0000 mon.a (mon.0) 3212 : cluster 0 osdmap e629: 8 total, 8 up, 8 in 2024-03-04T16:06:00.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:59 smithi117 bash[21480]: audit 2024-03-04T16:05:58.899768+0000 mon.a (mon.0) 3213 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:06:00.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:05:59 smithi117 bash[21480]: cluster 2024-03-04T16:05:59.166577+0000 mgr.y (mgr.24428) 628 : cluster 0 pgmap v1020: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:00.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:59 smithi117 bash[28843]: audit 2024-03-04T16:05:58.835369+0000 mon.a (mon.0) 3211 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:00.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:59 smithi117 bash[28843]: cluster 2024-03-04T16:05:58.847458+0000 mon.a (mon.0) 3212 : cluster 0 osdmap e629: 8 total, 8 up, 8 in 2024-03-04T16:06:00.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:59 smithi117 bash[28843]: audit 2024-03-04T16:05:58.899768+0000 mon.a (mon.0) 3213 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:06:00.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:05:59 smithi117 bash[28843]: cluster 2024-03-04T16:05:59.166577+0000 mgr.y (mgr.24428) 628 : cluster 0 pgmap v1020: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 692 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:01.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:00 smithi151 bash[23756]: audit 2024-03-04T16:05:59.844599+0000 mon.a (mon.0) 3214 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-122", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:06:01.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:00 smithi151 bash[23756]: cluster 2024-03-04T16:05:59.858598+0000 mon.a (mon.0) 3215 : cluster 0 osdmap e630: 8 total, 8 up, 8 in 2024-03-04T16:06:01.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:00 smithi151 bash[23756]: audit 2024-03-04T16:05:59.859950+0000 mon.a (mon.0) 3216 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-122"}]: dispatch 2024-03-04T16:06:01.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:00 smithi151 bash[23756]: audit 2024-03-04T16:05:59.944602+0000 mon.a (mon.0) 3217 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:06:01.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:00 smithi151 bash[23756]: audit 2024-03-04T16:05:59.949776+0000 mon.c (mon.2) 804 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:06:01.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:00 smithi117 bash[21480]: audit 2024-03-04T16:05:59.844599+0000 mon.a (mon.0) 3214 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-122", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:06:01.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:00 smithi117 bash[21480]: cluster 2024-03-04T16:05:59.858598+0000 mon.a (mon.0) 3215 : cluster 0 osdmap e630: 8 total, 8 up, 8 in 2024-03-04T16:06:01.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:00 smithi117 bash[21480]: audit 2024-03-04T16:05:59.859950+0000 mon.a (mon.0) 3216 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-122"}]: dispatch 2024-03-04T16:06:01.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:00 smithi117 bash[21480]: audit 2024-03-04T16:05:59.944602+0000 mon.a (mon.0) 3217 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:06:01.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:00 smithi117 bash[21480]: audit 2024-03-04T16:05:59.949776+0000 mon.c (mon.2) 804 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:06:01.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:00 smithi117 bash[28843]: audit 2024-03-04T16:05:59.844599+0000 mon.a (mon.0) 3214 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-122", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:06:01.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:00 smithi117 bash[28843]: cluster 2024-03-04T16:05:59.858598+0000 mon.a (mon.0) 3215 : cluster 0 osdmap e630: 8 total, 8 up, 8 in 2024-03-04T16:06:01.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:00 smithi117 bash[28843]: audit 2024-03-04T16:05:59.859950+0000 mon.a (mon.0) 3216 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-122"}]: dispatch 2024-03-04T16:06:01.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:00 smithi117 bash[28843]: audit 2024-03-04T16:05:59.944602+0000 mon.a (mon.0) 3217 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:06:01.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:00 smithi117 bash[28843]: audit 2024-03-04T16:05:59.949776+0000 mon.c (mon.2) 804 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:06:02.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:01 smithi151 bash[23756]: audit 2024-03-04T16:06:00.847491+0000 mon.a (mon.0) 3218 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-122"}]': finished 2024-03-04T16:06:02.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:01 smithi151 bash[23756]: cluster 2024-03-04T16:06:00.856639+0000 mon.a (mon.0) 3219 : cluster 0 osdmap e631: 8 total, 8 up, 8 in 2024-03-04T16:06:02.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:01 smithi151 bash[23756]: audit 2024-03-04T16:06:00.859729+0000 mon.a (mon.0) 3220 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-122", "mode": "writeback"}]: dispatch 2024-03-04T16:06:02.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:01 smithi151 bash[23756]: cluster 2024-03-04T16:06:01.167430+0000 mgr.y (mgr.24428) 629 : cluster 0 pgmap v1023: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 692 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:06:02.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:01 smithi151 bash[23756]: cluster 2024-03-04T16:06:01.847777+0000 mon.a (mon.0) 3221 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:06:02.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:01 smithi151 bash[23756]: audit 2024-03-04T16:06:01.851945+0000 mon.a (mon.0) 3222 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-122", "mode": "writeback"}]': finished 2024-03-04T16:06:02.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:01 smithi151 bash[23756]: cluster 2024-03-04T16:06:01.859278+0000 mon.a (mon.0) 3223 : cluster 0 osdmap e632: 8 total, 8 up, 8 in 2024-03-04T16:06:02.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:01 smithi117 bash[21480]: audit 2024-03-04T16:06:00.847491+0000 mon.a (mon.0) 3218 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-122"}]': finished 2024-03-04T16:06:02.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:01 smithi117 bash[21480]: cluster 2024-03-04T16:06:00.856639+0000 mon.a (mon.0) 3219 : cluster 0 osdmap e631: 8 total, 8 up, 8 in 2024-03-04T16:06:02.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:01 smithi117 bash[21480]: audit 2024-03-04T16:06:00.859729+0000 mon.a (mon.0) 3220 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-122", "mode": "writeback"}]: dispatch 2024-03-04T16:06:02.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:01 smithi117 bash[21480]: cluster 2024-03-04T16:06:01.167430+0000 mgr.y (mgr.24428) 629 : cluster 0 pgmap v1023: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 692 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:06:02.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:01 smithi117 bash[21480]: cluster 2024-03-04T16:06:01.847777+0000 mon.a (mon.0) 3221 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:06:02.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:01 smithi117 bash[21480]: audit 2024-03-04T16:06:01.851945+0000 mon.a (mon.0) 3222 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-122", "mode": "writeback"}]': finished 2024-03-04T16:06:02.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:01 smithi117 bash[21480]: cluster 2024-03-04T16:06:01.859278+0000 mon.a (mon.0) 3223 : cluster 0 osdmap e632: 8 total, 8 up, 8 in 2024-03-04T16:06:02.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:01 smithi117 bash[28843]: audit 2024-03-04T16:06:00.847491+0000 mon.a (mon.0) 3218 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-122"}]': finished 2024-03-04T16:06:02.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:01 smithi117 bash[28843]: cluster 2024-03-04T16:06:00.856639+0000 mon.a (mon.0) 3219 : cluster 0 osdmap e631: 8 total, 8 up, 8 in 2024-03-04T16:06:02.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:01 smithi117 bash[28843]: audit 2024-03-04T16:06:00.859729+0000 mon.a (mon.0) 3220 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-122", "mode": "writeback"}]: dispatch 2024-03-04T16:06:02.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:01 smithi117 bash[28843]: cluster 2024-03-04T16:06:01.167430+0000 mgr.y (mgr.24428) 629 : cluster 0 pgmap v1023: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 692 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:06:02.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:01 smithi117 bash[28843]: cluster 2024-03-04T16:06:01.847777+0000 mon.a (mon.0) 3221 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:06:02.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:01 smithi117 bash[28843]: audit 2024-03-04T16:06:01.851945+0000 mon.a (mon.0) 3222 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-122", "mode": "writeback"}]': finished 2024-03-04T16:06:02.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:01 smithi117 bash[28843]: cluster 2024-03-04T16:06:01.859278+0000 mon.a (mon.0) 3223 : cluster 0 osdmap e632: 8 total, 8 up, 8 in 2024-03-04T16:06:03.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:02 smithi151 bash[23756]: audit 2024-03-04T16:06:01.986135+0000 mon.a (mon.0) 3224 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:03.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:02 smithi117 bash[21480]: audit 2024-03-04T16:06:01.986135+0000 mon.a (mon.0) 3224 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:03.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:02 smithi117 bash[28843]: audit 2024-03-04T16:06:01.986135+0000 mon.a (mon.0) 3224 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:04.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:03 smithi151 bash[23756]: audit 2024-03-04T16:06:02.888821+0000 mon.a (mon.0) 3225 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:04.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:03 smithi151 bash[23756]: cluster 2024-03-04T16:06:02.896998+0000 mon.a (mon.0) 3226 : cluster 0 osdmap e633: 8 total, 8 up, 8 in 2024-03-04T16:06:04.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:03 smithi151 bash[23756]: audit 2024-03-04T16:06:02.897816+0000 mon.a (mon.0) 3227 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-122"}]: dispatch 2024-03-04T16:06:04.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:03 smithi151 bash[23756]: cluster 2024-03-04T16:06:03.168424+0000 mgr.y (mgr.24428) 630 : cluster 0 pgmap v1026: 204 pgs: 204 active+clean; 582 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:04.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:03 smithi117 bash[21480]: audit 2024-03-04T16:06:02.888821+0000 mon.a (mon.0) 3225 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:04.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:03 smithi117 bash[21480]: cluster 2024-03-04T16:06:02.896998+0000 mon.a (mon.0) 3226 : cluster 0 osdmap e633: 8 total, 8 up, 8 in 2024-03-04T16:06:04.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:03 smithi117 bash[21480]: audit 2024-03-04T16:06:02.897816+0000 mon.a (mon.0) 3227 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-122"}]: dispatch 2024-03-04T16:06:04.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:03 smithi117 bash[21480]: cluster 2024-03-04T16:06:03.168424+0000 mgr.y (mgr.24428) 630 : cluster 0 pgmap v1026: 204 pgs: 204 active+clean; 582 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:04.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:03 smithi117 bash[28843]: audit 2024-03-04T16:06:02.888821+0000 mon.a (mon.0) 3225 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:04.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:03 smithi117 bash[28843]: cluster 2024-03-04T16:06:02.896998+0000 mon.a (mon.0) 3226 : cluster 0 osdmap e633: 8 total, 8 up, 8 in 2024-03-04T16:06:04.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:03 smithi117 bash[28843]: audit 2024-03-04T16:06:02.897816+0000 mon.a (mon.0) 3227 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-122"}]: dispatch 2024-03-04T16:06:04.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:03 smithi117 bash[28843]: cluster 2024-03-04T16:06:03.168424+0000 mgr.y (mgr.24428) 630 : cluster 0 pgmap v1026: 204 pgs: 204 active+clean; 582 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:05.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:05 smithi151 bash[23756]: cluster 2024-03-04T16:06:03.889788+0000 mon.a (mon.0) 3228 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:06:05.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:05 smithi151 bash[23756]: audit 2024-03-04T16:06:03.901292+0000 mon.a (mon.0) 3229 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-122"}]': finished 2024-03-04T16:06:05.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:05 smithi151 bash[23756]: cluster 2024-03-04T16:06:03.912450+0000 mon.a (mon.0) 3230 : cluster 0 osdmap e634: 8 total, 8 up, 8 in 2024-03-04T16:06:05.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:05 smithi117 bash[21480]: cluster 2024-03-04T16:06:03.889788+0000 mon.a (mon.0) 3228 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:06:05.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:05 smithi117 bash[21480]: audit 2024-03-04T16:06:03.901292+0000 mon.a (mon.0) 3229 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-122"}]': finished 2024-03-04T16:06:05.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:05 smithi117 bash[21480]: cluster 2024-03-04T16:06:03.912450+0000 mon.a (mon.0) 3230 : cluster 0 osdmap e634: 8 total, 8 up, 8 in 2024-03-04T16:06:05.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:05 smithi117 bash[28843]: cluster 2024-03-04T16:06:03.889788+0000 mon.a (mon.0) 3228 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:06:05.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:05 smithi117 bash[28843]: audit 2024-03-04T16:06:03.901292+0000 mon.a (mon.0) 3229 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-122"}]': finished 2024-03-04T16:06:05.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:05 smithi117 bash[28843]: cluster 2024-03-04T16:06:03.912450+0000 mon.a (mon.0) 3230 : cluster 0 osdmap e634: 8 total, 8 up, 8 in 2024-03-04T16:06:06.363 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:06:06 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:06:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:06:06.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:06 smithi151 bash[23756]: cluster 2024-03-04T16:06:04.913012+0000 mon.a (mon.0) 3231 : cluster 0 osdmap e635: 8 total, 8 up, 8 in 2024-03-04T16:06:06.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:06 smithi151 bash[23756]: cluster 2024-03-04T16:06:05.169244+0000 mgr.y (mgr.24428) 631 : cluster 0 pgmap v1029: 172 pgs: 172 active+clean; 582 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:06.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:06 smithi151 bash[23756]: cluster 2024-03-04T16:06:05.915292+0000 mon.a (mon.0) 3232 : cluster 0 osdmap e636: 8 total, 8 up, 8 in 2024-03-04T16:06:06.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:06 smithi151 bash[23756]: audit 2024-03-04T16:06:05.922569+0000 mon.a (mon.0) 3233 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:06:06.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:06 smithi117 bash[21480]: cluster 2024-03-04T16:06:04.913012+0000 mon.a (mon.0) 3231 : cluster 0 osdmap e635: 8 total, 8 up, 8 in 2024-03-04T16:06:06.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:06 smithi117 bash[21480]: cluster 2024-03-04T16:06:05.169244+0000 mgr.y (mgr.24428) 631 : cluster 0 pgmap v1029: 172 pgs: 172 active+clean; 582 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:06.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:06 smithi117 bash[21480]: cluster 2024-03-04T16:06:05.915292+0000 mon.a (mon.0) 3232 : cluster 0 osdmap e636: 8 total, 8 up, 8 in 2024-03-04T16:06:06.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:06 smithi117 bash[21480]: audit 2024-03-04T16:06:05.922569+0000 mon.a (mon.0) 3233 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:06:06.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:06 smithi117 bash[28843]: cluster 2024-03-04T16:06:04.913012+0000 mon.a (mon.0) 3231 : cluster 0 osdmap e635: 8 total, 8 up, 8 in 2024-03-04T16:06:06.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:06 smithi117 bash[28843]: cluster 2024-03-04T16:06:05.169244+0000 mgr.y (mgr.24428) 631 : cluster 0 pgmap v1029: 172 pgs: 172 active+clean; 582 KiB data, 696 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:06.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:06 smithi117 bash[28843]: cluster 2024-03-04T16:06:05.915292+0000 mon.a (mon.0) 3232 : cluster 0 osdmap e636: 8 total, 8 up, 8 in 2024-03-04T16:06:06.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:06 smithi117 bash[28843]: audit 2024-03-04T16:06:05.922569+0000 mon.a (mon.0) 3233 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:06:07.432 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:06:07 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:06:08.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:07 smithi151 bash[23756]: audit 2024-03-04T16:06:06.910802+0000 mon.a (mon.0) 3234 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:08.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:07 smithi151 bash[23756]: cluster 2024-03-04T16:06:06.919542+0000 mon.a (mon.0) 3235 : cluster 0 osdmap e637: 8 total, 8 up, 8 in 2024-03-04T16:06:08.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:07 smithi151 bash[23756]: audit 2024-03-04T16:06:07.128149+0000 mgr.y (mgr.24428) 632 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:06:08.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:07 smithi151 bash[23756]: cluster 2024-03-04T16:06:07.170130+0000 mgr.y (mgr.24428) 633 : cluster 0 pgmap v1032: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 715 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:08.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:07 smithi117 bash[21480]: audit 2024-03-04T16:06:06.910802+0000 mon.a (mon.0) 3234 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:08.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:07 smithi117 bash[21480]: cluster 2024-03-04T16:06:06.919542+0000 mon.a (mon.0) 3235 : cluster 0 osdmap e637: 8 total, 8 up, 8 in 2024-03-04T16:06:08.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:07 smithi117 bash[21480]: audit 2024-03-04T16:06:07.128149+0000 mgr.y (mgr.24428) 632 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:06:08.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:07 smithi117 bash[21480]: cluster 2024-03-04T16:06:07.170130+0000 mgr.y (mgr.24428) 633 : cluster 0 pgmap v1032: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 715 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:08.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:07 smithi117 bash[28843]: audit 2024-03-04T16:06:06.910802+0000 mon.a (mon.0) 3234 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:08.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:07 smithi117 bash[28843]: cluster 2024-03-04T16:06:06.919542+0000 mon.a (mon.0) 3235 : cluster 0 osdmap e637: 8 total, 8 up, 8 in 2024-03-04T16:06:08.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:07 smithi117 bash[28843]: audit 2024-03-04T16:06:07.128149+0000 mgr.y (mgr.24428) 632 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:06:08.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:07 smithi117 bash[28843]: cluster 2024-03-04T16:06:07.170130+0000 mgr.y (mgr.24428) 633 : cluster 0 pgmap v1032: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 715 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:09.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:08 smithi117 bash[28843]: cluster 2024-03-04T16:06:07.932201+0000 mon.a (mon.0) 3236 : cluster 0 osdmap e638: 8 total, 8 up, 8 in 2024-03-04T16:06:09.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:08 smithi117 bash[28843]: audit 2024-03-04T16:06:07.967201+0000 mon.a (mon.0) 3237 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:06:09.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:08 smithi117 bash[21480]: cluster 2024-03-04T16:06:07.932201+0000 mon.a (mon.0) 3236 : cluster 0 osdmap e638: 8 total, 8 up, 8 in 2024-03-04T16:06:09.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:08 smithi117 bash[21480]: audit 2024-03-04T16:06:07.967201+0000 mon.a (mon.0) 3237 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:06:09.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:08 smithi151 bash[23756]: cluster 2024-03-04T16:06:07.932201+0000 mon.a (mon.0) 3236 : cluster 0 osdmap e638: 8 total, 8 up, 8 in 2024-03-04T16:06:09.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:08 smithi151 bash[23756]: audit 2024-03-04T16:06:07.967201+0000 mon.a (mon.0) 3237 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:06:10.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:09 smithi117 bash[21480]: audit 2024-03-04T16:06:08.922673+0000 mon.a (mon.0) 3238 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-124", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:06:10.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:09 smithi117 bash[21480]: cluster 2024-03-04T16:06:08.931992+0000 mon.a (mon.0) 3239 : cluster 0 osdmap e639: 8 total, 8 up, 8 in 2024-03-04T16:06:10.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:09 smithi117 bash[21480]: audit 2024-03-04T16:06:08.933037+0000 mon.a (mon.0) 3240 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-124"}]: dispatch 2024-03-04T16:06:10.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:09 smithi117 bash[21480]: cluster 2024-03-04T16:06:09.170943+0000 mgr.y (mgr.24428) 634 : cluster 0 pgmap v1035: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 715 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:10.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:09 smithi117 bash[21480]: audit 2024-03-04T16:06:09.925394+0000 mon.a (mon.0) 3241 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-124"}]': finished 2024-03-04T16:06:10.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:09 smithi117 bash[21480]: cluster 2024-03-04T16:06:09.931644+0000 mon.a (mon.0) 3242 : cluster 0 osdmap e640: 8 total, 8 up, 8 in 2024-03-04T16:06:10.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:09 smithi117 bash[21480]: audit 2024-03-04T16:06:09.933812+0000 mon.a (mon.0) 3243 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-124", "mode": "writeback"}]: dispatch 2024-03-04T16:06:10.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:09 smithi117 bash[28843]: audit 2024-03-04T16:06:08.922673+0000 mon.a (mon.0) 3238 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-124", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:06:10.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:09 smithi117 bash[28843]: cluster 2024-03-04T16:06:08.931992+0000 mon.a (mon.0) 3239 : cluster 0 osdmap e639: 8 total, 8 up, 8 in 2024-03-04T16:06:10.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:09 smithi117 bash[28843]: audit 2024-03-04T16:06:08.933037+0000 mon.a (mon.0) 3240 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-124"}]: dispatch 2024-03-04T16:06:10.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:09 smithi117 bash[28843]: cluster 2024-03-04T16:06:09.170943+0000 mgr.y (mgr.24428) 634 : cluster 0 pgmap v1035: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 715 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:10.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:09 smithi117 bash[28843]: audit 2024-03-04T16:06:09.925394+0000 mon.a (mon.0) 3241 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-124"}]': finished 2024-03-04T16:06:10.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:09 smithi117 bash[28843]: cluster 2024-03-04T16:06:09.931644+0000 mon.a (mon.0) 3242 : cluster 0 osdmap e640: 8 total, 8 up, 8 in 2024-03-04T16:06:10.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:09 smithi117 bash[28843]: audit 2024-03-04T16:06:09.933812+0000 mon.a (mon.0) 3243 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-124", "mode": "writeback"}]: dispatch 2024-03-04T16:06:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:09 smithi151 bash[23756]: audit 2024-03-04T16:06:08.922673+0000 mon.a (mon.0) 3238 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-124", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:06:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:09 smithi151 bash[23756]: cluster 2024-03-04T16:06:08.931992+0000 mon.a (mon.0) 3239 : cluster 0 osdmap e639: 8 total, 8 up, 8 in 2024-03-04T16:06:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:09 smithi151 bash[23756]: audit 2024-03-04T16:06:08.933037+0000 mon.a (mon.0) 3240 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-124"}]: dispatch 2024-03-04T16:06:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:09 smithi151 bash[23756]: cluster 2024-03-04T16:06:09.170943+0000 mgr.y (mgr.24428) 634 : cluster 0 pgmap v1035: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 715 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:09 smithi151 bash[23756]: audit 2024-03-04T16:06:09.925394+0000 mon.a (mon.0) 3241 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-124"}]': finished 2024-03-04T16:06:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:09 smithi151 bash[23756]: cluster 2024-03-04T16:06:09.931644+0000 mon.a (mon.0) 3242 : cluster 0 osdmap e640: 8 total, 8 up, 8 in 2024-03-04T16:06:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:09 smithi151 bash[23756]: audit 2024-03-04T16:06:09.933812+0000 mon.a (mon.0) 3243 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-124", "mode": "writeback"}]: dispatch 2024-03-04T16:06:11.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:10 smithi117 bash[28843]: cluster 2024-03-04T16:06:10.925794+0000 mon.a (mon.0) 3244 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:06:11.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:10 smithi117 bash[28843]: audit 2024-03-04T16:06:10.928399+0000 mon.a (mon.0) 3245 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-124", "mode": "writeback"}]': finished 2024-03-04T16:06:11.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:10 smithi117 bash[28843]: cluster 2024-03-04T16:06:10.937697+0000 mon.a (mon.0) 3246 : cluster 0 osdmap e641: 8 total, 8 up, 8 in 2024-03-04T16:06:11.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:10 smithi117 bash[21480]: cluster 2024-03-04T16:06:10.925794+0000 mon.a (mon.0) 3244 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:06:11.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:10 smithi117 bash[21480]: audit 2024-03-04T16:06:10.928399+0000 mon.a (mon.0) 3245 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-124", "mode": "writeback"}]': finished 2024-03-04T16:06:11.320 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:10 smithi117 bash[21480]: cluster 2024-03-04T16:06:10.937697+0000 mon.a (mon.0) 3246 : cluster 0 osdmap e641: 8 total, 8 up, 8 in 2024-03-04T16:06:11.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:10 smithi151 bash[23756]: cluster 2024-03-04T16:06:10.925794+0000 mon.a (mon.0) 3244 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:06:11.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:10 smithi151 bash[23756]: audit 2024-03-04T16:06:10.928399+0000 mon.a (mon.0) 3245 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-124", "mode": "writeback"}]': finished 2024-03-04T16:06:11.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:10 smithi151 bash[23756]: cluster 2024-03-04T16:06:10.937697+0000 mon.a (mon.0) 3246 : cluster 0 osdmap e641: 8 total, 8 up, 8 in 2024-03-04T16:06:12.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:11 smithi117 bash[21480]: cluster 2024-03-04T16:06:11.171783+0000 mgr.y (mgr.24428) 635 : cluster 0 pgmap v1038: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 715 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:06:12.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:11 smithi117 bash[28843]: cluster 2024-03-04T16:06:11.171783+0000 mgr.y (mgr.24428) 635 : cluster 0 pgmap v1038: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 715 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:06:12.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:11 smithi151 bash[23756]: cluster 2024-03-04T16:06:11.171783+0000 mgr.y (mgr.24428) 635 : cluster 0 pgmap v1038: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 715 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:06:13.316 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:12 smithi117 bash[21480]: cluster 2024-03-04T16:06:11.968048+0000 mon.a (mon.0) 3247 : cluster 0 osdmap e642: 8 total, 8 up, 8 in 2024-03-04T16:06:13.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:12 smithi117 bash[21480]: audit 2024-03-04T16:06:12.033654+0000 mon.a (mon.0) 3248 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:13.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:12 smithi117 bash[28843]: cluster 2024-03-04T16:06:11.968048+0000 mon.a (mon.0) 3247 : cluster 0 osdmap e642: 8 total, 8 up, 8 in 2024-03-04T16:06:13.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:12 smithi117 bash[28843]: audit 2024-03-04T16:06:12.033654+0000 mon.a (mon.0) 3248 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:13.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:12 smithi151 bash[23756]: cluster 2024-03-04T16:06:11.968048+0000 mon.a (mon.0) 3247 : cluster 0 osdmap e642: 8 total, 8 up, 8 in 2024-03-04T16:06:13.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:12 smithi151 bash[23756]: audit 2024-03-04T16:06:12.033654+0000 mon.a (mon.0) 3248 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:14.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:13 smithi117 bash[21480]: audit 2024-03-04T16:06:12.963533+0000 mon.a (mon.0) 3249 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:14.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:13 smithi117 bash[21480]: cluster 2024-03-04T16:06:12.970694+0000 mon.a (mon.0) 3250 : cluster 0 osdmap e643: 8 total, 8 up, 8 in 2024-03-04T16:06:14.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:13 smithi117 bash[21480]: audit 2024-03-04T16:06:12.972106+0000 mon.a (mon.0) 3251 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-124"}]: dispatch 2024-03-04T16:06:14.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:13 smithi117 bash[21480]: cluster 2024-03-04T16:06:13.172602+0000 mgr.y (mgr.24428) 636 : cluster 0 pgmap v1041: 204 pgs: 204 active+clean; 582 KiB data, 715 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-04T16:06:14.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:13 smithi117 bash[21480]: cluster 2024-03-04T16:06:13.963935+0000 mon.a (mon.0) 3252 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:06:14.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:13 smithi117 bash[21480]: audit 2024-03-04T16:06:13.966428+0000 mon.a (mon.0) 3253 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-124"}]': finished 2024-03-04T16:06:14.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:13 smithi117 bash[21480]: cluster 2024-03-04T16:06:13.972628+0000 mon.a (mon.0) 3254 : cluster 0 osdmap e644: 8 total, 8 up, 8 in 2024-03-04T16:06:14.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:13 smithi117 bash[28843]: audit 2024-03-04T16:06:12.963533+0000 mon.a (mon.0) 3249 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:14.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:13 smithi117 bash[28843]: cluster 2024-03-04T16:06:12.970694+0000 mon.a (mon.0) 3250 : cluster 0 osdmap e643: 8 total, 8 up, 8 in 2024-03-04T16:06:14.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:13 smithi117 bash[28843]: audit 2024-03-04T16:06:12.972106+0000 mon.a (mon.0) 3251 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-124"}]: dispatch 2024-03-04T16:06:14.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:13 smithi117 bash[28843]: cluster 2024-03-04T16:06:13.172602+0000 mgr.y (mgr.24428) 636 : cluster 0 pgmap v1041: 204 pgs: 204 active+clean; 582 KiB data, 715 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-04T16:06:14.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:13 smithi117 bash[28843]: cluster 2024-03-04T16:06:13.963935+0000 mon.a (mon.0) 3252 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:06:14.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:13 smithi117 bash[28843]: audit 2024-03-04T16:06:13.966428+0000 mon.a (mon.0) 3253 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-124"}]': finished 2024-03-04T16:06:14.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:13 smithi117 bash[28843]: cluster 2024-03-04T16:06:13.972628+0000 mon.a (mon.0) 3254 : cluster 0 osdmap e644: 8 total, 8 up, 8 in 2024-03-04T16:06:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:13 smithi151 bash[23756]: audit 2024-03-04T16:06:12.963533+0000 mon.a (mon.0) 3249 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:13 smithi151 bash[23756]: cluster 2024-03-04T16:06:12.970694+0000 mon.a (mon.0) 3250 : cluster 0 osdmap e643: 8 total, 8 up, 8 in 2024-03-04T16:06:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:13 smithi151 bash[23756]: audit 2024-03-04T16:06:12.972106+0000 mon.a (mon.0) 3251 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-124"}]: dispatch 2024-03-04T16:06:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:13 smithi151 bash[23756]: cluster 2024-03-04T16:06:13.172602+0000 mgr.y (mgr.24428) 636 : cluster 0 pgmap v1041: 204 pgs: 204 active+clean; 582 KiB data, 715 MiB used, 715 GiB / 715 GiB avail; 3.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-03-04T16:06:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:13 smithi151 bash[23756]: cluster 2024-03-04T16:06:13.963935+0000 mon.a (mon.0) 3252 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:06:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:13 smithi151 bash[23756]: audit 2024-03-04T16:06:13.966428+0000 mon.a (mon.0) 3253 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-124"}]': finished 2024-03-04T16:06:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:13 smithi151 bash[23756]: cluster 2024-03-04T16:06:13.972628+0000 mon.a (mon.0) 3254 : cluster 0 osdmap e644: 8 total, 8 up, 8 in 2024-03-04T16:06:15.816 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:15 smithi117 bash[21480]: audit 2024-03-04T16:06:14.982998+0000 mon.c (mon.2) 805 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:06:15.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:15 smithi117 bash[28843]: audit 2024-03-04T16:06:14.982998+0000 mon.c (mon.2) 805 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:06:15.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:15 smithi151 bash[23756]: audit 2024-03-04T16:06:14.982998+0000 mon.c (mon.2) 805 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:06:16.470 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:06:16 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:06:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:06:16.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:16 smithi117 bash[21480]: cluster 2024-03-04T16:06:15.004450+0000 mon.a (mon.0) 3255 : cluster 0 osdmap e645: 8 total, 8 up, 8 in 2024-03-04T16:06:16.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:16 smithi117 bash[21480]: cluster 2024-03-04T16:06:15.173379+0000 mgr.y (mgr.24428) 637 : cluster 0 pgmap v1044: 172 pgs: 172 active+clean; 582 KiB data, 715 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-04T16:06:16.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:16 smithi117 bash[21480]: cluster 2024-03-04T16:06:16.006532+0000 mon.a (mon.0) 3256 : cluster 0 osdmap e646: 8 total, 8 up, 8 in 2024-03-04T16:06:16.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:16 smithi117 bash[21480]: audit 2024-03-04T16:06:16.014108+0000 mon.a (mon.0) 3257 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:06:16.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:16 smithi117 bash[28843]: cluster 2024-03-04T16:06:15.004450+0000 mon.a (mon.0) 3255 : cluster 0 osdmap e645: 8 total, 8 up, 8 in 2024-03-04T16:06:16.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:16 smithi117 bash[28843]: cluster 2024-03-04T16:06:15.173379+0000 mgr.y (mgr.24428) 637 : cluster 0 pgmap v1044: 172 pgs: 172 active+clean; 582 KiB data, 715 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-04T16:06:16.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:16 smithi117 bash[28843]: cluster 2024-03-04T16:06:16.006532+0000 mon.a (mon.0) 3256 : cluster 0 osdmap e646: 8 total, 8 up, 8 in 2024-03-04T16:06:16.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:16 smithi117 bash[28843]: audit 2024-03-04T16:06:16.014108+0000 mon.a (mon.0) 3257 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:06:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:16 smithi151 bash[23756]: cluster 2024-03-04T16:06:15.004450+0000 mon.a (mon.0) 3255 : cluster 0 osdmap e645: 8 total, 8 up, 8 in 2024-03-04T16:06:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:16 smithi151 bash[23756]: cluster 2024-03-04T16:06:15.173379+0000 mgr.y (mgr.24428) 637 : cluster 0 pgmap v1044: 172 pgs: 172 active+clean; 582 KiB data, 715 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-03-04T16:06:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:16 smithi151 bash[23756]: cluster 2024-03-04T16:06:16.006532+0000 mon.a (mon.0) 3256 : cluster 0 osdmap e646: 8 total, 8 up, 8 in 2024-03-04T16:06:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:16 smithi151 bash[23756]: audit 2024-03-04T16:06:16.014108+0000 mon.a (mon.0) 3257 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:06:17.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:06:17 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:06:18.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:18 smithi117 bash[21480]: audit 2024-03-04T16:06:17.002967+0000 mon.a (mon.0) 3258 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:18.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:18 smithi117 bash[21480]: cluster 2024-03-04T16:06:17.012730+0000 mon.a (mon.0) 3259 : cluster 0 osdmap e647: 8 total, 8 up, 8 in 2024-03-04T16:06:18.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:18 smithi117 bash[21480]: audit 2024-03-04T16:06:17.015244+0000 mon.a (mon.0) 3260 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:06:18.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:18 smithi117 bash[21480]: audit 2024-03-04T16:06:17.136970+0000 mgr.y (mgr.24428) 638 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:06:18.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:18 smithi117 bash[21480]: cluster 2024-03-04T16:06:17.174142+0000 mgr.y (mgr.24428) 639 : cluster 0 pgmap v1047: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 716 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:06:18.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:18 smithi117 bash[28843]: audit 2024-03-04T16:06:17.002967+0000 mon.a (mon.0) 3258 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:18.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:18 smithi117 bash[28843]: cluster 2024-03-04T16:06:17.012730+0000 mon.a (mon.0) 3259 : cluster 0 osdmap e647: 8 total, 8 up, 8 in 2024-03-04T16:06:18.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:18 smithi117 bash[28843]: audit 2024-03-04T16:06:17.015244+0000 mon.a (mon.0) 3260 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:06:18.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:18 smithi117 bash[28843]: audit 2024-03-04T16:06:17.136970+0000 mgr.y (mgr.24428) 638 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:06:18.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:18 smithi117 bash[28843]: cluster 2024-03-04T16:06:17.174142+0000 mgr.y (mgr.24428) 639 : cluster 0 pgmap v1047: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 716 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:06:18.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:18 smithi151 bash[23756]: audit 2024-03-04T16:06:17.002967+0000 mon.a (mon.0) 3258 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:18.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:18 smithi151 bash[23756]: cluster 2024-03-04T16:06:17.012730+0000 mon.a (mon.0) 3259 : cluster 0 osdmap e647: 8 total, 8 up, 8 in 2024-03-04T16:06:18.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:18 smithi151 bash[23756]: audit 2024-03-04T16:06:17.015244+0000 mon.a (mon.0) 3260 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:06:18.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:18 smithi151 bash[23756]: audit 2024-03-04T16:06:17.136970+0000 mgr.y (mgr.24428) 638 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:06:18.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:18 smithi151 bash[23756]: cluster 2024-03-04T16:06:17.174142+0000 mgr.y (mgr.24428) 639 : cluster 0 pgmap v1047: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 716 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:06:19.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:19 smithi117 bash[21480]: audit 2024-03-04T16:06:18.014315+0000 mon.a (mon.0) 3261 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-126", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:06:19.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:19 smithi117 bash[21480]: cluster 2024-03-04T16:06:18.024185+0000 mon.a (mon.0) 3262 : cluster 0 osdmap e648: 8 total, 8 up, 8 in 2024-03-04T16:06:19.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:19 smithi117 bash[21480]: audit 2024-03-04T16:06:18.028168+0000 mon.a (mon.0) 3263 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-126"}]: dispatch 2024-03-04T16:06:19.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:19 smithi117 bash[28843]: audit 2024-03-04T16:06:18.014315+0000 mon.a (mon.0) 3261 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-126", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:06:19.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:19 smithi117 bash[28843]: cluster 2024-03-04T16:06:18.024185+0000 mon.a (mon.0) 3262 : cluster 0 osdmap e648: 8 total, 8 up, 8 in 2024-03-04T16:06:19.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:19 smithi117 bash[28843]: audit 2024-03-04T16:06:18.028168+0000 mon.a (mon.0) 3263 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-126"}]: dispatch 2024-03-04T16:06:19.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:19 smithi151 bash[23756]: audit 2024-03-04T16:06:18.014315+0000 mon.a (mon.0) 3261 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-126", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:06:19.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:19 smithi151 bash[23756]: cluster 2024-03-04T16:06:18.024185+0000 mon.a (mon.0) 3262 : cluster 0 osdmap e648: 8 total, 8 up, 8 in 2024-03-04T16:06:19.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:19 smithi151 bash[23756]: audit 2024-03-04T16:06:18.028168+0000 mon.a (mon.0) 3263 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-126"}]: dispatch 2024-03-04T16:06:20.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:20 smithi117 bash[21480]: audit 2024-03-04T16:06:19.017080+0000 mon.a (mon.0) 3264 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-126"}]': finished 2024-03-04T16:06:20.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:20 smithi117 bash[21480]: cluster 2024-03-04T16:06:19.024908+0000 mon.a (mon.0) 3265 : cluster 0 osdmap e649: 8 total, 8 up, 8 in 2024-03-04T16:06:20.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:20 smithi117 bash[21480]: audit 2024-03-04T16:06:19.026364+0000 mon.a (mon.0) 3266 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-126", "mode": "writeback"}]: dispatch 2024-03-04T16:06:20.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:20 smithi117 bash[21480]: cluster 2024-03-04T16:06:19.174961+0000 mgr.y (mgr.24428) 640 : cluster 0 pgmap v1050: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:06:20.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:20 smithi117 bash[21480]: cluster 2024-03-04T16:06:19.767733+0000 mon.a (mon.0) 3267 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:06:20.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:20 smithi117 bash[21480]: audit 2024-03-04T16:06:19.770446+0000 mon.a (mon.0) 3268 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-126", "mode": "writeback"}]': finished 2024-03-04T16:06:20.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:20 smithi117 bash[21480]: cluster 2024-03-04T16:06:19.777800+0000 mon.a (mon.0) 3269 : cluster 0 osdmap e650: 8 total, 8 up, 8 in 2024-03-04T16:06:20.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:20 smithi117 bash[21480]: audit 2024-03-04T16:06:19.969825+0000 mon.a (mon.0) 3270 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:20.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:20 smithi117 bash[28843]: audit 2024-03-04T16:06:19.017080+0000 mon.a (mon.0) 3264 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-126"}]': finished 2024-03-04T16:06:20.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:20 smithi117 bash[28843]: cluster 2024-03-04T16:06:19.024908+0000 mon.a (mon.0) 3265 : cluster 0 osdmap e649: 8 total, 8 up, 8 in 2024-03-04T16:06:20.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:20 smithi117 bash[28843]: audit 2024-03-04T16:06:19.026364+0000 mon.a (mon.0) 3266 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-126", "mode": "writeback"}]: dispatch 2024-03-04T16:06:20.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:20 smithi117 bash[28843]: cluster 2024-03-04T16:06:19.174961+0000 mgr.y (mgr.24428) 640 : cluster 0 pgmap v1050: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:06:20.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:20 smithi117 bash[28843]: cluster 2024-03-04T16:06:19.767733+0000 mon.a (mon.0) 3267 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:06:20.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:20 smithi117 bash[28843]: audit 2024-03-04T16:06:19.770446+0000 mon.a (mon.0) 3268 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-126", "mode": "writeback"}]': finished 2024-03-04T16:06:20.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:20 smithi117 bash[28843]: cluster 2024-03-04T16:06:19.777800+0000 mon.a (mon.0) 3269 : cluster 0 osdmap e650: 8 total, 8 up, 8 in 2024-03-04T16:06:20.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:20 smithi117 bash[28843]: audit 2024-03-04T16:06:19.969825+0000 mon.a (mon.0) 3270 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:20.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:20 smithi151 bash[23756]: audit 2024-03-04T16:06:19.017080+0000 mon.a (mon.0) 3264 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-126"}]': finished 2024-03-04T16:06:20.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:20 smithi151 bash[23756]: cluster 2024-03-04T16:06:19.024908+0000 mon.a (mon.0) 3265 : cluster 0 osdmap e649: 8 total, 8 up, 8 in 2024-03-04T16:06:20.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:20 smithi151 bash[23756]: audit 2024-03-04T16:06:19.026364+0000 mon.a (mon.0) 3266 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-126", "mode": "writeback"}]: dispatch 2024-03-04T16:06:20.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:20 smithi151 bash[23756]: cluster 2024-03-04T16:06:19.174961+0000 mgr.y (mgr.24428) 640 : cluster 0 pgmap v1050: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 716 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:06:20.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:20 smithi151 bash[23756]: cluster 2024-03-04T16:06:19.767733+0000 mon.a (mon.0) 3267 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:06:20.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:20 smithi151 bash[23756]: audit 2024-03-04T16:06:19.770446+0000 mon.a (mon.0) 3268 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-126", "mode": "writeback"}]': finished 2024-03-04T16:06:20.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:20 smithi151 bash[23756]: cluster 2024-03-04T16:06:19.777800+0000 mon.a (mon.0) 3269 : cluster 0 osdmap e650: 8 total, 8 up, 8 in 2024-03-04T16:06:20.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:20 smithi151 bash[23756]: audit 2024-03-04T16:06:19.969825+0000 mon.a (mon.0) 3270 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:22.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:21 smithi117 bash[28843]: audit 2024-03-04T16:06:20.792077+0000 mon.a (mon.0) 3271 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:22.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:21 smithi117 bash[28843]: cluster 2024-03-04T16:06:20.799712+0000 mon.a (mon.0) 3272 : cluster 0 osdmap e651: 8 total, 8 up, 8 in 2024-03-04T16:06:22.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:21 smithi117 bash[28843]: audit 2024-03-04T16:06:20.801331+0000 mon.a (mon.0) 3273 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-126"}]: dispatch 2024-03-04T16:06:22.067 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:21 smithi117 bash[28843]: cluster 2024-03-04T16:06:21.175826+0000 mgr.y (mgr.24428) 641 : cluster 0 pgmap v1053: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 716 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:06:22.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:21 smithi117 bash[21480]: audit 2024-03-04T16:06:20.792077+0000 mon.a (mon.0) 3271 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:22.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:21 smithi117 bash[21480]: cluster 2024-03-04T16:06:20.799712+0000 mon.a (mon.0) 3272 : cluster 0 osdmap e651: 8 total, 8 up, 8 in 2024-03-04T16:06:22.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:21 smithi117 bash[21480]: audit 2024-03-04T16:06:20.801331+0000 mon.a (mon.0) 3273 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-126"}]: dispatch 2024-03-04T16:06:22.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:21 smithi117 bash[21480]: cluster 2024-03-04T16:06:21.175826+0000 mgr.y (mgr.24428) 641 : cluster 0 pgmap v1053: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 716 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:06:22.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:21 smithi151 bash[23756]: audit 2024-03-04T16:06:20.792077+0000 mon.a (mon.0) 3271 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:22.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:21 smithi151 bash[23756]: cluster 2024-03-04T16:06:20.799712+0000 mon.a (mon.0) 3272 : cluster 0 osdmap e651: 8 total, 8 up, 8 in 2024-03-04T16:06:22.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:21 smithi151 bash[23756]: audit 2024-03-04T16:06:20.801331+0000 mon.a (mon.0) 3273 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-126"}]: dispatch 2024-03-04T16:06:22.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:21 smithi151 bash[23756]: cluster 2024-03-04T16:06:21.175826+0000 mgr.y (mgr.24428) 641 : cluster 0 pgmap v1053: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 716 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:06:22.823 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:06:22.823 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9086 ms) 2024-03-04T16:06:22.823 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-03-04T16:06:22.823 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:06:22.823 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6069 ms) 2024-03-04T16:06:22.823 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-03-04T16:06:22.823 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:06:22.823 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5068 ms) 2024-03-04T16:06:22.823 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-03-04T16:06:22.823 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:06:22.823 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (25458 ms) 2024-03-04T16:06:22.823 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-03-04T16:06:22.824 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:06:22.824 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (14102 ms) 2024-03-04T16:06:22.824 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-03-04T16:06:22.824 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12146 ms) 2024-03-04T16:06:22.824 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-03-04T16:06:22.824 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-03-04T16:06:22.824 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: 00000010 69 73 |is| 2024-03-04T16:06:22.824 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: 00000012 2024-03-04T16:06:22.824 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12078 ms) 2024-03-04T16:06:22.824 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-03-04T16:06:22.824 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: require_osd_release = squid 2024-03-04T16:06:22.824 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6023 ms) 2024-03-04T16:06:22.824 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (576280 ms total) 2024-03-04T16:06:22.824 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: 2024-03-04T16:06:22.824 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-03-04T16:06:22.825 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-03-04T16:06:22.825 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1024 ms) 2024-03-04T16:06:22.825 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-03-04T16:06:22.825 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (11069 ms) 2024-03-04T16:06:22.825 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-03-04T16:06:22.825 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18211 ms) 2024-03-04T16:06:22.825 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-03-04T16:06:22.825 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (3 ms) 2024-03-04T16:06:22.825 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30308 ms total) 2024-03-04T16:06:22.825 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: 2024-03-04T16:06:22.825 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-03-04T16:06:22.825 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-03-04T16:06:22.825 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7064 ms) 2024-03-04T16:06:22.825 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-03-04T16:06:22.825 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8127 ms) 2024-03-04T16:06:22.825 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-03-04T16:06:22.826 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: waiting for scrub... 2024-03-04T16:06:22.826 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: done waiting 2024-03-04T16:06:22.826 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24198 ms) 2024-03-04T16:06:22.826 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-03-04T16:06:22.826 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9550 ms) 2024-03-04T16:06:22.826 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-03-04T16:06:22.826 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (7994 ms) 2024-03-04T16:06:22.826 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-03-04T16:06:22.826 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (8074 ms) 2024-03-04T16:06:22.826 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-03-04T16:06:22.826 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10099 ms) 2024-03-04T16:06:22.826 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-03-04T16:06:23.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:22 smithi117 bash[21480]: cluster 2024-03-04T16:06:21.793018+0000 mon.a (mon.0) 3274 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:06:23.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:22 smithi117 bash[21480]: audit 2024-03-04T16:06:21.801371+0000 mon.a (mon.0) 3275 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-126"}]': finished 2024-03-04T16:06:23.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:22 smithi117 bash[21480]: cluster 2024-03-04T16:06:21.810794+0000 mon.a (mon.0) 3276 : cluster 0 osdmap e652: 8 total, 8 up, 8 in 2024-03-04T16:06:23.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:22 smithi117 bash[28843]: cluster 2024-03-04T16:06:21.793018+0000 mon.a (mon.0) 3274 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:06:23.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:22 smithi117 bash[28843]: audit 2024-03-04T16:06:21.801371+0000 mon.a (mon.0) 3275 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-126"}]': finished 2024-03-04T16:06:23.070 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:22 smithi117 bash[28843]: cluster 2024-03-04T16:06:21.810794+0000 mon.a (mon.0) 3276 : cluster 0 osdmap e652: 8 total, 8 up, 8 in 2024-03-04T16:06:23.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:22 smithi151 bash[23756]: cluster 2024-03-04T16:06:21.793018+0000 mon.a (mon.0) 3274 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:06:23.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:22 smithi151 bash[23756]: audit 2024-03-04T16:06:21.801371+0000 mon.a (mon.0) 3275 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-126"}]': finished 2024-03-04T16:06:23.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:22 smithi151 bash[23756]: cluster 2024-03-04T16:06:21.810794+0000 mon.a (mon.0) 3276 : cluster 0 osdmap e652: 8 total, 8 up, 8 in 2024-03-04T16:06:24.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:23 smithi151 bash[23756]: cluster 2024-03-04T16:06:22.818495+0000 mon.a (mon.0) 3277 : cluster 0 osdmap e653: 8 total, 8 up, 8 in 2024-03-04T16:06:24.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:23 smithi151 bash[23756]: cluster 2024-03-04T16:06:23.176574+0000 mgr.y (mgr.24428) 642 : cluster 0 pgmap v1056: 172 pgs: 172 active+clean; 582 KiB data, 717 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:24.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:23 smithi117 bash[21480]: cluster 2024-03-04T16:06:22.818495+0000 mon.a (mon.0) 3277 : cluster 0 osdmap e653: 8 total, 8 up, 8 in 2024-03-04T16:06:24.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:23 smithi117 bash[21480]: cluster 2024-03-04T16:06:23.176574+0000 mgr.y (mgr.24428) 642 : cluster 0 pgmap v1056: 172 pgs: 172 active+clean; 582 KiB data, 717 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:24.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:23 smithi117 bash[28843]: cluster 2024-03-04T16:06:22.818495+0000 mon.a (mon.0) 3277 : cluster 0 osdmap e653: 8 total, 8 up, 8 in 2024-03-04T16:06:24.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:23 smithi117 bash[28843]: cluster 2024-03-04T16:06:23.176574+0000 mgr.y (mgr.24428) 642 : cluster 0 pgmap v1056: 172 pgs: 172 active+clean; 582 KiB data, 717 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:25.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:24 smithi151 bash[23756]: cluster 2024-03-04T16:06:23.835718+0000 mon.a (mon.0) 3278 : cluster 0 osdmap e654: 8 total, 8 up, 8 in 2024-03-04T16:06:25.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:24 smithi151 bash[23756]: audit 2024-03-04T16:06:23.837960+0000 mon.a (mon.0) 3279 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:06:25.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:24 smithi117 bash[21480]: cluster 2024-03-04T16:06:23.835718+0000 mon.a (mon.0) 3278 : cluster 0 osdmap e654: 8 total, 8 up, 8 in 2024-03-04T16:06:25.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:24 smithi117 bash[21480]: audit 2024-03-04T16:06:23.837960+0000 mon.a (mon.0) 3279 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:06:25.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:24 smithi117 bash[28843]: cluster 2024-03-04T16:06:23.835718+0000 mon.a (mon.0) 3278 : cluster 0 osdmap e654: 8 total, 8 up, 8 in 2024-03-04T16:06:25.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:24 smithi117 bash[28843]: audit 2024-03-04T16:06:23.837960+0000 mon.a (mon.0) 3279 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:06:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:25 smithi151 bash[23756]: audit 2024-03-04T16:06:24.827749+0000 mon.a (mon.0) 3280 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:25 smithi151 bash[23756]: cluster 2024-03-04T16:06:24.835875+0000 mon.a (mon.0) 3281 : cluster 0 osdmap e655: 8 total, 8 up, 8 in 2024-03-04T16:06:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:25 smithi151 bash[23756]: audit 2024-03-04T16:06:24.839578+0000 mon.a (mon.0) 3282 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:06:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:25 smithi151 bash[23756]: cluster 2024-03-04T16:06:25.177409+0000 mgr.y (mgr.24428) 643 : cluster 0 pgmap v1059: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 717 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:25 smithi151 bash[23756]: audit 2024-03-04T16:06:25.830428+0000 mon.a (mon.0) 3283 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-128", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:06:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:25 smithi151 bash[23756]: cluster 2024-03-04T16:06:25.838195+0000 mon.a (mon.0) 3284 : cluster 0 osdmap e656: 8 total, 8 up, 8 in 2024-03-04T16:06:26.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:25 smithi151 bash[23756]: audit 2024-03-04T16:06:25.840046+0000 mon.a (mon.0) 3285 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-128"}]: dispatch 2024-03-04T16:06:26.316 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:06:26 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:06:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:06:26.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:25 smithi117 bash[21480]: audit 2024-03-04T16:06:24.827749+0000 mon.a (mon.0) 3280 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:26.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:25 smithi117 bash[21480]: cluster 2024-03-04T16:06:24.835875+0000 mon.a (mon.0) 3281 : cluster 0 osdmap e655: 8 total, 8 up, 8 in 2024-03-04T16:06:26.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:25 smithi117 bash[21480]: audit 2024-03-04T16:06:24.839578+0000 mon.a (mon.0) 3282 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:06:26.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:25 smithi117 bash[21480]: cluster 2024-03-04T16:06:25.177409+0000 mgr.y (mgr.24428) 643 : cluster 0 pgmap v1059: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 717 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:26.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:25 smithi117 bash[21480]: audit 2024-03-04T16:06:25.830428+0000 mon.a (mon.0) 3283 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-128", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:06:26.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:25 smithi117 bash[21480]: cluster 2024-03-04T16:06:25.838195+0000 mon.a (mon.0) 3284 : cluster 0 osdmap e656: 8 total, 8 up, 8 in 2024-03-04T16:06:26.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:25 smithi117 bash[21480]: audit 2024-03-04T16:06:25.840046+0000 mon.a (mon.0) 3285 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-128"}]: dispatch 2024-03-04T16:06:26.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:25 smithi117 bash[28843]: audit 2024-03-04T16:06:24.827749+0000 mon.a (mon.0) 3280 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:26.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:25 smithi117 bash[28843]: cluster 2024-03-04T16:06:24.835875+0000 mon.a (mon.0) 3281 : cluster 0 osdmap e655: 8 total, 8 up, 8 in 2024-03-04T16:06:26.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:25 smithi117 bash[28843]: audit 2024-03-04T16:06:24.839578+0000 mon.a (mon.0) 3282 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:06:26.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:25 smithi117 bash[28843]: cluster 2024-03-04T16:06:25.177409+0000 mgr.y (mgr.24428) 643 : cluster 0 pgmap v1059: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 717 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:26.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:25 smithi117 bash[28843]: audit 2024-03-04T16:06:25.830428+0000 mon.a (mon.0) 3283 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-128", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:06:26.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:25 smithi117 bash[28843]: cluster 2024-03-04T16:06:25.838195+0000 mon.a (mon.0) 3284 : cluster 0 osdmap e656: 8 total, 8 up, 8 in 2024-03-04T16:06:26.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:25 smithi117 bash[28843]: audit 2024-03-04T16:06:25.840046+0000 mon.a (mon.0) 3285 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-128"}]: dispatch 2024-03-04T16:06:27.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:06:27 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:06:28.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:27 smithi151 bash[23756]: audit 2024-03-04T16:06:26.834697+0000 mon.a (mon.0) 3286 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-128"}]': finished 2024-03-04T16:06:28.267 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:27 smithi151 bash[23756]: cluster 2024-03-04T16:06:26.848197+0000 mon.a (mon.0) 3287 : cluster 0 osdmap e657: 8 total, 8 up, 8 in 2024-03-04T16:06:28.267 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:27 smithi151 bash[23756]: audit 2024-03-04T16:06:26.852914+0000 mon.a (mon.0) 3288 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-128", "mode": "writeback"}]: dispatch 2024-03-04T16:06:28.267 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:27 smithi151 bash[23756]: audit 2024-03-04T16:06:27.148318+0000 mgr.y (mgr.24428) 644 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:06:28.267 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:27 smithi151 bash[23756]: cluster 2024-03-04T16:06:27.178159+0000 mgr.y (mgr.24428) 645 : cluster 0 pgmap v1062: 204 pgs: 204 active+clean; 582 KiB data, 721 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:06:28.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:27 smithi117 bash[21480]: audit 2024-03-04T16:06:26.834697+0000 mon.a (mon.0) 3286 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-128"}]': finished 2024-03-04T16:06:28.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:27 smithi117 bash[21480]: cluster 2024-03-04T16:06:26.848197+0000 mon.a (mon.0) 3287 : cluster 0 osdmap e657: 8 total, 8 up, 8 in 2024-03-04T16:06:28.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:27 smithi117 bash[21480]: audit 2024-03-04T16:06:26.852914+0000 mon.a (mon.0) 3288 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-128", "mode": "writeback"}]: dispatch 2024-03-04T16:06:28.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:27 smithi117 bash[21480]: audit 2024-03-04T16:06:27.148318+0000 mgr.y (mgr.24428) 644 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:06:28.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:27 smithi117 bash[21480]: cluster 2024-03-04T16:06:27.178159+0000 mgr.y (mgr.24428) 645 : cluster 0 pgmap v1062: 204 pgs: 204 active+clean; 582 KiB data, 721 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:06:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:27 smithi117 bash[28843]: audit 2024-03-04T16:06:26.834697+0000 mon.a (mon.0) 3286 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-128"}]': finished 2024-03-04T16:06:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:27 smithi117 bash[28843]: cluster 2024-03-04T16:06:26.848197+0000 mon.a (mon.0) 3287 : cluster 0 osdmap e657: 8 total, 8 up, 8 in 2024-03-04T16:06:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:27 smithi117 bash[28843]: audit 2024-03-04T16:06:26.852914+0000 mon.a (mon.0) 3288 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-128", "mode": "writeback"}]: dispatch 2024-03-04T16:06:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:27 smithi117 bash[28843]: audit 2024-03-04T16:06:27.148318+0000 mgr.y (mgr.24428) 644 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:06:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:27 smithi117 bash[28843]: cluster 2024-03-04T16:06:27.178159+0000 mgr.y (mgr.24428) 645 : cluster 0 pgmap v1062: 204 pgs: 204 active+clean; 582 KiB data, 721 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:06:29.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:28 smithi151 bash[23756]: cluster 2024-03-04T16:06:27.835544+0000 mon.a (mon.0) 3289 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:06:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:28 smithi151 bash[23756]: audit 2024-03-04T16:06:27.844908+0000 mon.a (mon.0) 3290 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-128", "mode": "writeback"}]': finished 2024-03-04T16:06:29.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:28 smithi151 bash[23756]: cluster 2024-03-04T16:06:27.855250+0000 mon.a (mon.0) 3291 : cluster 0 osdmap e658: 8 total, 8 up, 8 in 2024-03-04T16:06:29.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:28 smithi117 bash[21480]: cluster 2024-03-04T16:06:27.835544+0000 mon.a (mon.0) 3289 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:06:29.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:28 smithi117 bash[21480]: audit 2024-03-04T16:06:27.844908+0000 mon.a (mon.0) 3290 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-128", "mode": "writeback"}]': finished 2024-03-04T16:06:29.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:28 smithi117 bash[21480]: cluster 2024-03-04T16:06:27.855250+0000 mon.a (mon.0) 3291 : cluster 0 osdmap e658: 8 total, 8 up, 8 in 2024-03-04T16:06:29.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:28 smithi117 bash[28843]: cluster 2024-03-04T16:06:27.835544+0000 mon.a (mon.0) 3289 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:06:29.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:28 smithi117 bash[28843]: audit 2024-03-04T16:06:27.844908+0000 mon.a (mon.0) 3290 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-128", "mode": "writeback"}]': finished 2024-03-04T16:06:29.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:28 smithi117 bash[28843]: cluster 2024-03-04T16:06:27.855250+0000 mon.a (mon.0) 3291 : cluster 0 osdmap e658: 8 total, 8 up, 8 in 2024-03-04T16:06:30.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:29 smithi151 bash[23756]: cluster 2024-03-04T16:06:29.179012+0000 mgr.y (mgr.24428) 646 : cluster 0 pgmap v1064: 204 pgs: 204 active+clean; 582 KiB data, 721 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-04T16:06:30.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:29 smithi117 bash[21480]: cluster 2024-03-04T16:06:29.179012+0000 mgr.y (mgr.24428) 646 : cluster 0 pgmap v1064: 204 pgs: 204 active+clean; 582 KiB data, 721 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-04T16:06:30.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:29 smithi117 bash[28843]: cluster 2024-03-04T16:06:29.179012+0000 mgr.y (mgr.24428) 646 : cluster 0 pgmap v1064: 204 pgs: 204 active+clean; 582 KiB data, 721 MiB used, 715 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-03-04T16:06:31.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:31 smithi117 bash[21480]: audit 2024-03-04T16:06:30.040127+0000 mon.a (mon.0) 3292 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:06:31.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:31 smithi117 bash[21480]: audit 2024-03-04T16:06:30.043109+0000 mon.c (mon.2) 806 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:06:31.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:31 smithi117 bash[28843]: audit 2024-03-04T16:06:30.040127+0000 mon.a (mon.0) 3292 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:06:31.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:31 smithi117 bash[28843]: audit 2024-03-04T16:06:30.043109+0000 mon.c (mon.2) 806 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:06:31.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:31 smithi151 bash[23756]: audit 2024-03-04T16:06:30.040127+0000 mon.a (mon.0) 3292 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:06:31.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:31 smithi151 bash[23756]: audit 2024-03-04T16:06:30.043109+0000 mon.c (mon.2) 806 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:06:32.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:32 smithi117 bash[21480]: cluster 2024-03-04T16:06:31.179838+0000 mgr.y (mgr.24428) 647 : cluster 0 pgmap v1065: 204 pgs: 204 active+clean; 582 KiB data, 721 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:06:32.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:32 smithi117 bash[28843]: cluster 2024-03-04T16:06:31.179838+0000 mgr.y (mgr.24428) 647 : cluster 0 pgmap v1065: 204 pgs: 204 active+clean; 582 KiB data, 721 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:06:32.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:32 smithi151 bash[23756]: cluster 2024-03-04T16:06:31.179838+0000 mgr.y (mgr.24428) 647 : cluster 0 pgmap v1065: 204 pgs: 204 active+clean; 582 KiB data, 721 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:06:33.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:33 smithi117 bash[21480]: audit 2024-03-04T16:06:33.008908+0000 mon.a (mon.0) 3293 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:33.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:33 smithi117 bash[28843]: audit 2024-03-04T16:06:33.008908+0000 mon.a (mon.0) 3293 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:33.437 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:33 smithi151 bash[23756]: audit 2024-03-04T16:06:33.008908+0000 mon.a (mon.0) 3293 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:34.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:34 smithi117 bash[21480]: audit 2024-03-04T16:06:33.058097+0000 mon.a (mon.0) 3294 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:34.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:34 smithi117 bash[21480]: cluster 2024-03-04T16:06:33.065637+0000 mon.a (mon.0) 3295 : cluster 0 osdmap e659: 8 total, 8 up, 8 in 2024-03-04T16:06:34.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:34 smithi117 bash[21480]: audit 2024-03-04T16:06:33.067111+0000 mon.a (mon.0) 3296 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-128"}]: dispatch 2024-03-04T16:06:34.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:34 smithi117 bash[21480]: cluster 2024-03-04T16:06:33.180668+0000 mgr.y (mgr.24428) 648 : cluster 0 pgmap v1067: 204 pgs: 204 active+clean; 582 KiB data, 722 MiB used, 715 GiB / 715 GiB avail; 968 B/s rd, 322 B/s wr, 1 op/s 2024-03-04T16:06:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:34 smithi117 bash[28843]: audit 2024-03-04T16:06:33.058097+0000 mon.a (mon.0) 3294 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:34 smithi117 bash[28843]: cluster 2024-03-04T16:06:33.065637+0000 mon.a (mon.0) 3295 : cluster 0 osdmap e659: 8 total, 8 up, 8 in 2024-03-04T16:06:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:34 smithi117 bash[28843]: audit 2024-03-04T16:06:33.067111+0000 mon.a (mon.0) 3296 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-128"}]: dispatch 2024-03-04T16:06:34.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:34 smithi117 bash[28843]: cluster 2024-03-04T16:06:33.180668+0000 mgr.y (mgr.24428) 648 : cluster 0 pgmap v1067: 204 pgs: 204 active+clean; 582 KiB data, 722 MiB used, 715 GiB / 715 GiB avail; 968 B/s rd, 322 B/s wr, 1 op/s 2024-03-04T16:06:34.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:34 smithi151 bash[23756]: audit 2024-03-04T16:06:33.058097+0000 mon.a (mon.0) 3294 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:34 smithi151 bash[23756]: cluster 2024-03-04T16:06:33.065637+0000 mon.a (mon.0) 3295 : cluster 0 osdmap e659: 8 total, 8 up, 8 in 2024-03-04T16:06:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:34 smithi151 bash[23756]: audit 2024-03-04T16:06:33.067111+0000 mon.a (mon.0) 3296 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-128"}]: dispatch 2024-03-04T16:06:34.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:34 smithi151 bash[23756]: cluster 2024-03-04T16:06:33.180668+0000 mgr.y (mgr.24428) 648 : cluster 0 pgmap v1067: 204 pgs: 204 active+clean; 582 KiB data, 722 MiB used, 715 GiB / 715 GiB avail; 968 B/s rd, 322 B/s wr, 1 op/s 2024-03-04T16:06:35.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:35 smithi117 bash[21480]: cluster 2024-03-04T16:06:34.059174+0000 mon.a (mon.0) 3297 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:06:35.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:35 smithi117 bash[21480]: audit 2024-03-04T16:06:34.066954+0000 mon.a (mon.0) 3298 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-128"}]': finished 2024-03-04T16:06:35.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:35 smithi117 bash[21480]: cluster 2024-03-04T16:06:34.075550+0000 mon.a (mon.0) 3299 : cluster 0 osdmap e660: 8 total, 8 up, 8 in 2024-03-04T16:06:35.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:35 smithi117 bash[28843]: cluster 2024-03-04T16:06:34.059174+0000 mon.a (mon.0) 3297 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:06:35.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:35 smithi117 bash[28843]: audit 2024-03-04T16:06:34.066954+0000 mon.a (mon.0) 3298 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-128"}]': finished 2024-03-04T16:06:35.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:35 smithi117 bash[28843]: cluster 2024-03-04T16:06:34.075550+0000 mon.a (mon.0) 3299 : cluster 0 osdmap e660: 8 total, 8 up, 8 in 2024-03-04T16:06:35.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:35 smithi151 bash[23756]: cluster 2024-03-04T16:06:34.059174+0000 mon.a (mon.0) 3297 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:06:35.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:35 smithi151 bash[23756]: audit 2024-03-04T16:06:34.066954+0000 mon.a (mon.0) 3298 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-128"}]': finished 2024-03-04T16:06:35.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:35 smithi151 bash[23756]: cluster 2024-03-04T16:06:34.075550+0000 mon.a (mon.0) 3299 : cluster 0 osdmap e660: 8 total, 8 up, 8 in 2024-03-04T16:06:36.527 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:06:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:06:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:06:36.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:36 smithi117 bash[21480]: cluster 2024-03-04T16:06:35.075692+0000 mon.a (mon.0) 3300 : cluster 0 osdmap e661: 8 total, 8 up, 8 in 2024-03-04T16:06:36.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:36 smithi117 bash[21480]: cluster 2024-03-04T16:06:35.181572+0000 mgr.y (mgr.24428) 649 : cluster 0 pgmap v1070: 172 pgs: 172 active+clean; 582 KiB data, 722 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:06:36.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:36 smithi117 bash[21480]: cluster 2024-03-04T16:06:36.086975+0000 mon.a (mon.0) 3301 : cluster 0 osdmap e662: 8 total, 8 up, 8 in 2024-03-04T16:06:36.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:36 smithi117 bash[21480]: audit 2024-03-04T16:06:36.088826+0000 mon.a (mon.0) 3302 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:06:36.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:36 smithi117 bash[28843]: cluster 2024-03-04T16:06:35.075692+0000 mon.a (mon.0) 3300 : cluster 0 osdmap e661: 8 total, 8 up, 8 in 2024-03-04T16:06:36.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:36 smithi117 bash[28843]: cluster 2024-03-04T16:06:35.181572+0000 mgr.y (mgr.24428) 649 : cluster 0 pgmap v1070: 172 pgs: 172 active+clean; 582 KiB data, 722 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:06:36.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:36 smithi117 bash[28843]: cluster 2024-03-04T16:06:36.086975+0000 mon.a (mon.0) 3301 : cluster 0 osdmap e662: 8 total, 8 up, 8 in 2024-03-04T16:06:36.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:36 smithi117 bash[28843]: audit 2024-03-04T16:06:36.088826+0000 mon.a (mon.0) 3302 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:06:36.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:36 smithi151 bash[23756]: cluster 2024-03-04T16:06:35.075692+0000 mon.a (mon.0) 3300 : cluster 0 osdmap e661: 8 total, 8 up, 8 in 2024-03-04T16:06:36.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:36 smithi151 bash[23756]: cluster 2024-03-04T16:06:35.181572+0000 mgr.y (mgr.24428) 649 : cluster 0 pgmap v1070: 172 pgs: 172 active+clean; 582 KiB data, 722 MiB used, 715 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:06:36.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:36 smithi151 bash[23756]: cluster 2024-03-04T16:06:36.086975+0000 mon.a (mon.0) 3301 : cluster 0 osdmap e662: 8 total, 8 up, 8 in 2024-03-04T16:06:36.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:36 smithi151 bash[23756]: audit 2024-03-04T16:06:36.088826+0000 mon.a (mon.0) 3302 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:06:37.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:06:37 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:06:38.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:38 smithi151 bash[23756]: audit 2024-03-04T16:06:37.076037+0000 mon.a (mon.0) 3303 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:38.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:38 smithi151 bash[23756]: cluster 2024-03-04T16:06:37.084819+0000 mon.a (mon.0) 3304 : cluster 0 osdmap e663: 8 total, 8 up, 8 in 2024-03-04T16:06:38.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:38 smithi151 bash[23756]: audit 2024-03-04T16:06:37.090700+0000 mon.a (mon.0) 3305 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:06:38.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:38 smithi151 bash[23756]: audit 2024-03-04T16:06:37.155738+0000 mgr.y (mgr.24428) 650 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:06:38.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:38 smithi151 bash[23756]: cluster 2024-03-04T16:06:37.182550+0000 mgr.y (mgr.24428) 651 : cluster 0 pgmap v1073: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 722 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:06:38.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:38 smithi117 bash[21480]: audit 2024-03-04T16:06:37.076037+0000 mon.a (mon.0) 3303 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:38.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:38 smithi117 bash[21480]: cluster 2024-03-04T16:06:37.084819+0000 mon.a (mon.0) 3304 : cluster 0 osdmap e663: 8 total, 8 up, 8 in 2024-03-04T16:06:38.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:38 smithi117 bash[21480]: audit 2024-03-04T16:06:37.090700+0000 mon.a (mon.0) 3305 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:06:38.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:38 smithi117 bash[21480]: audit 2024-03-04T16:06:37.155738+0000 mgr.y (mgr.24428) 650 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:06:38.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:38 smithi117 bash[21480]: cluster 2024-03-04T16:06:37.182550+0000 mgr.y (mgr.24428) 651 : cluster 0 pgmap v1073: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 722 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:06:38.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:38 smithi117 bash[28843]: audit 2024-03-04T16:06:37.076037+0000 mon.a (mon.0) 3303 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:38.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:38 smithi117 bash[28843]: cluster 2024-03-04T16:06:37.084819+0000 mon.a (mon.0) 3304 : cluster 0 osdmap e663: 8 total, 8 up, 8 in 2024-03-04T16:06:38.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:38 smithi117 bash[28843]: audit 2024-03-04T16:06:37.090700+0000 mon.a (mon.0) 3305 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:06:38.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:38 smithi117 bash[28843]: audit 2024-03-04T16:06:37.155738+0000 mgr.y (mgr.24428) 650 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:06:38.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:38 smithi117 bash[28843]: cluster 2024-03-04T16:06:37.182550+0000 mgr.y (mgr.24428) 651 : cluster 0 pgmap v1073: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 722 MiB used, 715 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:06:39.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:39 smithi151 bash[23756]: audit 2024-03-04T16:06:38.085220+0000 mon.a (mon.0) 3306 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-130", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:06:39.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:39 smithi151 bash[23756]: cluster 2024-03-04T16:06:38.094143+0000 mon.a (mon.0) 3307 : cluster 0 osdmap e664: 8 total, 8 up, 8 in 2024-03-04T16:06:39.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:39 smithi151 bash[23756]: audit 2024-03-04T16:06:38.095729+0000 mon.a (mon.0) 3308 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-130"}]: dispatch 2024-03-04T16:06:39.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:39 smithi117 bash[21480]: audit 2024-03-04T16:06:38.085220+0000 mon.a (mon.0) 3306 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-130", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:06:39.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:39 smithi117 bash[21480]: cluster 2024-03-04T16:06:38.094143+0000 mon.a (mon.0) 3307 : cluster 0 osdmap e664: 8 total, 8 up, 8 in 2024-03-04T16:06:39.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:39 smithi117 bash[21480]: audit 2024-03-04T16:06:38.095729+0000 mon.a (mon.0) 3308 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-130"}]: dispatch 2024-03-04T16:06:39.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:39 smithi117 bash[28843]: audit 2024-03-04T16:06:38.085220+0000 mon.a (mon.0) 3306 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-130", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:06:39.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:39 smithi117 bash[28843]: cluster 2024-03-04T16:06:38.094143+0000 mon.a (mon.0) 3307 : cluster 0 osdmap e664: 8 total, 8 up, 8 in 2024-03-04T16:06:39.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:39 smithi117 bash[28843]: audit 2024-03-04T16:06:38.095729+0000 mon.a (mon.0) 3308 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-130"}]: dispatch 2024-03-04T16:06:40.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:40 smithi151 bash[23756]: audit 2024-03-04T16:06:39.087753+0000 mon.a (mon.0) 3309 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-130"}]': finished 2024-03-04T16:06:40.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:40 smithi151 bash[23756]: cluster 2024-03-04T16:06:39.095476+0000 mon.a (mon.0) 3310 : cluster 0 osdmap e665: 8 total, 8 up, 8 in 2024-03-04T16:06:40.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:40 smithi151 bash[23756]: audit 2024-03-04T16:06:39.096903+0000 mon.a (mon.0) 3311 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-130", "mode": "writeback"}]: dispatch 2024-03-04T16:06:40.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:40 smithi151 bash[23756]: cluster 2024-03-04T16:06:39.183401+0000 mgr.y (mgr.24428) 652 : cluster 0 pgmap v1076: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 722 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:06:40.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:40 smithi151 bash[23756]: cluster 2024-03-04T16:06:40.088083+0000 mon.a (mon.0) 3312 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:06:40.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:40 smithi151 bash[23756]: audit 2024-03-04T16:06:40.090427+0000 mon.a (mon.0) 3313 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-130", "mode": "writeback"}]': finished 2024-03-04T16:06:40.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:40 smithi151 bash[23756]: cluster 2024-03-04T16:06:40.096735+0000 mon.a (mon.0) 3314 : cluster 0 osdmap e666: 8 total, 8 up, 8 in 2024-03-04T16:06:40.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:40 smithi117 bash[21480]: audit 2024-03-04T16:06:39.087753+0000 mon.a (mon.0) 3309 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-130"}]': finished 2024-03-04T16:06:40.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:40 smithi117 bash[21480]: cluster 2024-03-04T16:06:39.095476+0000 mon.a (mon.0) 3310 : cluster 0 osdmap e665: 8 total, 8 up, 8 in 2024-03-04T16:06:40.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:40 smithi117 bash[21480]: audit 2024-03-04T16:06:39.096903+0000 mon.a (mon.0) 3311 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-130", "mode": "writeback"}]: dispatch 2024-03-04T16:06:40.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:40 smithi117 bash[21480]: cluster 2024-03-04T16:06:39.183401+0000 mgr.y (mgr.24428) 652 : cluster 0 pgmap v1076: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 722 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:06:40.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:40 smithi117 bash[21480]: cluster 2024-03-04T16:06:40.088083+0000 mon.a (mon.0) 3312 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:06:40.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:40 smithi117 bash[21480]: audit 2024-03-04T16:06:40.090427+0000 mon.a (mon.0) 3313 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-130", "mode": "writeback"}]': finished 2024-03-04T16:06:40.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:40 smithi117 bash[21480]: cluster 2024-03-04T16:06:40.096735+0000 mon.a (mon.0) 3314 : cluster 0 osdmap e666: 8 total, 8 up, 8 in 2024-03-04T16:06:40.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:40 smithi117 bash[28843]: audit 2024-03-04T16:06:39.087753+0000 mon.a (mon.0) 3309 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-130"}]': finished 2024-03-04T16:06:40.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:40 smithi117 bash[28843]: cluster 2024-03-04T16:06:39.095476+0000 mon.a (mon.0) 3310 : cluster 0 osdmap e665: 8 total, 8 up, 8 in 2024-03-04T16:06:40.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:40 smithi117 bash[28843]: audit 2024-03-04T16:06:39.096903+0000 mon.a (mon.0) 3311 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-130", "mode": "writeback"}]: dispatch 2024-03-04T16:06:40.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:40 smithi117 bash[28843]: cluster 2024-03-04T16:06:39.183401+0000 mgr.y (mgr.24428) 652 : cluster 0 pgmap v1076: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 722 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:06:40.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:40 smithi117 bash[28843]: cluster 2024-03-04T16:06:40.088083+0000 mon.a (mon.0) 3312 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:06:40.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:40 smithi117 bash[28843]: audit 2024-03-04T16:06:40.090427+0000 mon.a (mon.0) 3313 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-130", "mode": "writeback"}]': finished 2024-03-04T16:06:40.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:40 smithi117 bash[28843]: cluster 2024-03-04T16:06:40.096735+0000 mon.a (mon.0) 3314 : cluster 0 osdmap e666: 8 total, 8 up, 8 in 2024-03-04T16:06:41.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:41 smithi151 bash[23756]: audit 2024-03-04T16:06:40.297643+0000 mon.a (mon.0) 3315 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:41.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:41 smithi117 bash[21480]: audit 2024-03-04T16:06:40.297643+0000 mon.a (mon.0) 3315 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:41.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:41 smithi117 bash[28843]: audit 2024-03-04T16:06:40.297643+0000 mon.a (mon.0) 3315 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:42.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:42 smithi151 bash[23756]: audit 2024-03-04T16:06:41.123263+0000 mon.a (mon.0) 3316 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:42.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:42 smithi151 bash[23756]: cluster 2024-03-04T16:06:41.130131+0000 mon.a (mon.0) 3317 : cluster 0 osdmap e667: 8 total, 8 up, 8 in 2024-03-04T16:06:42.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:42 smithi151 bash[23756]: audit 2024-03-04T16:06:41.130917+0000 mon.a (mon.0) 3318 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-130"}]: dispatch 2024-03-04T16:06:42.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:42 smithi151 bash[23756]: cluster 2024-03-04T16:06:41.184214+0000 mgr.y (mgr.24428) 653 : cluster 0 pgmap v1079: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 722 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:06:42.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:42 smithi117 bash[21480]: audit 2024-03-04T16:06:41.123263+0000 mon.a (mon.0) 3316 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:42.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:42 smithi117 bash[21480]: cluster 2024-03-04T16:06:41.130131+0000 mon.a (mon.0) 3317 : cluster 0 osdmap e667: 8 total, 8 up, 8 in 2024-03-04T16:06:42.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:42 smithi117 bash[21480]: audit 2024-03-04T16:06:41.130917+0000 mon.a (mon.0) 3318 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-130"}]: dispatch 2024-03-04T16:06:42.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:42 smithi117 bash[21480]: cluster 2024-03-04T16:06:41.184214+0000 mgr.y (mgr.24428) 653 : cluster 0 pgmap v1079: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 722 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:06:42.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:42 smithi117 bash[28843]: audit 2024-03-04T16:06:41.123263+0000 mon.a (mon.0) 3316 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:42.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:42 smithi117 bash[28843]: cluster 2024-03-04T16:06:41.130131+0000 mon.a (mon.0) 3317 : cluster 0 osdmap e667: 8 total, 8 up, 8 in 2024-03-04T16:06:42.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:42 smithi117 bash[28843]: audit 2024-03-04T16:06:41.130917+0000 mon.a (mon.0) 3318 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-130"}]: dispatch 2024-03-04T16:06:42.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:42 smithi117 bash[28843]: cluster 2024-03-04T16:06:41.184214+0000 mgr.y (mgr.24428) 653 : cluster 0 pgmap v1079: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 722 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:06:43.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:43 smithi151 bash[23756]: cluster 2024-03-04T16:06:42.124315+0000 mon.a (mon.0) 3319 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:06:43.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:43 smithi151 bash[23756]: audit 2024-03-04T16:06:42.132523+0000 mon.a (mon.0) 3320 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-130"}]': finished 2024-03-04T16:06:43.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:43 smithi151 bash[23756]: cluster 2024-03-04T16:06:42.142707+0000 mon.a (mon.0) 3321 : cluster 0 osdmap e668: 8 total, 8 up, 8 in 2024-03-04T16:06:43.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:43 smithi117 bash[21480]: cluster 2024-03-04T16:06:42.124315+0000 mon.a (mon.0) 3319 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:06:43.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:43 smithi117 bash[21480]: audit 2024-03-04T16:06:42.132523+0000 mon.a (mon.0) 3320 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-130"}]': finished 2024-03-04T16:06:43.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:43 smithi117 bash[21480]: cluster 2024-03-04T16:06:42.142707+0000 mon.a (mon.0) 3321 : cluster 0 osdmap e668: 8 total, 8 up, 8 in 2024-03-04T16:06:43.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:43 smithi117 bash[28843]: cluster 2024-03-04T16:06:42.124315+0000 mon.a (mon.0) 3319 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:06:43.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:43 smithi117 bash[28843]: audit 2024-03-04T16:06:42.132523+0000 mon.a (mon.0) 3320 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-130"}]': finished 2024-03-04T16:06:43.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:43 smithi117 bash[28843]: cluster 2024-03-04T16:06:42.142707+0000 mon.a (mon.0) 3321 : cluster 0 osdmap e668: 8 total, 8 up, 8 in 2024-03-04T16:06:44.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:44 smithi151 bash[23756]: cluster 2024-03-04T16:06:43.150020+0000 mon.a (mon.0) 3322 : cluster 0 osdmap e669: 8 total, 8 up, 8 in 2024-03-04T16:06:44.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:44 smithi151 bash[23756]: cluster 2024-03-04T16:06:43.185081+0000 mgr.y (mgr.24428) 654 : cluster 0 pgmap v1082: 172 pgs: 172 active+clean; 582 KiB data, 740 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:44.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:44 smithi117 bash[21480]: cluster 2024-03-04T16:06:43.150020+0000 mon.a (mon.0) 3322 : cluster 0 osdmap e669: 8 total, 8 up, 8 in 2024-03-04T16:06:44.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:44 smithi117 bash[21480]: cluster 2024-03-04T16:06:43.185081+0000 mgr.y (mgr.24428) 654 : cluster 0 pgmap v1082: 172 pgs: 172 active+clean; 582 KiB data, 740 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:44.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:44 smithi117 bash[28843]: cluster 2024-03-04T16:06:43.150020+0000 mon.a (mon.0) 3322 : cluster 0 osdmap e669: 8 total, 8 up, 8 in 2024-03-04T16:06:44.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:44 smithi117 bash[28843]: cluster 2024-03-04T16:06:43.185081+0000 mgr.y (mgr.24428) 654 : cluster 0 pgmap v1082: 172 pgs: 172 active+clean; 582 KiB data, 740 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:45.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:45 smithi151 bash[23756]: cluster 2024-03-04T16:06:44.163498+0000 mon.a (mon.0) 3323 : cluster 0 osdmap e670: 8 total, 8 up, 8 in 2024-03-04T16:06:45.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:45 smithi151 bash[23756]: audit 2024-03-04T16:06:44.169895+0000 mon.a (mon.0) 3324 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:06:45.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:45 smithi151 bash[23756]: audit 2024-03-04T16:06:45.083283+0000 mon.a (mon.0) 3325 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:06:45.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:45 smithi151 bash[23756]: audit 2024-03-04T16:06:45.086206+0000 mon.c (mon.2) 807 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:06:45.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:45 smithi117 bash[28843]: cluster 2024-03-04T16:06:44.163498+0000 mon.a (mon.0) 3323 : cluster 0 osdmap e670: 8 total, 8 up, 8 in 2024-03-04T16:06:45.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:45 smithi117 bash[28843]: audit 2024-03-04T16:06:44.169895+0000 mon.a (mon.0) 3324 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:06:45.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:45 smithi117 bash[28843]: audit 2024-03-04T16:06:45.083283+0000 mon.a (mon.0) 3325 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:06:45.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:45 smithi117 bash[28843]: audit 2024-03-04T16:06:45.086206+0000 mon.c (mon.2) 807 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:06:45.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:45 smithi117 bash[21480]: cluster 2024-03-04T16:06:44.163498+0000 mon.a (mon.0) 3323 : cluster 0 osdmap e670: 8 total, 8 up, 8 in 2024-03-04T16:06:45.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:45 smithi117 bash[21480]: audit 2024-03-04T16:06:44.169895+0000 mon.a (mon.0) 3324 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:06:45.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:45 smithi117 bash[21480]: audit 2024-03-04T16:06:45.083283+0000 mon.a (mon.0) 3325 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:06:45.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:45 smithi117 bash[21480]: audit 2024-03-04T16:06:45.086206+0000 mon.c (mon.2) 807 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:06:46.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:46 smithi151 bash[23756]: audit 2024-03-04T16:06:45.166234+0000 mon.a (mon.0) 3326 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:46.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:46 smithi151 bash[23756]: cluster 2024-03-04T16:06:45.178352+0000 mon.a (mon.0) 3327 : cluster 0 osdmap e671: 8 total, 8 up, 8 in 2024-03-04T16:06:46.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:46 smithi151 bash[23756]: audit 2024-03-04T16:06:45.185043+0000 mon.a (mon.0) 3328 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:06:46.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:46 smithi151 bash[23756]: cluster 2024-03-04T16:06:45.187093+0000 mgr.y (mgr.24428) 655 : cluster 0 pgmap v1085: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 740 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:46.567 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:06:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:06:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:06:46.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:46 smithi117 bash[21480]: audit 2024-03-04T16:06:45.166234+0000 mon.a (mon.0) 3326 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:46.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:46 smithi117 bash[21480]: cluster 2024-03-04T16:06:45.178352+0000 mon.a (mon.0) 3327 : cluster 0 osdmap e671: 8 total, 8 up, 8 in 2024-03-04T16:06:46.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:46 smithi117 bash[21480]: audit 2024-03-04T16:06:45.185043+0000 mon.a (mon.0) 3328 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:06:46.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:46 smithi117 bash[21480]: cluster 2024-03-04T16:06:45.187093+0000 mgr.y (mgr.24428) 655 : cluster 0 pgmap v1085: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 740 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:46.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:46 smithi117 bash[28843]: audit 2024-03-04T16:06:45.166234+0000 mon.a (mon.0) 3326 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:46.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:46 smithi117 bash[28843]: cluster 2024-03-04T16:06:45.178352+0000 mon.a (mon.0) 3327 : cluster 0 osdmap e671: 8 total, 8 up, 8 in 2024-03-04T16:06:46.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:46 smithi117 bash[28843]: audit 2024-03-04T16:06:45.185043+0000 mon.a (mon.0) 3328 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:06:46.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:46 smithi117 bash[28843]: cluster 2024-03-04T16:06:45.187093+0000 mgr.y (mgr.24428) 655 : cluster 0 pgmap v1085: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 740 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-03-04T16:06:47.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:06:47 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:06:47.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:47 smithi151 bash[23756]: audit 2024-03-04T16:06:46.177732+0000 mon.a (mon.0) 3329 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-132", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:06:47.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:47 smithi151 bash[23756]: cluster 2024-03-04T16:06:46.185699+0000 mon.a (mon.0) 3330 : cluster 0 osdmap e672: 8 total, 8 up, 8 in 2024-03-04T16:06:47.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:47 smithi151 bash[23756]: audit 2024-03-04T16:06:46.187626+0000 mon.a (mon.0) 3331 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-132"}]: dispatch 2024-03-04T16:06:47.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:47 smithi117 bash[21480]: audit 2024-03-04T16:06:46.177732+0000 mon.a (mon.0) 3329 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-132", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:06:47.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:47 smithi117 bash[21480]: cluster 2024-03-04T16:06:46.185699+0000 mon.a (mon.0) 3330 : cluster 0 osdmap e672: 8 total, 8 up, 8 in 2024-03-04T16:06:47.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:47 smithi117 bash[21480]: audit 2024-03-04T16:06:46.187626+0000 mon.a (mon.0) 3331 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-132"}]: dispatch 2024-03-04T16:06:47.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:47 smithi117 bash[28843]: audit 2024-03-04T16:06:46.177732+0000 mon.a (mon.0) 3329 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-132", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:06:47.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:47 smithi117 bash[28843]: cluster 2024-03-04T16:06:46.185699+0000 mon.a (mon.0) 3330 : cluster 0 osdmap e672: 8 total, 8 up, 8 in 2024-03-04T16:06:47.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:47 smithi117 bash[28843]: audit 2024-03-04T16:06:46.187626+0000 mon.a (mon.0) 3331 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-132"}]: dispatch 2024-03-04T16:06:48.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:48 smithi117 bash[28843]: audit 2024-03-04T16:06:47.166355+0000 mgr.y (mgr.24428) 656 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:06:48.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:48 smithi117 bash[28843]: audit 2024-03-04T16:06:47.180533+0000 mon.a (mon.0) 3332 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-132"}]': finished 2024-03-04T16:06:48.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:48 smithi117 bash[28843]: cluster 2024-03-04T16:06:47.188896+0000 mon.a (mon.0) 3333 : cluster 0 osdmap e673: 8 total, 8 up, 8 in 2024-03-04T16:06:48.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:48 smithi117 bash[28843]: audit 2024-03-04T16:06:47.190002+0000 mon.a (mon.0) 3334 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-132", "mode": "writeback"}]: dispatch 2024-03-04T16:06:48.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:48 smithi117 bash[28843]: cluster 2024-03-04T16:06:47.190890+0000 mgr.y (mgr.24428) 657 : cluster 0 pgmap v1088: 204 pgs: 204 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:06:48.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:48 smithi117 bash[28843]: cluster 2024-03-04T16:06:48.180848+0000 mon.a (mon.0) 3335 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:06:48.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:48 smithi117 bash[28843]: audit 2024-03-04T16:06:48.183118+0000 mon.a (mon.0) 3336 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-132", "mode": "writeback"}]': finished 2024-03-04T16:06:48.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:48 smithi117 bash[28843]: cluster 2024-03-04T16:06:48.190765+0000 mon.a (mon.0) 3337 : cluster 0 osdmap e674: 8 total, 8 up, 8 in 2024-03-04T16:06:48.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:48 smithi117 bash[21480]: audit 2024-03-04T16:06:47.166355+0000 mgr.y (mgr.24428) 656 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:06:48.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:48 smithi117 bash[21480]: audit 2024-03-04T16:06:47.180533+0000 mon.a (mon.0) 3332 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-132"}]': finished 2024-03-04T16:06:48.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:48 smithi117 bash[21480]: cluster 2024-03-04T16:06:47.188896+0000 mon.a (mon.0) 3333 : cluster 0 osdmap e673: 8 total, 8 up, 8 in 2024-03-04T16:06:48.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:48 smithi117 bash[21480]: audit 2024-03-04T16:06:47.190002+0000 mon.a (mon.0) 3334 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-132", "mode": "writeback"}]: dispatch 2024-03-04T16:06:48.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:48 smithi117 bash[21480]: cluster 2024-03-04T16:06:47.190890+0000 mgr.y (mgr.24428) 657 : cluster 0 pgmap v1088: 204 pgs: 204 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:06:48.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:48 smithi117 bash[21480]: cluster 2024-03-04T16:06:48.180848+0000 mon.a (mon.0) 3335 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:06:48.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:48 smithi117 bash[21480]: audit 2024-03-04T16:06:48.183118+0000 mon.a (mon.0) 3336 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-132", "mode": "writeback"}]': finished 2024-03-04T16:06:48.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:48 smithi117 bash[21480]: cluster 2024-03-04T16:06:48.190765+0000 mon.a (mon.0) 3337 : cluster 0 osdmap e674: 8 total, 8 up, 8 in 2024-03-04T16:06:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:48 smithi151 bash[23756]: audit 2024-03-04T16:06:47.166355+0000 mgr.y (mgr.24428) 656 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:06:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:48 smithi151 bash[23756]: audit 2024-03-04T16:06:47.180533+0000 mon.a (mon.0) 3332 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-132"}]': finished 2024-03-04T16:06:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:48 smithi151 bash[23756]: cluster 2024-03-04T16:06:47.188896+0000 mon.a (mon.0) 3333 : cluster 0 osdmap e673: 8 total, 8 up, 8 in 2024-03-04T16:06:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:48 smithi151 bash[23756]: audit 2024-03-04T16:06:47.190002+0000 mon.a (mon.0) 3334 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-132", "mode": "writeback"}]: dispatch 2024-03-04T16:06:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:48 smithi151 bash[23756]: cluster 2024-03-04T16:06:47.190890+0000 mgr.y (mgr.24428) 657 : cluster 0 pgmap v1088: 204 pgs: 204 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:06:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:48 smithi151 bash[23756]: cluster 2024-03-04T16:06:48.180848+0000 mon.a (mon.0) 3335 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:06:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:48 smithi151 bash[23756]: audit 2024-03-04T16:06:48.183118+0000 mon.a (mon.0) 3336 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-132", "mode": "writeback"}]': finished 2024-03-04T16:06:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:48 smithi151 bash[23756]: cluster 2024-03-04T16:06:48.190765+0000 mon.a (mon.0) 3337 : cluster 0 osdmap e674: 8 total, 8 up, 8 in 2024-03-04T16:06:50.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:50 smithi117 bash[21480]: cluster 2024-03-04T16:06:49.191760+0000 mgr.y (mgr.24428) 658 : cluster 0 pgmap v1090: 204 pgs: 204 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:06:50.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:50 smithi117 bash[21480]: cluster 2024-03-04T16:06:49.207997+0000 mon.a (mon.0) 3338 : cluster 0 osdmap e675: 8 total, 8 up, 8 in 2024-03-04T16:06:50.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:50 smithi117 bash[28843]: cluster 2024-03-04T16:06:49.191760+0000 mgr.y (mgr.24428) 658 : cluster 0 pgmap v1090: 204 pgs: 204 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:06:50.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:50 smithi117 bash[28843]: cluster 2024-03-04T16:06:49.207997+0000 mon.a (mon.0) 3338 : cluster 0 osdmap e675: 8 total, 8 up, 8 in 2024-03-04T16:06:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:50 smithi151 bash[23756]: cluster 2024-03-04T16:06:49.191760+0000 mgr.y (mgr.24428) 658 : cluster 0 pgmap v1090: 204 pgs: 204 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:06:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:50 smithi151 bash[23756]: cluster 2024-03-04T16:06:49.207997+0000 mon.a (mon.0) 3338 : cluster 0 osdmap e675: 8 total, 8 up, 8 in 2024-03-04T16:06:51.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:51 smithi117 bash[21480]: cluster 2024-03-04T16:06:50.222929+0000 mon.a (mon.0) 3339 : cluster 0 osdmap e676: 8 total, 8 up, 8 in 2024-03-04T16:06:51.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:51 smithi117 bash[21480]: audit 2024-03-04T16:06:50.315016+0000 mon.a (mon.0) 3340 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:51.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:51 smithi117 bash[28843]: cluster 2024-03-04T16:06:50.222929+0000 mon.a (mon.0) 3339 : cluster 0 osdmap e676: 8 total, 8 up, 8 in 2024-03-04T16:06:51.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:51 smithi117 bash[28843]: audit 2024-03-04T16:06:50.315016+0000 mon.a (mon.0) 3340 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:51.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:51 smithi151 bash[23756]: cluster 2024-03-04T16:06:50.222929+0000 mon.a (mon.0) 3339 : cluster 0 osdmap e676: 8 total, 8 up, 8 in 2024-03-04T16:06:51.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:51 smithi151 bash[23756]: audit 2024-03-04T16:06:50.315016+0000 mon.a (mon.0) 3340 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:52 smithi117 bash[28843]: cluster 2024-03-04T16:06:51.192595+0000 mgr.y (mgr.24428) 659 : cluster 0 pgmap v1093: 204 pgs: 204 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:06:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:52 smithi117 bash[28843]: audit 2024-03-04T16:06:51.226225+0000 mon.a (mon.0) 3341 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:52 smithi117 bash[28843]: cluster 2024-03-04T16:06:51.237084+0000 mon.a (mon.0) 3342 : cluster 0 osdmap e677: 8 total, 8 up, 8 in 2024-03-04T16:06:52.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:52 smithi117 bash[28843]: audit 2024-03-04T16:06:51.249513+0000 mon.a (mon.0) 3343 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-132"}]: dispatch 2024-03-04T16:06:52.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:52 smithi117 bash[21480]: cluster 2024-03-04T16:06:51.192595+0000 mgr.y (mgr.24428) 659 : cluster 0 pgmap v1093: 204 pgs: 204 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:06:52.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:52 smithi117 bash[21480]: audit 2024-03-04T16:06:51.226225+0000 mon.a (mon.0) 3341 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:52.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:52 smithi117 bash[21480]: cluster 2024-03-04T16:06:51.237084+0000 mon.a (mon.0) 3342 : cluster 0 osdmap e677: 8 total, 8 up, 8 in 2024-03-04T16:06:52.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:52 smithi117 bash[21480]: audit 2024-03-04T16:06:51.249513+0000 mon.a (mon.0) 3343 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-132"}]: dispatch 2024-03-04T16:06:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:52 smithi151 bash[23756]: cluster 2024-03-04T16:06:51.192595+0000 mgr.y (mgr.24428) 659 : cluster 0 pgmap v1093: 204 pgs: 204 active+clean; 582 KiB data, 725 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:06:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:52 smithi151 bash[23756]: audit 2024-03-04T16:06:51.226225+0000 mon.a (mon.0) 3341 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:52 smithi151 bash[23756]: cluster 2024-03-04T16:06:51.237084+0000 mon.a (mon.0) 3342 : cluster 0 osdmap e677: 8 total, 8 up, 8 in 2024-03-04T16:06:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:52 smithi151 bash[23756]: audit 2024-03-04T16:06:51.249513+0000 mon.a (mon.0) 3343 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-132"}]: dispatch 2024-03-04T16:06:53.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:53 smithi117 bash[21480]: audit 2024-03-04T16:06:52.238270+0000 mon.a (mon.0) 3344 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-132"}]': finished 2024-03-04T16:06:53.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:53 smithi117 bash[21480]: cluster 2024-03-04T16:06:52.245939+0000 mon.a (mon.0) 3345 : cluster 0 osdmap e678: 8 total, 8 up, 8 in 2024-03-04T16:06:53.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:53 smithi117 bash[28843]: audit 2024-03-04T16:06:52.238270+0000 mon.a (mon.0) 3344 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-132"}]': finished 2024-03-04T16:06:53.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:53 smithi117 bash[28843]: cluster 2024-03-04T16:06:52.245939+0000 mon.a (mon.0) 3345 : cluster 0 osdmap e678: 8 total, 8 up, 8 in 2024-03-04T16:06:53.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:53 smithi151 bash[23756]: audit 2024-03-04T16:06:52.238270+0000 mon.a (mon.0) 3344 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-132"}]': finished 2024-03-04T16:06:53.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:53 smithi151 bash[23756]: cluster 2024-03-04T16:06:52.245939+0000 mon.a (mon.0) 3345 : cluster 0 osdmap e678: 8 total, 8 up, 8 in 2024-03-04T16:06:54.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:54 smithi117 bash[21480]: cluster 2024-03-04T16:06:53.193388+0000 mgr.y (mgr.24428) 660 : cluster 0 pgmap v1096: 204 pgs: 204 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-04T16:06:54.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:54 smithi117 bash[21480]: cluster 2024-03-04T16:06:53.266070+0000 mon.a (mon.0) 3346 : cluster 0 osdmap e679: 8 total, 8 up, 8 in 2024-03-04T16:06:54.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:54 smithi117 bash[21480]: audit 2024-03-04T16:06:53.280163+0000 mon.c (mon.2) 808 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:06:54.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:54 smithi117 bash[21480]: audit 2024-03-04T16:06:53.321538+0000 mon.a (mon.0) 3347 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:54.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:54 smithi117 bash[21480]: audit 2024-03-04T16:06:53.962705+0000 mon.c (mon.2) 809 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:06:54.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:54 smithi117 bash[21480]: audit 2024-03-04T16:06:53.964528+0000 mon.c (mon.2) 810 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:06:54.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:54 smithi117 bash[21480]: audit 2024-03-04T16:06:53.975436+0000 mon.a (mon.0) 3348 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:06:54.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:54 smithi117 bash[28843]: cluster 2024-03-04T16:06:53.193388+0000 mgr.y (mgr.24428) 660 : cluster 0 pgmap v1096: 204 pgs: 204 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-04T16:06:54.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:54 smithi117 bash[28843]: cluster 2024-03-04T16:06:53.266070+0000 mon.a (mon.0) 3346 : cluster 0 osdmap e679: 8 total, 8 up, 8 in 2024-03-04T16:06:54.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:54 smithi117 bash[28843]: audit 2024-03-04T16:06:53.280163+0000 mon.c (mon.2) 808 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:06:54.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:54 smithi117 bash[28843]: audit 2024-03-04T16:06:53.321538+0000 mon.a (mon.0) 3347 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:54.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:54 smithi117 bash[28843]: audit 2024-03-04T16:06:53.962705+0000 mon.c (mon.2) 809 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:06:54.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:54 smithi117 bash[28843]: audit 2024-03-04T16:06:53.964528+0000 mon.c (mon.2) 810 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:06:54.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:54 smithi117 bash[28843]: audit 2024-03-04T16:06:53.975436+0000 mon.a (mon.0) 3348 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:06:54.681 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 16:06:54 smithi151 bash[51061]: logger=sqlstore.transactions t=2024-03-04T16:06:54.216605379Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-03-04T16:06:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:54 smithi151 bash[23756]: cluster 2024-03-04T16:06:53.193388+0000 mgr.y (mgr.24428) 660 : cluster 0 pgmap v1096: 204 pgs: 204 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-04T16:06:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:54 smithi151 bash[23756]: cluster 2024-03-04T16:06:53.266070+0000 mon.a (mon.0) 3346 : cluster 0 osdmap e679: 8 total, 8 up, 8 in 2024-03-04T16:06:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:54 smithi151 bash[23756]: audit 2024-03-04T16:06:53.280163+0000 mon.c (mon.2) 808 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:06:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:54 smithi151 bash[23756]: audit 2024-03-04T16:06:53.321538+0000 mon.a (mon.0) 3347 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:06:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:54 smithi151 bash[23756]: audit 2024-03-04T16:06:53.962705+0000 mon.c (mon.2) 809 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:06:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:54 smithi151 bash[23756]: audit 2024-03-04T16:06:53.964528+0000 mon.c (mon.2) 810 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:06:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:54 smithi151 bash[23756]: audit 2024-03-04T16:06:53.975436+0000 mon.a (mon.0) 3348 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:06:55.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:55 smithi117 bash[28843]: audit 2024-03-04T16:06:54.253803+0000 mon.a (mon.0) 3349 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:55.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:55 smithi117 bash[28843]: cluster 2024-03-04T16:06:54.261841+0000 mon.a (mon.0) 3350 : cluster 0 osdmap e680: 8 total, 8 up, 8 in 2024-03-04T16:06:55.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:55 smithi117 bash[28843]: audit 2024-03-04T16:06:54.263424+0000 mon.a (mon.0) 3351 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-132"}]: dispatch 2024-03-04T16:06:55.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:55 smithi117 bash[28843]: cluster 2024-03-04T16:06:55.254258+0000 mon.a (mon.0) 3352 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:06:55.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:55 smithi117 bash[28843]: audit 2024-03-04T16:06:55.256742+0000 mon.a (mon.0) 3353 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-132"}]': finished 2024-03-04T16:06:55.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:55 smithi117 bash[28843]: cluster 2024-03-04T16:06:55.264088+0000 mon.a (mon.0) 3354 : cluster 0 osdmap e681: 8 total, 8 up, 8 in 2024-03-04T16:06:55.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:55 smithi117 bash[21480]: audit 2024-03-04T16:06:54.253803+0000 mon.a (mon.0) 3349 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:55.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:55 smithi117 bash[21480]: cluster 2024-03-04T16:06:54.261841+0000 mon.a (mon.0) 3350 : cluster 0 osdmap e680: 8 total, 8 up, 8 in 2024-03-04T16:06:55.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:55 smithi117 bash[21480]: audit 2024-03-04T16:06:54.263424+0000 mon.a (mon.0) 3351 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-132"}]: dispatch 2024-03-04T16:06:55.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:55 smithi117 bash[21480]: cluster 2024-03-04T16:06:55.254258+0000 mon.a (mon.0) 3352 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:06:55.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:55 smithi117 bash[21480]: audit 2024-03-04T16:06:55.256742+0000 mon.a (mon.0) 3353 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-132"}]': finished 2024-03-04T16:06:55.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:55 smithi117 bash[21480]: cluster 2024-03-04T16:06:55.264088+0000 mon.a (mon.0) 3354 : cluster 0 osdmap e681: 8 total, 8 up, 8 in 2024-03-04T16:06:55.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:55 smithi151 bash[23756]: audit 2024-03-04T16:06:54.253803+0000 mon.a (mon.0) 3349 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:06:55.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:55 smithi151 bash[23756]: cluster 2024-03-04T16:06:54.261841+0000 mon.a (mon.0) 3350 : cluster 0 osdmap e680: 8 total, 8 up, 8 in 2024-03-04T16:06:55.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:55 smithi151 bash[23756]: audit 2024-03-04T16:06:54.263424+0000 mon.a (mon.0) 3351 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-132"}]: dispatch 2024-03-04T16:06:55.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:55 smithi151 bash[23756]: cluster 2024-03-04T16:06:55.254258+0000 mon.a (mon.0) 3352 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:06:55.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:55 smithi151 bash[23756]: audit 2024-03-04T16:06:55.256742+0000 mon.a (mon.0) 3353 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-132"}]': finished 2024-03-04T16:06:55.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:55 smithi151 bash[23756]: cluster 2024-03-04T16:06:55.264088+0000 mon.a (mon.0) 3354 : cluster 0 osdmap e681: 8 total, 8 up, 8 in 2024-03-04T16:06:56.567 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:06:56 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:06:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:06:56.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:56 smithi117 bash[28843]: cluster 2024-03-04T16:06:55.194165+0000 mgr.y (mgr.24428) 661 : cluster 0 pgmap v1099: 204 pgs: 204 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-04T16:06:56.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:56 smithi117 bash[21480]: cluster 2024-03-04T16:06:55.194165+0000 mgr.y (mgr.24428) 661 : cluster 0 pgmap v1099: 204 pgs: 204 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-04T16:06:56.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:56 smithi151 bash[23756]: cluster 2024-03-04T16:06:55.194165+0000 mgr.y (mgr.24428) 661 : cluster 0 pgmap v1099: 204 pgs: 204 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-03-04T16:06:57.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:57 smithi151 bash[23756]: cluster 2024-03-04T16:06:56.304060+0000 mon.a (mon.0) 3355 : cluster 0 osdmap e682: 8 total, 8 up, 8 in 2024-03-04T16:06:57.432 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:06:57 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:06:57.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:57 smithi117 bash[21480]: cluster 2024-03-04T16:06:56.304060+0000 mon.a (mon.0) 3355 : cluster 0 osdmap e682: 8 total, 8 up, 8 in 2024-03-04T16:06:57.567 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:57 smithi117 bash[28843]: cluster 2024-03-04T16:06:56.304060+0000 mon.a (mon.0) 3355 : cluster 0 osdmap e682: 8 total, 8 up, 8 in 2024-03-04T16:06:58.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:58 smithi151 bash[23756]: audit 2024-03-04T16:06:57.175597+0000 mgr.y (mgr.24428) 662 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:06:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:58 smithi151 bash[23756]: cluster 2024-03-04T16:06:57.195952+0000 mgr.y (mgr.24428) 663 : cluster 0 pgmap v1102: 172 pgs: 172 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-04T16:06:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:58 smithi151 bash[23756]: cluster 2024-03-04T16:06:57.325802+0000 mon.a (mon.0) 3356 : cluster 0 osdmap e683: 8 total, 8 up, 8 in 2024-03-04T16:06:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:58 smithi151 bash[23756]: audit 2024-03-04T16:06:57.338375+0000 mon.a (mon.0) 3357 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:06:58.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:58 smithi117 bash[28843]: audit 2024-03-04T16:06:57.175597+0000 mgr.y (mgr.24428) 662 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:06:58.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:58 smithi117 bash[28843]: cluster 2024-03-04T16:06:57.195952+0000 mgr.y (mgr.24428) 663 : cluster 0 pgmap v1102: 172 pgs: 172 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-04T16:06:58.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:58 smithi117 bash[28843]: cluster 2024-03-04T16:06:57.325802+0000 mon.a (mon.0) 3356 : cluster 0 osdmap e683: 8 total, 8 up, 8 in 2024-03-04T16:06:58.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:58 smithi117 bash[28843]: audit 2024-03-04T16:06:57.338375+0000 mon.a (mon.0) 3357 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:06:58.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:58 smithi117 bash[21480]: audit 2024-03-04T16:06:57.175597+0000 mgr.y (mgr.24428) 662 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:06:58.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:58 smithi117 bash[21480]: cluster 2024-03-04T16:06:57.195952+0000 mgr.y (mgr.24428) 663 : cluster 0 pgmap v1102: 172 pgs: 172 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-04T16:06:58.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:58 smithi117 bash[21480]: cluster 2024-03-04T16:06:57.325802+0000 mon.a (mon.0) 3356 : cluster 0 osdmap e683: 8 total, 8 up, 8 in 2024-03-04T16:06:58.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:58 smithi117 bash[21480]: audit 2024-03-04T16:06:57.338375+0000 mon.a (mon.0) 3357 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:06:59.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:59 smithi151 bash[23756]: audit 2024-03-04T16:06:58.313638+0000 mon.a (mon.0) 3358 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:59 smithi151 bash[23756]: cluster 2024-03-04T16:06:58.327887+0000 mon.a (mon.0) 3359 : cluster 0 osdmap e684: 8 total, 8 up, 8 in 2024-03-04T16:06:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:06:59 smithi151 bash[23756]: audit 2024-03-04T16:06:58.357725+0000 mon.a (mon.0) 3360 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:06:59.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:59 smithi117 bash[28843]: audit 2024-03-04T16:06:58.313638+0000 mon.a (mon.0) 3358 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:59.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:59 smithi117 bash[28843]: cluster 2024-03-04T16:06:58.327887+0000 mon.a (mon.0) 3359 : cluster 0 osdmap e684: 8 total, 8 up, 8 in 2024-03-04T16:06:59.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:06:59 smithi117 bash[28843]: audit 2024-03-04T16:06:58.357725+0000 mon.a (mon.0) 3360 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:06:59.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:59 smithi117 bash[21480]: audit 2024-03-04T16:06:58.313638+0000 mon.a (mon.0) 3358 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:06:59.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:59 smithi117 bash[21480]: cluster 2024-03-04T16:06:58.327887+0000 mon.a (mon.0) 3359 : cluster 0 osdmap e684: 8 total, 8 up, 8 in 2024-03-04T16:06:59.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:06:59 smithi117 bash[21480]: audit 2024-03-04T16:06:58.357725+0000 mon.a (mon.0) 3360 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:07:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:00 smithi151 bash[23756]: cluster 2024-03-04T16:06:59.196769+0000 mgr.y (mgr.24428) 664 : cluster 0 pgmap v1105: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-04T16:07:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:00 smithi151 bash[23756]: audit 2024-03-04T16:06:59.348106+0000 mon.a (mon.0) 3361 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-134", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:07:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:00 smithi151 bash[23756]: cluster 2024-03-04T16:06:59.357835+0000 mon.a (mon.0) 3362 : cluster 0 osdmap e685: 8 total, 8 up, 8 in 2024-03-04T16:07:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:00 smithi151 bash[23756]: audit 2024-03-04T16:06:59.359842+0000 mon.a (mon.0) 3363 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-134"}]: dispatch 2024-03-04T16:07:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:00 smithi151 bash[23756]: audit 2024-03-04T16:06:59.787239+0000 mon.a (mon.0) 3364 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-134"}]': finished 2024-03-04T16:07:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:00 smithi151 bash[23756]: cluster 2024-03-04T16:06:59.794381+0000 mon.a (mon.0) 3365 : cluster 0 osdmap e686: 8 total, 8 up, 8 in 2024-03-04T16:07:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:00 smithi151 bash[23756]: audit 2024-03-04T16:06:59.796679+0000 mon.a (mon.0) 3366 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-134", "mode": "writeback"}]: dispatch 2024-03-04T16:07:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:00 smithi151 bash[23756]: audit 2024-03-04T16:07:00.119155+0000 mon.c (mon.2) 811 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:07:00.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:00 smithi117 bash[28843]: cluster 2024-03-04T16:06:59.196769+0000 mgr.y (mgr.24428) 664 : cluster 0 pgmap v1105: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-04T16:07:00.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:00 smithi117 bash[28843]: audit 2024-03-04T16:06:59.348106+0000 mon.a (mon.0) 3361 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-134", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:07:00.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:00 smithi117 bash[28843]: cluster 2024-03-04T16:06:59.357835+0000 mon.a (mon.0) 3362 : cluster 0 osdmap e685: 8 total, 8 up, 8 in 2024-03-04T16:07:00.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:00 smithi117 bash[28843]: audit 2024-03-04T16:06:59.359842+0000 mon.a (mon.0) 3363 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-134"}]: dispatch 2024-03-04T16:07:00.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:00 smithi117 bash[28843]: audit 2024-03-04T16:06:59.787239+0000 mon.a (mon.0) 3364 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-134"}]': finished 2024-03-04T16:07:00.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:00 smithi117 bash[28843]: cluster 2024-03-04T16:06:59.794381+0000 mon.a (mon.0) 3365 : cluster 0 osdmap e686: 8 total, 8 up, 8 in 2024-03-04T16:07:00.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:00 smithi117 bash[28843]: audit 2024-03-04T16:06:59.796679+0000 mon.a (mon.0) 3366 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-134", "mode": "writeback"}]: dispatch 2024-03-04T16:07:00.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:00 smithi117 bash[28843]: audit 2024-03-04T16:07:00.119155+0000 mon.c (mon.2) 811 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:07:00.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:00 smithi117 bash[21480]: cluster 2024-03-04T16:06:59.196769+0000 mgr.y (mgr.24428) 664 : cluster 0 pgmap v1105: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-03-04T16:07:00.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:00 smithi117 bash[21480]: audit 2024-03-04T16:06:59.348106+0000 mon.a (mon.0) 3361 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-134", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:07:00.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:00 smithi117 bash[21480]: cluster 2024-03-04T16:06:59.357835+0000 mon.a (mon.0) 3362 : cluster 0 osdmap e685: 8 total, 8 up, 8 in 2024-03-04T16:07:00.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:00 smithi117 bash[21480]: audit 2024-03-04T16:06:59.359842+0000 mon.a (mon.0) 3363 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-134"}]: dispatch 2024-03-04T16:07:00.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:00 smithi117 bash[21480]: audit 2024-03-04T16:06:59.787239+0000 mon.a (mon.0) 3364 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-134"}]': finished 2024-03-04T16:07:00.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:00 smithi117 bash[21480]: cluster 2024-03-04T16:06:59.794381+0000 mon.a (mon.0) 3365 : cluster 0 osdmap e686: 8 total, 8 up, 8 in 2024-03-04T16:07:00.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:00 smithi117 bash[21480]: audit 2024-03-04T16:06:59.796679+0000 mon.a (mon.0) 3366 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-134", "mode": "writeback"}]: dispatch 2024-03-04T16:07:00.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:00 smithi117 bash[21480]: audit 2024-03-04T16:07:00.119155+0000 mon.c (mon.2) 811 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:07:01.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:01 smithi151 bash[23756]: cluster 2024-03-04T16:07:00.787517+0000 mon.a (mon.0) 3367 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:07:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:01 smithi151 bash[23756]: audit 2024-03-04T16:07:00.789968+0000 mon.a (mon.0) 3368 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-134", "mode": "writeback"}]': finished 2024-03-04T16:07:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:01 smithi151 bash[23756]: cluster 2024-03-04T16:07:00.796926+0000 mon.a (mon.0) 3369 : cluster 0 osdmap e687: 8 total, 8 up, 8 in 2024-03-04T16:07:01.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:01 smithi151 bash[23756]: audit 2024-03-04T16:07:00.896571+0000 mon.a (mon.0) 3370 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:07:01.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:01 smithi117 bash[21480]: cluster 2024-03-04T16:07:00.787517+0000 mon.a (mon.0) 3367 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:07:01.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:01 smithi117 bash[21480]: audit 2024-03-04T16:07:00.789968+0000 mon.a (mon.0) 3368 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-134", "mode": "writeback"}]': finished 2024-03-04T16:07:01.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:01 smithi117 bash[21480]: cluster 2024-03-04T16:07:00.796926+0000 mon.a (mon.0) 3369 : cluster 0 osdmap e687: 8 total, 8 up, 8 in 2024-03-04T16:07:01.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:01 smithi117 bash[21480]: audit 2024-03-04T16:07:00.896571+0000 mon.a (mon.0) 3370 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:07:01.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:01 smithi117 bash[28843]: cluster 2024-03-04T16:07:00.787517+0000 mon.a (mon.0) 3367 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:07:01.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:01 smithi117 bash[28843]: audit 2024-03-04T16:07:00.789968+0000 mon.a (mon.0) 3368 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-134", "mode": "writeback"}]': finished 2024-03-04T16:07:01.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:01 smithi117 bash[28843]: cluster 2024-03-04T16:07:00.796926+0000 mon.a (mon.0) 3369 : cluster 0 osdmap e687: 8 total, 8 up, 8 in 2024-03-04T16:07:01.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:01 smithi117 bash[28843]: audit 2024-03-04T16:07:00.896571+0000 mon.a (mon.0) 3370 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:07:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:02 smithi151 bash[23756]: cluster 2024-03-04T16:07:01.197585+0000 mgr.y (mgr.24428) 665 : cluster 0 pgmap v1109: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:07:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:02 smithi151 bash[23756]: audit 2024-03-04T16:07:01.801275+0000 mon.a (mon.0) 3371 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:07:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:02 smithi151 bash[23756]: cluster 2024-03-04T16:07:01.809771+0000 mon.a (mon.0) 3372 : cluster 0 osdmap e688: 8 total, 8 up, 8 in 2024-03-04T16:07:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:02 smithi151 bash[23756]: audit 2024-03-04T16:07:01.810643+0000 mon.a (mon.0) 3373 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-134"}]: dispatch 2024-03-04T16:07:02.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:02 smithi117 bash[21480]: cluster 2024-03-04T16:07:01.197585+0000 mgr.y (mgr.24428) 665 : cluster 0 pgmap v1109: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:07:02.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:02 smithi117 bash[21480]: audit 2024-03-04T16:07:01.801275+0000 mon.a (mon.0) 3371 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:07:02.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:02 smithi117 bash[21480]: cluster 2024-03-04T16:07:01.809771+0000 mon.a (mon.0) 3372 : cluster 0 osdmap e688: 8 total, 8 up, 8 in 2024-03-04T16:07:02.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:02 smithi117 bash[21480]: audit 2024-03-04T16:07:01.810643+0000 mon.a (mon.0) 3373 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-134"}]: dispatch 2024-03-04T16:07:02.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:02 smithi117 bash[28843]: cluster 2024-03-04T16:07:01.197585+0000 mgr.y (mgr.24428) 665 : cluster 0 pgmap v1109: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 730 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:07:02.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:02 smithi117 bash[28843]: audit 2024-03-04T16:07:01.801275+0000 mon.a (mon.0) 3371 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:07:02.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:02 smithi117 bash[28843]: cluster 2024-03-04T16:07:01.809771+0000 mon.a (mon.0) 3372 : cluster 0 osdmap e688: 8 total, 8 up, 8 in 2024-03-04T16:07:02.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:02 smithi117 bash[28843]: audit 2024-03-04T16:07:01.810643+0000 mon.a (mon.0) 3373 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-134"}]: dispatch 2024-03-04T16:07:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:03 smithi151 bash[23756]: cluster 2024-03-04T16:07:02.801555+0000 mon.a (mon.0) 3374 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:07:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:03 smithi151 bash[23756]: audit 2024-03-04T16:07:02.804021+0000 mon.a (mon.0) 3375 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-134"}]': finished 2024-03-04T16:07:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:03 smithi151 bash[23756]: cluster 2024-03-04T16:07:02.812101+0000 mon.a (mon.0) 3376 : cluster 0 osdmap e689: 8 total, 8 up, 8 in 2024-03-04T16:07:03.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:03 smithi117 bash[21480]: cluster 2024-03-04T16:07:02.801555+0000 mon.a (mon.0) 3374 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:07:03.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:03 smithi117 bash[21480]: audit 2024-03-04T16:07:02.804021+0000 mon.a (mon.0) 3375 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-134"}]': finished 2024-03-04T16:07:03.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:03 smithi117 bash[21480]: cluster 2024-03-04T16:07:02.812101+0000 mon.a (mon.0) 3376 : cluster 0 osdmap e689: 8 total, 8 up, 8 in 2024-03-04T16:07:03.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:03 smithi117 bash[28843]: cluster 2024-03-04T16:07:02.801555+0000 mon.a (mon.0) 3374 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:07:03.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:03 smithi117 bash[28843]: audit 2024-03-04T16:07:02.804021+0000 mon.a (mon.0) 3375 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-134"}]': finished 2024-03-04T16:07:03.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:03 smithi117 bash[28843]: cluster 2024-03-04T16:07:02.812101+0000 mon.a (mon.0) 3376 : cluster 0 osdmap e689: 8 total, 8 up, 8 in 2024-03-04T16:07:04.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:04 smithi151 bash[23756]: cluster 2024-03-04T16:07:03.198462+0000 mgr.y (mgr.24428) 666 : cluster 0 pgmap v1112: 204 pgs: 204 active+clean; 582 KiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-03-04T16:07:04.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:04 smithi151 bash[23756]: cluster 2024-03-04T16:07:03.803911+0000 mon.a (mon.0) 3377 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:04.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:04 smithi151 bash[23756]: cluster 2024-03-04T16:07:03.825634+0000 mon.a (mon.0) 3378 : cluster 0 osdmap e690: 8 total, 8 up, 8 in 2024-03-04T16:07:04.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:04 smithi117 bash[21480]: cluster 2024-03-04T16:07:03.198462+0000 mgr.y (mgr.24428) 666 : cluster 0 pgmap v1112: 204 pgs: 204 active+clean; 582 KiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-03-04T16:07:04.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:04 smithi117 bash[21480]: cluster 2024-03-04T16:07:03.803911+0000 mon.a (mon.0) 3377 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:04.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:04 smithi117 bash[21480]: cluster 2024-03-04T16:07:03.825634+0000 mon.a (mon.0) 3378 : cluster 0 osdmap e690: 8 total, 8 up, 8 in 2024-03-04T16:07:04.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:04 smithi117 bash[28843]: cluster 2024-03-04T16:07:03.198462+0000 mgr.y (mgr.24428) 666 : cluster 0 pgmap v1112: 204 pgs: 204 active+clean; 582 KiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.3 KiB/s rd, 2.3 KiB/s wr, 3 op/s 2024-03-04T16:07:04.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:04 smithi117 bash[28843]: cluster 2024-03-04T16:07:03.803911+0000 mon.a (mon.0) 3377 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:04.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:04 smithi117 bash[28843]: cluster 2024-03-04T16:07:03.825634+0000 mon.a (mon.0) 3378 : cluster 0 osdmap e690: 8 total, 8 up, 8 in 2024-03-04T16:07:06.092 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:05 smithi117 bash[21480]: cluster 2024-03-04T16:07:04.828566+0000 mon.a (mon.0) 3379 : cluster 0 osdmap e691: 8 total, 8 up, 8 in 2024-03-04T16:07:06.093 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:05 smithi117 bash[21480]: audit 2024-03-04T16:07:04.841318+0000 mon.a (mon.0) 3380 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:07:06.093 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:05 smithi117 bash[21480]: cluster 2024-03-04T16:07:05.199441+0000 mgr.y (mgr.24428) 667 : cluster 0 pgmap v1115: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T16:07:06.093 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:05 smithi117 bash[28843]: cluster 2024-03-04T16:07:04.828566+0000 mon.a (mon.0) 3379 : cluster 0 osdmap e691: 8 total, 8 up, 8 in 2024-03-04T16:07:06.093 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:05 smithi117 bash[28843]: audit 2024-03-04T16:07:04.841318+0000 mon.a (mon.0) 3380 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:07:06.093 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:05 smithi117 bash[28843]: cluster 2024-03-04T16:07:05.199441+0000 mgr.y (mgr.24428) 667 : cluster 0 pgmap v1115: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T16:07:06.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:05 smithi151 bash[23756]: cluster 2024-03-04T16:07:04.828566+0000 mon.a (mon.0) 3379 : cluster 0 osdmap e691: 8 total, 8 up, 8 in 2024-03-04T16:07:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:05 smithi151 bash[23756]: audit 2024-03-04T16:07:04.841318+0000 mon.a (mon.0) 3380 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:07:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:05 smithi151 bash[23756]: cluster 2024-03-04T16:07:05.199441+0000 mgr.y (mgr.24428) 667 : cluster 0 pgmap v1115: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 735 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-03-04T16:07:06.567 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:07:06 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:07:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:07:07.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:06 smithi151 bash[23756]: audit 2024-03-04T16:07:05.828957+0000 mon.a (mon.0) 3381 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:07:07.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:06 smithi151 bash[23756]: cluster 2024-03-04T16:07:05.842500+0000 mon.a (mon.0) 3382 : cluster 0 osdmap e692: 8 total, 8 up, 8 in 2024-03-04T16:07:07.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:06 smithi151 bash[23756]: audit 2024-03-04T16:07:05.867543+0000 mon.a (mon.0) 3383 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:07:07.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:06 smithi117 bash[21480]: audit 2024-03-04T16:07:05.828957+0000 mon.a (mon.0) 3381 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:07:07.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:06 smithi117 bash[21480]: cluster 2024-03-04T16:07:05.842500+0000 mon.a (mon.0) 3382 : cluster 0 osdmap e692: 8 total, 8 up, 8 in 2024-03-04T16:07:07.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:06 smithi117 bash[21480]: audit 2024-03-04T16:07:05.867543+0000 mon.a (mon.0) 3383 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:07:07.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:06 smithi117 bash[28843]: audit 2024-03-04T16:07:05.828957+0000 mon.a (mon.0) 3381 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:07:07.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:06 smithi117 bash[28843]: cluster 2024-03-04T16:07:05.842500+0000 mon.a (mon.0) 3382 : cluster 0 osdmap e692: 8 total, 8 up, 8 in 2024-03-04T16:07:07.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:06 smithi117 bash[28843]: audit 2024-03-04T16:07:05.867543+0000 mon.a (mon.0) 3383 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:07:07.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:07:07 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:07:08.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:07 smithi151 bash[23756]: audit 2024-03-04T16:07:06.844998+0000 mon.a (mon.0) 3384 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-136", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:07:08.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:07 smithi151 bash[23756]: cluster 2024-03-04T16:07:06.853737+0000 mon.a (mon.0) 3385 : cluster 0 osdmap e693: 8 total, 8 up, 8 in 2024-03-04T16:07:08.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:07 smithi151 bash[23756]: audit 2024-03-04T16:07:06.857127+0000 mon.a (mon.0) 3386 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-136"}]: dispatch 2024-03-04T16:07:08.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:07 smithi151 bash[23756]: audit 2024-03-04T16:07:07.186921+0000 mgr.y (mgr.24428) 668 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:07:08.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:07 smithi151 bash[23756]: cluster 2024-03-04T16:07:07.200299+0000 mgr.y (mgr.24428) 669 : cluster 0 pgmap v1118: 204 pgs: 204 active+clean; 582 KiB data, 736 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:08.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:07 smithi117 bash[21480]: audit 2024-03-04T16:07:06.844998+0000 mon.a (mon.0) 3384 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-136", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:07:08.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:07 smithi117 bash[21480]: cluster 2024-03-04T16:07:06.853737+0000 mon.a (mon.0) 3385 : cluster 0 osdmap e693: 8 total, 8 up, 8 in 2024-03-04T16:07:08.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:07 smithi117 bash[21480]: audit 2024-03-04T16:07:06.857127+0000 mon.a (mon.0) 3386 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-136"}]: dispatch 2024-03-04T16:07:08.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:07 smithi117 bash[21480]: audit 2024-03-04T16:07:07.186921+0000 mgr.y (mgr.24428) 668 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:07:08.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:07 smithi117 bash[21480]: cluster 2024-03-04T16:07:07.200299+0000 mgr.y (mgr.24428) 669 : cluster 0 pgmap v1118: 204 pgs: 204 active+clean; 582 KiB data, 736 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:08.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:07 smithi117 bash[28843]: audit 2024-03-04T16:07:06.844998+0000 mon.a (mon.0) 3384 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-136", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:07:08.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:07 smithi117 bash[28843]: cluster 2024-03-04T16:07:06.853737+0000 mon.a (mon.0) 3385 : cluster 0 osdmap e693: 8 total, 8 up, 8 in 2024-03-04T16:07:08.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:07 smithi117 bash[28843]: audit 2024-03-04T16:07:06.857127+0000 mon.a (mon.0) 3386 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-136"}]: dispatch 2024-03-04T16:07:08.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:07 smithi117 bash[28843]: audit 2024-03-04T16:07:07.186921+0000 mgr.y (mgr.24428) 668 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:07:08.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:07 smithi117 bash[28843]: cluster 2024-03-04T16:07:07.200299+0000 mgr.y (mgr.24428) 669 : cluster 0 pgmap v1118: 204 pgs: 204 active+clean; 582 KiB data, 736 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:09.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:08 smithi151 bash[23756]: audit 2024-03-04T16:07:07.856027+0000 mon.a (mon.0) 3387 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-136"}]': finished 2024-03-04T16:07:09.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:08 smithi151 bash[23756]: cluster 2024-03-04T16:07:07.866245+0000 mon.a (mon.0) 3388 : cluster 0 osdmap e694: 8 total, 8 up, 8 in 2024-03-04T16:07:09.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:08 smithi151 bash[23756]: audit 2024-03-04T16:07:07.867804+0000 mon.a (mon.0) 3389 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-136", "mode": "writeback"}]: dispatch 2024-03-04T16:07:09.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:08 smithi117 bash[28843]: audit 2024-03-04T16:07:07.856027+0000 mon.a (mon.0) 3387 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-136"}]': finished 2024-03-04T16:07:09.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:08 smithi117 bash[28843]: cluster 2024-03-04T16:07:07.866245+0000 mon.a (mon.0) 3388 : cluster 0 osdmap e694: 8 total, 8 up, 8 in 2024-03-04T16:07:09.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:08 smithi117 bash[28843]: audit 2024-03-04T16:07:07.867804+0000 mon.a (mon.0) 3389 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-136", "mode": "writeback"}]: dispatch 2024-03-04T16:07:09.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:08 smithi117 bash[21480]: audit 2024-03-04T16:07:07.856027+0000 mon.a (mon.0) 3387 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-136"}]': finished 2024-03-04T16:07:09.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:08 smithi117 bash[21480]: cluster 2024-03-04T16:07:07.866245+0000 mon.a (mon.0) 3388 : cluster 0 osdmap e694: 8 total, 8 up, 8 in 2024-03-04T16:07:09.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:08 smithi117 bash[21480]: audit 2024-03-04T16:07:07.867804+0000 mon.a (mon.0) 3389 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-136", "mode": "writeback"}]: dispatch 2024-03-04T16:07:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:09 smithi151 bash[23756]: cluster 2024-03-04T16:07:08.856443+0000 mon.a (mon.0) 3390 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:07:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:09 smithi151 bash[23756]: audit 2024-03-04T16:07:08.858940+0000 mon.a (mon.0) 3391 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-136", "mode": "writeback"}]': finished 2024-03-04T16:07:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:09 smithi151 bash[23756]: cluster 2024-03-04T16:07:08.868229+0000 mon.a (mon.0) 3392 : cluster 0 osdmap e695: 8 total, 8 up, 8 in 2024-03-04T16:07:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:09 smithi151 bash[23756]: audit 2024-03-04T16:07:09.007272+0000 mon.a (mon.0) 3393 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:07:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:09 smithi151 bash[23756]: cluster 2024-03-04T16:07:09.201114+0000 mgr.y (mgr.24428) 670 : cluster 0 pgmap v1121: 204 pgs: 204 active+clean; 582 KiB data, 736 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:10.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:09 smithi151 bash[23756]: cluster 2024-03-04T16:07:09.787829+0000 mon.a (mon.0) 3394 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:10.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:09 smithi117 bash[21480]: cluster 2024-03-04T16:07:08.856443+0000 mon.a (mon.0) 3390 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:07:10.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:09 smithi117 bash[21480]: audit 2024-03-04T16:07:08.858940+0000 mon.a (mon.0) 3391 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-136", "mode": "writeback"}]': finished 2024-03-04T16:07:10.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:09 smithi117 bash[21480]: cluster 2024-03-04T16:07:08.868229+0000 mon.a (mon.0) 3392 : cluster 0 osdmap e695: 8 total, 8 up, 8 in 2024-03-04T16:07:10.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:09 smithi117 bash[21480]: audit 2024-03-04T16:07:09.007272+0000 mon.a (mon.0) 3393 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:07:10.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:09 smithi117 bash[21480]: cluster 2024-03-04T16:07:09.201114+0000 mgr.y (mgr.24428) 670 : cluster 0 pgmap v1121: 204 pgs: 204 active+clean; 582 KiB data, 736 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:10.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:09 smithi117 bash[21480]: cluster 2024-03-04T16:07:09.787829+0000 mon.a (mon.0) 3394 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:10.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:09 smithi117 bash[28843]: cluster 2024-03-04T16:07:08.856443+0000 mon.a (mon.0) 3390 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:07:10.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:09 smithi117 bash[28843]: audit 2024-03-04T16:07:08.858940+0000 mon.a (mon.0) 3391 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-136", "mode": "writeback"}]': finished 2024-03-04T16:07:10.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:09 smithi117 bash[28843]: cluster 2024-03-04T16:07:08.868229+0000 mon.a (mon.0) 3392 : cluster 0 osdmap e695: 8 total, 8 up, 8 in 2024-03-04T16:07:10.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:09 smithi117 bash[28843]: audit 2024-03-04T16:07:09.007272+0000 mon.a (mon.0) 3393 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:07:10.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:09 smithi117 bash[28843]: cluster 2024-03-04T16:07:09.201114+0000 mgr.y (mgr.24428) 670 : cluster 0 pgmap v1121: 204 pgs: 204 active+clean; 582 KiB data, 736 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:10.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:09 smithi117 bash[28843]: cluster 2024-03-04T16:07:09.787829+0000 mon.a (mon.0) 3394 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:11.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:10 smithi151 bash[23756]: audit 2024-03-04T16:07:09.886348+0000 mon.a (mon.0) 3395 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:07:11.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:10 smithi151 bash[23756]: cluster 2024-03-04T16:07:09.894681+0000 mon.a (mon.0) 3396 : cluster 0 osdmap e696: 8 total, 8 up, 8 in 2024-03-04T16:07:11.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:10 smithi151 bash[23756]: audit 2024-03-04T16:07:09.896243+0000 mon.a (mon.0) 3397 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-136"}]: dispatch 2024-03-04T16:07:11.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:10 smithi117 bash[21480]: audit 2024-03-04T16:07:09.886348+0000 mon.a (mon.0) 3395 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:07:11.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:10 smithi117 bash[21480]: cluster 2024-03-04T16:07:09.894681+0000 mon.a (mon.0) 3396 : cluster 0 osdmap e696: 8 total, 8 up, 8 in 2024-03-04T16:07:11.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:10 smithi117 bash[21480]: audit 2024-03-04T16:07:09.896243+0000 mon.a (mon.0) 3397 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-136"}]: dispatch 2024-03-04T16:07:11.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:10 smithi117 bash[28843]: audit 2024-03-04T16:07:09.886348+0000 mon.a (mon.0) 3395 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:07:11.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:10 smithi117 bash[28843]: cluster 2024-03-04T16:07:09.894681+0000 mon.a (mon.0) 3396 : cluster 0 osdmap e696: 8 total, 8 up, 8 in 2024-03-04T16:07:11.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:10 smithi117 bash[28843]: audit 2024-03-04T16:07:09.896243+0000 mon.a (mon.0) 3397 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-136"}]: dispatch 2024-03-04T16:07:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:11 smithi151 bash[23756]: cluster 2024-03-04T16:07:10.886836+0000 mon.a (mon.0) 3398 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:07:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:11 smithi151 bash[23756]: audit 2024-03-04T16:07:10.889173+0000 mon.a (mon.0) 3399 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-136"}]': finished 2024-03-04T16:07:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:11 smithi151 bash[23756]: cluster 2024-03-04T16:07:10.895863+0000 mon.a (mon.0) 3400 : cluster 0 osdmap e697: 8 total, 8 up, 8 in 2024-03-04T16:07:12.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:11 smithi151 bash[23756]: cluster 2024-03-04T16:07:11.201914+0000 mgr.y (mgr.24428) 671 : cluster 0 pgmap v1124: 204 pgs: 204 active+clean; 582 KiB data, 736 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:07:12.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:11 smithi117 bash[21480]: cluster 2024-03-04T16:07:10.886836+0000 mon.a (mon.0) 3398 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:07:12.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:11 smithi117 bash[21480]: audit 2024-03-04T16:07:10.889173+0000 mon.a (mon.0) 3399 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-136"}]': finished 2024-03-04T16:07:12.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:11 smithi117 bash[21480]: cluster 2024-03-04T16:07:10.895863+0000 mon.a (mon.0) 3400 : cluster 0 osdmap e697: 8 total, 8 up, 8 in 2024-03-04T16:07:12.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:11 smithi117 bash[21480]: cluster 2024-03-04T16:07:11.201914+0000 mgr.y (mgr.24428) 671 : cluster 0 pgmap v1124: 204 pgs: 204 active+clean; 582 KiB data, 736 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:07:12.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:11 smithi117 bash[28843]: cluster 2024-03-04T16:07:10.886836+0000 mon.a (mon.0) 3398 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:07:12.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:11 smithi117 bash[28843]: audit 2024-03-04T16:07:10.889173+0000 mon.a (mon.0) 3399 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-136"}]': finished 2024-03-04T16:07:12.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:11 smithi117 bash[28843]: cluster 2024-03-04T16:07:10.895863+0000 mon.a (mon.0) 3400 : cluster 0 osdmap e697: 8 total, 8 up, 8 in 2024-03-04T16:07:12.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:11 smithi117 bash[28843]: cluster 2024-03-04T16:07:11.201914+0000 mgr.y (mgr.24428) 671 : cluster 0 pgmap v1124: 204 pgs: 204 active+clean; 582 KiB data, 736 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:07:13.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:12 smithi117 bash[21480]: cluster 2024-03-04T16:07:11.922251+0000 mon.a (mon.0) 3401 : cluster 0 osdmap e698: 8 total, 8 up, 8 in 2024-03-04T16:07:13.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:12 smithi117 bash[28843]: cluster 2024-03-04T16:07:11.922251+0000 mon.a (mon.0) 3401 : cluster 0 osdmap e698: 8 total, 8 up, 8 in 2024-03-04T16:07:13.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:12 smithi151 bash[23756]: cluster 2024-03-04T16:07:11.922251+0000 mon.a (mon.0) 3401 : cluster 0 osdmap e698: 8 total, 8 up, 8 in 2024-03-04T16:07:14.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:13 smithi117 bash[21480]: cluster 2024-03-04T16:07:12.927063+0000 mon.a (mon.0) 3402 : cluster 0 osdmap e699: 8 total, 8 up, 8 in 2024-03-04T16:07:14.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:13 smithi117 bash[21480]: audit 2024-03-04T16:07:12.940845+0000 mon.a (mon.0) 3403 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:07:14.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:13 smithi117 bash[21480]: cluster 2024-03-04T16:07:13.202877+0000 mgr.y (mgr.24428) 672 : cluster 0 pgmap v1127: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-04T16:07:14.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:13 smithi117 bash[21480]: audit 2024-03-04T16:07:13.917434+0000 mon.a (mon.0) 3404 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:07:14.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:13 smithi117 bash[21480]: cluster 2024-03-04T16:07:13.925198+0000 mon.a (mon.0) 3405 : cluster 0 osdmap e700: 8 total, 8 up, 8 in 2024-03-04T16:07:14.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:13 smithi117 bash[21480]: audit 2024-03-04T16:07:13.935374+0000 mon.a (mon.0) 3406 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:07:14.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:13 smithi117 bash[28843]: cluster 2024-03-04T16:07:12.927063+0000 mon.a (mon.0) 3402 : cluster 0 osdmap e699: 8 total, 8 up, 8 in 2024-03-04T16:07:14.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:13 smithi117 bash[28843]: audit 2024-03-04T16:07:12.940845+0000 mon.a (mon.0) 3403 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:07:14.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:13 smithi117 bash[28843]: cluster 2024-03-04T16:07:13.202877+0000 mgr.y (mgr.24428) 672 : cluster 0 pgmap v1127: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-04T16:07:14.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:13 smithi117 bash[28843]: audit 2024-03-04T16:07:13.917434+0000 mon.a (mon.0) 3404 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:07:14.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:13 smithi117 bash[28843]: cluster 2024-03-04T16:07:13.925198+0000 mon.a (mon.0) 3405 : cluster 0 osdmap e700: 8 total, 8 up, 8 in 2024-03-04T16:07:14.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:13 smithi117 bash[28843]: audit 2024-03-04T16:07:13.935374+0000 mon.a (mon.0) 3406 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:07:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:13 smithi151 bash[23756]: cluster 2024-03-04T16:07:12.927063+0000 mon.a (mon.0) 3402 : cluster 0 osdmap e699: 8 total, 8 up, 8 in 2024-03-04T16:07:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:13 smithi151 bash[23756]: audit 2024-03-04T16:07:12.940845+0000 mon.a (mon.0) 3403 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:07:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:13 smithi151 bash[23756]: cluster 2024-03-04T16:07:13.202877+0000 mgr.y (mgr.24428) 672 : cluster 0 pgmap v1127: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-04T16:07:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:13 smithi151 bash[23756]: audit 2024-03-04T16:07:13.917434+0000 mon.a (mon.0) 3404 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:07:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:13 smithi151 bash[23756]: cluster 2024-03-04T16:07:13.925198+0000 mon.a (mon.0) 3405 : cluster 0 osdmap e700: 8 total, 8 up, 8 in 2024-03-04T16:07:14.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:13 smithi151 bash[23756]: audit 2024-03-04T16:07:13.935374+0000 mon.a (mon.0) 3406 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:07:15.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:14 smithi117 bash[21480]: cluster 2024-03-04T16:07:14.789000+0000 mon.a (mon.0) 3407 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:15.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:14 smithi117 bash[21480]: audit 2024-03-04T16:07:14.920306+0000 mon.a (mon.0) 3408 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-138", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:07:15.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:14 smithi117 bash[21480]: cluster 2024-03-04T16:07:14.927017+0000 mon.a (mon.0) 3409 : cluster 0 osdmap e701: 8 total, 8 up, 8 in 2024-03-04T16:07:15.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:14 smithi117 bash[21480]: audit 2024-03-04T16:07:14.927831+0000 mon.a (mon.0) 3410 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:07:15.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:14 smithi117 bash[28843]: cluster 2024-03-04T16:07:14.789000+0000 mon.a (mon.0) 3407 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:15.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:14 smithi117 bash[28843]: audit 2024-03-04T16:07:14.920306+0000 mon.a (mon.0) 3408 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-138", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:07:15.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:14 smithi117 bash[28843]: cluster 2024-03-04T16:07:14.927017+0000 mon.a (mon.0) 3409 : cluster 0 osdmap e701: 8 total, 8 up, 8 in 2024-03-04T16:07:15.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:14 smithi117 bash[28843]: audit 2024-03-04T16:07:14.927831+0000 mon.a (mon.0) 3410 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:07:15.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:14 smithi151 bash[23756]: cluster 2024-03-04T16:07:14.789000+0000 mon.a (mon.0) 3407 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:15.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:14 smithi151 bash[23756]: audit 2024-03-04T16:07:14.920306+0000 mon.a (mon.0) 3408 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-138", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:07:15.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:14 smithi151 bash[23756]: cluster 2024-03-04T16:07:14.927017+0000 mon.a (mon.0) 3409 : cluster 0 osdmap e701: 8 total, 8 up, 8 in 2024-03-04T16:07:15.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:14 smithi151 bash[23756]: audit 2024-03-04T16:07:14.927831+0000 mon.a (mon.0) 3410 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-138","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:07:16.317 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:07:16 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:07:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:07:16.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:15 smithi117 bash[21480]: audit 2024-03-04T16:07:15.151264+0000 mon.c (mon.2) 812 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:07:16.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:15 smithi117 bash[21480]: cluster 2024-03-04T16:07:15.203689+0000 mgr.y (mgr.24428) 673 : cluster 0 pgmap v1130: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-04T16:07:16.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:15 smithi117 bash[21480]: audit 2024-03-04T16:07:15.923237+0000 mon.a (mon.0) 3411 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-138","var": "hit_set_count","val": "2"}]': finished 2024-03-04T16:07:16.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:15 smithi117 bash[21480]: cluster 2024-03-04T16:07:15.931995+0000 mon.a (mon.0) 3412 : cluster 0 osdmap e702: 8 total, 8 up, 8 in 2024-03-04T16:07:16.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:15 smithi117 bash[21480]: audit 2024-03-04T16:07:15.932728+0000 mon.a (mon.0) 3413 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:07:16.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:15 smithi117 bash[28843]: audit 2024-03-04T16:07:15.151264+0000 mon.c (mon.2) 812 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:07:16.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:15 smithi117 bash[28843]: cluster 2024-03-04T16:07:15.203689+0000 mgr.y (mgr.24428) 673 : cluster 0 pgmap v1130: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-04T16:07:16.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:15 smithi117 bash[28843]: audit 2024-03-04T16:07:15.923237+0000 mon.a (mon.0) 3411 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-138","var": "hit_set_count","val": "2"}]': finished 2024-03-04T16:07:16.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:15 smithi117 bash[28843]: cluster 2024-03-04T16:07:15.931995+0000 mon.a (mon.0) 3412 : cluster 0 osdmap e702: 8 total, 8 up, 8 in 2024-03-04T16:07:16.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:15 smithi117 bash[28843]: audit 2024-03-04T16:07:15.932728+0000 mon.a (mon.0) 3413 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:07:16.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:15 smithi151 bash[23756]: audit 2024-03-04T16:07:15.151264+0000 mon.c (mon.2) 812 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:07:16.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:15 smithi151 bash[23756]: cluster 2024-03-04T16:07:15.203689+0000 mgr.y (mgr.24428) 673 : cluster 0 pgmap v1130: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 742 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 976 KiB/s wr, 1 op/s 2024-03-04T16:07:16.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:15 smithi151 bash[23756]: audit 2024-03-04T16:07:15.923237+0000 mon.a (mon.0) 3411 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-138","var": "hit_set_count","val": "2"}]': finished 2024-03-04T16:07:16.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:15 smithi151 bash[23756]: cluster 2024-03-04T16:07:15.931995+0000 mon.a (mon.0) 3412 : cluster 0 osdmap e702: 8 total, 8 up, 8 in 2024-03-04T16:07:16.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:15 smithi151 bash[23756]: audit 2024-03-04T16:07:15.932728+0000 mon.a (mon.0) 3413 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-138","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:07:17.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:07:17 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:07:18.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:17 smithi117 bash[21480]: audit 2024-03-04T16:07:16.925767+0000 mon.a (mon.0) 3414 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-138","var": "hit_set_period","val": "600"}]': finished 2024-03-04T16:07:18.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:17 smithi117 bash[21480]: cluster 2024-03-04T16:07:16.934984+0000 mon.a (mon.0) 3415 : cluster 0 osdmap e703: 8 total, 8 up, 8 in 2024-03-04T16:07:18.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:17 smithi117 bash[21480]: audit 2024-03-04T16:07:16.937312+0000 mon.a (mon.0) 3416 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T16:07:18.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:17 smithi117 bash[21480]: audit 2024-03-04T16:07:17.198353+0000 mgr.y (mgr.24428) 674 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:07:18.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:17 smithi117 bash[21480]: cluster 2024-03-04T16:07:17.204498+0000 mgr.y (mgr.24428) 675 : cluster 0 pgmap v1133: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:18.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:17 smithi117 bash[28843]: audit 2024-03-04T16:07:16.925767+0000 mon.a (mon.0) 3414 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-138","var": "hit_set_period","val": "600"}]': finished 2024-03-04T16:07:18.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:17 smithi117 bash[28843]: cluster 2024-03-04T16:07:16.934984+0000 mon.a (mon.0) 3415 : cluster 0 osdmap e703: 8 total, 8 up, 8 in 2024-03-04T16:07:18.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:17 smithi117 bash[28843]: audit 2024-03-04T16:07:16.937312+0000 mon.a (mon.0) 3416 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T16:07:18.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:17 smithi117 bash[28843]: audit 2024-03-04T16:07:17.198353+0000 mgr.y (mgr.24428) 674 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:07:18.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:17 smithi117 bash[28843]: cluster 2024-03-04T16:07:17.204498+0000 mgr.y (mgr.24428) 675 : cluster 0 pgmap v1133: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:18.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:17 smithi151 bash[23756]: audit 2024-03-04T16:07:16.925767+0000 mon.a (mon.0) 3414 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-138","var": "hit_set_period","val": "600"}]': finished 2024-03-04T16:07:18.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:17 smithi151 bash[23756]: cluster 2024-03-04T16:07:16.934984+0000 mon.a (mon.0) 3415 : cluster 0 osdmap e703: 8 total, 8 up, 8 in 2024-03-04T16:07:18.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:17 smithi151 bash[23756]: audit 2024-03-04T16:07:16.937312+0000 mon.a (mon.0) 3416 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-03-04T16:07:18.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:17 smithi151 bash[23756]: audit 2024-03-04T16:07:17.198353+0000 mgr.y (mgr.24428) 674 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:07:18.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:17 smithi151 bash[23756]: cluster 2024-03-04T16:07:17.204498+0000 mgr.y (mgr.24428) 675 : cluster 0 pgmap v1133: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:19.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:18 smithi117 bash[21480]: audit 2024-03-04T16:07:17.935032+0000 mon.a (mon.0) 3417 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-04T16:07:19.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:18 smithi117 bash[21480]: cluster 2024-03-04T16:07:17.945474+0000 mon.a (mon.0) 3418 : cluster 0 osdmap e704: 8 total, 8 up, 8 in 2024-03-04T16:07:19.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:18 smithi117 bash[28843]: audit 2024-03-04T16:07:17.935032+0000 mon.a (mon.0) 3417 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-04T16:07:19.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:18 smithi117 bash[28843]: cluster 2024-03-04T16:07:17.945474+0000 mon.a (mon.0) 3418 : cluster 0 osdmap e704: 8 total, 8 up, 8 in 2024-03-04T16:07:19.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:18 smithi151 bash[23756]: audit 2024-03-04T16:07:17.935032+0000 mon.a (mon.0) 3417 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-03-04T16:07:19.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:18 smithi151 bash[23756]: cluster 2024-03-04T16:07:17.945474+0000 mon.a (mon.0) 3418 : cluster 0 osdmap e704: 8 total, 8 up, 8 in 2024-03-04T16:07:20.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:19 smithi117 bash[21480]: audit 2024-03-04T16:07:18.989345+0000 mon.a (mon.0) 3419 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:07:20.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:19 smithi117 bash[21480]: audit 2024-03-04T16:07:18.990258+0000 mon.a (mon.0) 3420 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-138"}]: dispatch 2024-03-04T16:07:20.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:19 smithi117 bash[21480]: cluster 2024-03-04T16:07:19.205316+0000 mgr.y (mgr.24428) 676 : cluster 0 pgmap v1135: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:20.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:19 smithi117 bash[21480]: cluster 2024-03-04T16:07:19.790439+0000 mon.a (mon.0) 3421 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:20.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:19 smithi117 bash[28843]: audit 2024-03-04T16:07:18.989345+0000 mon.a (mon.0) 3419 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:07:20.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:19 smithi117 bash[28843]: audit 2024-03-04T16:07:18.990258+0000 mon.a (mon.0) 3420 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-138"}]: dispatch 2024-03-04T16:07:20.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:19 smithi117 bash[28843]: cluster 2024-03-04T16:07:19.205316+0000 mgr.y (mgr.24428) 676 : cluster 0 pgmap v1135: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:20.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:19 smithi117 bash[28843]: cluster 2024-03-04T16:07:19.790439+0000 mon.a (mon.0) 3421 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:20.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:19 smithi151 bash[23756]: audit 2024-03-04T16:07:18.989345+0000 mon.a (mon.0) 3419 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:07:20.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:19 smithi151 bash[23756]: audit 2024-03-04T16:07:18.990258+0000 mon.a (mon.0) 3420 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-138"}]: dispatch 2024-03-04T16:07:20.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:19 smithi151 bash[23756]: cluster 2024-03-04T16:07:19.205316+0000 mgr.y (mgr.24428) 676 : cluster 0 pgmap v1135: 204 pgs: 204 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:20.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:19 smithi151 bash[23756]: cluster 2024-03-04T16:07:19.790439+0000 mon.a (mon.0) 3421 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:21.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:21 smithi151 bash[23756]: audit 2024-03-04T16:07:19.946021+0000 mon.a (mon.0) 3422 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-138"}]': finished 2024-03-04T16:07:21.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:21 smithi151 bash[23756]: cluster 2024-03-04T16:07:19.956304+0000 mon.a (mon.0) 3423 : cluster 0 osdmap e705: 8 total, 8 up, 8 in 2024-03-04T16:07:21.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:21 smithi117 bash[21480]: audit 2024-03-04T16:07:19.946021+0000 mon.a (mon.0) 3422 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-138"}]': finished 2024-03-04T16:07:21.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:21 smithi117 bash[21480]: cluster 2024-03-04T16:07:19.956304+0000 mon.a (mon.0) 3423 : cluster 0 osdmap e705: 8 total, 8 up, 8 in 2024-03-04T16:07:21.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:21 smithi117 bash[28843]: audit 2024-03-04T16:07:19.946021+0000 mon.a (mon.0) 3422 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-138"}]': finished 2024-03-04T16:07:21.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:21 smithi117 bash[28843]: cluster 2024-03-04T16:07:19.956304+0000 mon.a (mon.0) 3423 : cluster 0 osdmap e705: 8 total, 8 up, 8 in 2024-03-04T16:07:22.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:22 smithi151 bash[23756]: cluster 2024-03-04T16:07:20.956322+0000 mon.a (mon.0) 3424 : cluster 0 osdmap e706: 8 total, 8 up, 8 in 2024-03-04T16:07:22.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:22 smithi151 bash[23756]: cluster 2024-03-04T16:07:21.206061+0000 mgr.y (mgr.24428) 677 : cluster 0 pgmap v1138: 172 pgs: 172 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:07:22.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:22 smithi151 bash[23756]: cluster 2024-03-04T16:07:21.966733+0000 mon.a (mon.0) 3425 : cluster 0 osdmap e707: 8 total, 8 up, 8 in 2024-03-04T16:07:22.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:22 smithi151 bash[23756]: audit 2024-03-04T16:07:21.977117+0000 mon.a (mon.0) 3426 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:07:22.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:22 smithi117 bash[21480]: cluster 2024-03-04T16:07:20.956322+0000 mon.a (mon.0) 3424 : cluster 0 osdmap e706: 8 total, 8 up, 8 in 2024-03-04T16:07:22.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:22 smithi117 bash[21480]: cluster 2024-03-04T16:07:21.206061+0000 mgr.y (mgr.24428) 677 : cluster 0 pgmap v1138: 172 pgs: 172 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:07:22.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:22 smithi117 bash[21480]: cluster 2024-03-04T16:07:21.966733+0000 mon.a (mon.0) 3425 : cluster 0 osdmap e707: 8 total, 8 up, 8 in 2024-03-04T16:07:22.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:22 smithi117 bash[21480]: audit 2024-03-04T16:07:21.977117+0000 mon.a (mon.0) 3426 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:07:22.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:22 smithi117 bash[28843]: cluster 2024-03-04T16:07:20.956322+0000 mon.a (mon.0) 3424 : cluster 0 osdmap e706: 8 total, 8 up, 8 in 2024-03-04T16:07:22.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:22 smithi117 bash[28843]: cluster 2024-03-04T16:07:21.206061+0000 mgr.y (mgr.24428) 677 : cluster 0 pgmap v1138: 172 pgs: 172 active+clean; 4.4 MiB data, 726 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:07:22.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:22 smithi117 bash[28843]: cluster 2024-03-04T16:07:21.966733+0000 mon.a (mon.0) 3425 : cluster 0 osdmap e707: 8 total, 8 up, 8 in 2024-03-04T16:07:22.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:22 smithi117 bash[28843]: audit 2024-03-04T16:07:21.977117+0000 mon.a (mon.0) 3426 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:07:24.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:23 smithi117 bash[21480]: audit 2024-03-04T16:07:22.957115+0000 mon.a (mon.0) 3427 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:07:24.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:23 smithi117 bash[21480]: cluster 2024-03-04T16:07:22.967766+0000 mon.a (mon.0) 3428 : cluster 0 osdmap e708: 8 total, 8 up, 8 in 2024-03-04T16:07:24.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:23 smithi117 bash[21480]: audit 2024-03-04T16:07:22.972133+0000 mon.a (mon.0) 3429 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:07:24.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:23 smithi117 bash[21480]: cluster 2024-03-04T16:07:23.206907+0000 mgr.y (mgr.24428) 678 : cluster 0 pgmap v1141: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 727 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:24.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:23 smithi117 bash[28843]: audit 2024-03-04T16:07:22.957115+0000 mon.a (mon.0) 3427 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:07:24.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:23 smithi117 bash[28843]: cluster 2024-03-04T16:07:22.967766+0000 mon.a (mon.0) 3428 : cluster 0 osdmap e708: 8 total, 8 up, 8 in 2024-03-04T16:07:24.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:23 smithi117 bash[28843]: audit 2024-03-04T16:07:22.972133+0000 mon.a (mon.0) 3429 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:07:24.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:23 smithi117 bash[28843]: cluster 2024-03-04T16:07:23.206907+0000 mgr.y (mgr.24428) 678 : cluster 0 pgmap v1141: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 727 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:24.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:23 smithi151 bash[23756]: audit 2024-03-04T16:07:22.957115+0000 mon.a (mon.0) 3427 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:07:24.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:23 smithi151 bash[23756]: cluster 2024-03-04T16:07:22.967766+0000 mon.a (mon.0) 3428 : cluster 0 osdmap e708: 8 total, 8 up, 8 in 2024-03-04T16:07:24.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:23 smithi151 bash[23756]: audit 2024-03-04T16:07:22.972133+0000 mon.a (mon.0) 3429 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:07:24.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:23 smithi151 bash[23756]: cluster 2024-03-04T16:07:23.206907+0000 mgr.y (mgr.24428) 678 : cluster 0 pgmap v1141: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 727 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:25.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:24 smithi117 bash[21480]: audit 2024-03-04T16:07:23.966141+0000 mon.a (mon.0) 3430 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-140", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:07:25.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:24 smithi117 bash[21480]: cluster 2024-03-04T16:07:23.975177+0000 mon.a (mon.0) 3431 : cluster 0 osdmap e709: 8 total, 8 up, 8 in 2024-03-04T16:07:25.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:24 smithi117 bash[21480]: audit 2024-03-04T16:07:23.976272+0000 mon.a (mon.0) 3432 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-04T16:07:25.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:24 smithi117 bash[28843]: audit 2024-03-04T16:07:23.966141+0000 mon.a (mon.0) 3430 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-140", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:07:25.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:24 smithi117 bash[28843]: cluster 2024-03-04T16:07:23.975177+0000 mon.a (mon.0) 3431 : cluster 0 osdmap e709: 8 total, 8 up, 8 in 2024-03-04T16:07:25.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:24 smithi117 bash[28843]: audit 2024-03-04T16:07:23.976272+0000 mon.a (mon.0) 3432 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-04T16:07:25.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:24 smithi151 bash[23756]: audit 2024-03-04T16:07:23.966141+0000 mon.a (mon.0) 3430 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-140", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:07:25.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:24 smithi151 bash[23756]: cluster 2024-03-04T16:07:23.975177+0000 mon.a (mon.0) 3431 : cluster 0 osdmap e709: 8 total, 8 up, 8 in 2024-03-04T16:07:25.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:24 smithi151 bash[23756]: audit 2024-03-04T16:07:23.976272+0000 mon.a (mon.0) 3432 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_count","val": "3"}]: dispatch 2024-03-04T16:07:26.317 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:07:26 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:07:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:07:26.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:25 smithi117 bash[21480]: audit 2024-03-04T16:07:24.968657+0000 mon.a (mon.0) 3433 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_count","val": "3"}]': finished 2024-03-04T16:07:26.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:25 smithi117 bash[21480]: cluster 2024-03-04T16:07:24.977539+0000 mon.a (mon.0) 3434 : cluster 0 osdmap e710: 8 total, 8 up, 8 in 2024-03-04T16:07:26.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:25 smithi117 bash[21480]: audit 2024-03-04T16:07:24.979837+0000 mon.a (mon.0) 3435 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-04T16:07:26.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:25 smithi117 bash[21480]: cluster 2024-03-04T16:07:25.207674+0000 mgr.y (mgr.24428) 679 : cluster 0 pgmap v1144: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 727 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:26.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:25 smithi117 bash[21480]: audit 2024-03-04T16:07:25.971416+0000 mon.a (mon.0) 3436 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_period","val": "3"}]': finished 2024-03-04T16:07:26.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:25 smithi117 bash[21480]: cluster 2024-03-04T16:07:25.979289+0000 mon.a (mon.0) 3437 : cluster 0 osdmap e711: 8 total, 8 up, 8 in 2024-03-04T16:07:26.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:25 smithi117 bash[21480]: audit 2024-03-04T16:07:25.980064+0000 mon.a (mon.0) 3438 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T16:07:26.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:25 smithi117 bash[28843]: audit 2024-03-04T16:07:24.968657+0000 mon.a (mon.0) 3433 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_count","val": "3"}]': finished 2024-03-04T16:07:26.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:25 smithi117 bash[28843]: cluster 2024-03-04T16:07:24.977539+0000 mon.a (mon.0) 3434 : cluster 0 osdmap e710: 8 total, 8 up, 8 in 2024-03-04T16:07:26.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:25 smithi117 bash[28843]: audit 2024-03-04T16:07:24.979837+0000 mon.a (mon.0) 3435 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-04T16:07:26.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:25 smithi117 bash[28843]: cluster 2024-03-04T16:07:25.207674+0000 mgr.y (mgr.24428) 679 : cluster 0 pgmap v1144: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 727 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:26.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:25 smithi117 bash[28843]: audit 2024-03-04T16:07:25.971416+0000 mon.a (mon.0) 3436 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_period","val": "3"}]': finished 2024-03-04T16:07:26.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:25 smithi117 bash[28843]: cluster 2024-03-04T16:07:25.979289+0000 mon.a (mon.0) 3437 : cluster 0 osdmap e711: 8 total, 8 up, 8 in 2024-03-04T16:07:26.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:25 smithi117 bash[28843]: audit 2024-03-04T16:07:25.980064+0000 mon.a (mon.0) 3438 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T16:07:26.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:25 smithi151 bash[23756]: audit 2024-03-04T16:07:24.968657+0000 mon.a (mon.0) 3433 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_count","val": "3"}]': finished 2024-03-04T16:07:26.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:25 smithi151 bash[23756]: cluster 2024-03-04T16:07:24.977539+0000 mon.a (mon.0) 3434 : cluster 0 osdmap e710: 8 total, 8 up, 8 in 2024-03-04T16:07:26.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:25 smithi151 bash[23756]: audit 2024-03-04T16:07:24.979837+0000 mon.a (mon.0) 3435 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_period","val": "3"}]: dispatch 2024-03-04T16:07:26.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:25 smithi151 bash[23756]: cluster 2024-03-04T16:07:25.207674+0000 mgr.y (mgr.24428) 679 : cluster 0 pgmap v1144: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 727 MiB used, 715 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:26.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:25 smithi151 bash[23756]: audit 2024-03-04T16:07:25.971416+0000 mon.a (mon.0) 3436 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_period","val": "3"}]': finished 2024-03-04T16:07:26.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:25 smithi151 bash[23756]: cluster 2024-03-04T16:07:25.979289+0000 mon.a (mon.0) 3437 : cluster 0 osdmap e711: 8 total, 8 up, 8 in 2024-03-04T16:07:26.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:25 smithi151 bash[23756]: audit 2024-03-04T16:07:25.980064+0000 mon.a (mon.0) 3438 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T16:07:27.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:07:27 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:07:28.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:27 smithi117 bash[21480]: audit 2024-03-04T16:07:26.974288+0000 mon.a (mon.0) 3439 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T16:07:28.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:27 smithi117 bash[21480]: cluster 2024-03-04T16:07:26.982383+0000 mon.a (mon.0) 3440 : cluster 0 osdmap e712: 8 total, 8 up, 8 in 2024-03-04T16:07:28.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:27 smithi117 bash[21480]: audit 2024-03-04T16:07:26.983687+0000 mon.a (mon.0) 3441 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-04T16:07:28.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:27 smithi117 bash[21480]: cluster 2024-03-04T16:07:27.208494+0000 mgr.y (mgr.24428) 680 : cluster 0 pgmap v1147: 204 pgs: 204 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:28.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:27 smithi117 bash[21480]: audit 2024-03-04T16:07:27.209701+0000 mgr.y (mgr.24428) 681 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:07:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:27 smithi117 bash[28843]: audit 2024-03-04T16:07:26.974288+0000 mon.a (mon.0) 3439 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T16:07:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:27 smithi117 bash[28843]: cluster 2024-03-04T16:07:26.982383+0000 mon.a (mon.0) 3440 : cluster 0 osdmap e712: 8 total, 8 up, 8 in 2024-03-04T16:07:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:27 smithi117 bash[28843]: audit 2024-03-04T16:07:26.983687+0000 mon.a (mon.0) 3441 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-04T16:07:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:27 smithi117 bash[28843]: cluster 2024-03-04T16:07:27.208494+0000 mgr.y (mgr.24428) 680 : cluster 0 pgmap v1147: 204 pgs: 204 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:28.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:27 smithi117 bash[28843]: audit 2024-03-04T16:07:27.209701+0000 mgr.y (mgr.24428) 681 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:07:28.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:27 smithi151 bash[23756]: audit 2024-03-04T16:07:26.974288+0000 mon.a (mon.0) 3439 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T16:07:28.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:27 smithi151 bash[23756]: cluster 2024-03-04T16:07:26.982383+0000 mon.a (mon.0) 3440 : cluster 0 osdmap e712: 8 total, 8 up, 8 in 2024-03-04T16:07:28.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:27 smithi151 bash[23756]: audit 2024-03-04T16:07:26.983687+0000 mon.a (mon.0) 3441 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-03-04T16:07:28.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:27 smithi151 bash[23756]: cluster 2024-03-04T16:07:27.208494+0000 mgr.y (mgr.24428) 680 : cluster 0 pgmap v1147: 204 pgs: 204 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:28.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:27 smithi151 bash[23756]: audit 2024-03-04T16:07:27.209701+0000 mgr.y (mgr.24428) 681 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:07:29.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:28 smithi117 bash[21480]: audit 2024-03-04T16:07:27.985407+0000 mon.a (mon.0) 3442 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-04T16:07:29.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:28 smithi117 bash[21480]: cluster 2024-03-04T16:07:27.995495+0000 mon.a (mon.0) 3443 : cluster 0 osdmap e713: 8 total, 8 up, 8 in 2024-03-04T16:07:29.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:28 smithi117 bash[28843]: audit 2024-03-04T16:07:27.985407+0000 mon.a (mon.0) 3442 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-04T16:07:29.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:28 smithi117 bash[28843]: cluster 2024-03-04T16:07:27.995495+0000 mon.a (mon.0) 3443 : cluster 0 osdmap e713: 8 total, 8 up, 8 in 2024-03-04T16:07:29.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:28 smithi151 bash[23756]: audit 2024-03-04T16:07:27.985407+0000 mon.a (mon.0) 3442 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-03-04T16:07:29.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:28 smithi151 bash[23756]: cluster 2024-03-04T16:07:27.995495+0000 mon.a (mon.0) 3443 : cluster 0 osdmap e713: 8 total, 8 up, 8 in 2024-03-04T16:07:30.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:29 smithi117 bash[21480]: cluster 2024-03-04T16:07:29.209339+0000 mgr.y (mgr.24428) 682 : cluster 0 pgmap v1149: 204 pgs: 204 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:30.317 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:29 smithi117 bash[28843]: cluster 2024-03-04T16:07:29.209339+0000 mgr.y (mgr.24428) 682 : cluster 0 pgmap v1149: 204 pgs: 204 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:30.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:29 smithi151 bash[23756]: cluster 2024-03-04T16:07:29.209339+0000 mgr.y (mgr.24428) 682 : cluster 0 pgmap v1149: 204 pgs: 204 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:31.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:30 smithi117 bash[21480]: audit 2024-03-04T16:07:30.183688+0000 mon.c (mon.2) 813 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:07:31.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:30 smithi117 bash[28843]: audit 2024-03-04T16:07:30.183688+0000 mon.c (mon.2) 813 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:07:31.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:30 smithi151 bash[23756]: audit 2024-03-04T16:07:30.183688+0000 mon.c (mon.2) 813 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:07:32.317 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:32 smithi117 bash[21480]: cluster 2024-03-04T16:07:31.210132+0000 mgr.y (mgr.24428) 683 : cluster 0 pgmap v1150: 204 pgs: 204 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:07:32.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:32 smithi117 bash[28843]: cluster 2024-03-04T16:07:31.210132+0000 mgr.y (mgr.24428) 683 : cluster 0 pgmap v1150: 204 pgs: 204 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:07:32.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:32 smithi151 bash[23756]: cluster 2024-03-04T16:07:31.210132+0000 mgr.y (mgr.24428) 683 : cluster 0 pgmap v1150: 204 pgs: 204 active+clean; 4.4 MiB data, 761 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:07:34.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:34 smithi117 bash[21480]: cluster 2024-03-04T16:07:33.212257+0000 mgr.y (mgr.24428) 684 : cluster 0 pgmap v1151: 204 pgs: 204 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 4.6 KiB/s wr, 2 op/s 2024-03-04T16:07:34.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:34 smithi117 bash[28843]: cluster 2024-03-04T16:07:33.212257+0000 mgr.y (mgr.24428) 684 : cluster 0 pgmap v1151: 204 pgs: 204 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 4.6 KiB/s wr, 2 op/s 2024-03-04T16:07:34.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:34 smithi151 bash[23756]: cluster 2024-03-04T16:07:33.212257+0000 mgr.y (mgr.24428) 684 : cluster 0 pgmap v1151: 204 pgs: 204 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 4.6 KiB/s wr, 2 op/s 2024-03-04T16:07:36.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:36 smithi117 bash[21480]: cluster 2024-03-04T16:07:35.213088+0000 mgr.y (mgr.24428) 685 : cluster 0 pgmap v1152: 204 pgs: 204 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 621 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-03-04T16:07:36.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:07:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:07:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:07:36.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:36 smithi117 bash[28843]: cluster 2024-03-04T16:07:35.213088+0000 mgr.y (mgr.24428) 685 : cluster 0 pgmap v1152: 204 pgs: 204 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 621 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-03-04T16:07:36.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:36 smithi151 bash[23756]: cluster 2024-03-04T16:07:35.213088+0000 mgr.y (mgr.24428) 685 : cluster 0 pgmap v1152: 204 pgs: 204 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 621 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-03-04T16:07:37.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:07:37 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:07:38.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:38 smithi117 bash[21480]: cluster 2024-03-04T16:07:37.215243+0000 mgr.y (mgr.24428) 686 : cluster 0 pgmap v1153: 204 pgs: 204 active+clean; 4.5 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 7.4 KiB/s wr, 2 op/s 2024-03-04T16:07:38.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:38 smithi117 bash[21480]: audit 2024-03-04T16:07:37.215510+0000 mgr.y (mgr.24428) 687 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:07:38.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:38 smithi117 bash[28843]: cluster 2024-03-04T16:07:37.215243+0000 mgr.y (mgr.24428) 686 : cluster 0 pgmap v1153: 204 pgs: 204 active+clean; 4.5 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 7.4 KiB/s wr, 2 op/s 2024-03-04T16:07:38.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:38 smithi117 bash[28843]: audit 2024-03-04T16:07:37.215510+0000 mgr.y (mgr.24428) 687 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:07:38.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:38 smithi151 bash[23756]: cluster 2024-03-04T16:07:37.215243+0000 mgr.y (mgr.24428) 686 : cluster 0 pgmap v1153: 204 pgs: 204 active+clean; 4.5 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 7.4 KiB/s wr, 2 op/s 2024-03-04T16:07:38.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:38 smithi151 bash[23756]: audit 2024-03-04T16:07:37.215510+0000 mgr.y (mgr.24428) 687 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:07:40.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:40 smithi117 bash[21480]: cluster 2024-03-04T16:07:39.216072+0000 mgr.y (mgr.24428) 688 : cluster 0 pgmap v1154: 204 pgs: 204 active+clean; 4.5 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 912 B/s rd, 6.6 KiB/s wr, 1 op/s 2024-03-04T16:07:40.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:40 smithi117 bash[21480]: audit 2024-03-04T16:07:40.096094+0000 mon.a (mon.0) 3444 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:07:40.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:40 smithi117 bash[21480]: audit 2024-03-04T16:07:40.097080+0000 mon.a (mon.0) 3445 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-140"}]: dispatch 2024-03-04T16:07:40.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:40 smithi117 bash[28843]: cluster 2024-03-04T16:07:39.216072+0000 mgr.y (mgr.24428) 688 : cluster 0 pgmap v1154: 204 pgs: 204 active+clean; 4.5 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 912 B/s rd, 6.6 KiB/s wr, 1 op/s 2024-03-04T16:07:40.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:40 smithi117 bash[28843]: audit 2024-03-04T16:07:40.096094+0000 mon.a (mon.0) 3444 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:07:40.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:40 smithi117 bash[28843]: audit 2024-03-04T16:07:40.097080+0000 mon.a (mon.0) 3445 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-140"}]: dispatch 2024-03-04T16:07:40.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:40 smithi151 bash[23756]: cluster 2024-03-04T16:07:39.216072+0000 mgr.y (mgr.24428) 688 : cluster 0 pgmap v1154: 204 pgs: 204 active+clean; 4.5 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 912 B/s rd, 6.6 KiB/s wr, 1 op/s 2024-03-04T16:07:40.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:40 smithi151 bash[23756]: audit 2024-03-04T16:07:40.096094+0000 mon.a (mon.0) 3444 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:07:40.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:40 smithi151 bash[23756]: audit 2024-03-04T16:07:40.097080+0000 mon.a (mon.0) 3445 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-140"}]: dispatch 2024-03-04T16:07:41.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:41 smithi117 bash[21480]: audit 2024-03-04T16:07:40.284781+0000 mon.a (mon.0) 3446 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-140"}]': finished 2024-03-04T16:07:41.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:41 smithi117 bash[21480]: cluster 2024-03-04T16:07:40.293861+0000 mon.a (mon.0) 3447 : cluster 0 osdmap e714: 8 total, 8 up, 8 in 2024-03-04T16:07:41.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:41 smithi117 bash[28843]: audit 2024-03-04T16:07:40.284781+0000 mon.a (mon.0) 3446 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-140"}]': finished 2024-03-04T16:07:41.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:41 smithi117 bash[28843]: cluster 2024-03-04T16:07:40.293861+0000 mon.a (mon.0) 3447 : cluster 0 osdmap e714: 8 total, 8 up, 8 in 2024-03-04T16:07:41.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:41 smithi151 bash[23756]: audit 2024-03-04T16:07:40.284781+0000 mon.a (mon.0) 3446 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-140"}]': finished 2024-03-04T16:07:41.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:41 smithi151 bash[23756]: cluster 2024-03-04T16:07:40.293861+0000 mon.a (mon.0) 3447 : cluster 0 osdmap e714: 8 total, 8 up, 8 in 2024-03-04T16:07:42.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:42 smithi117 bash[21480]: cluster 2024-03-04T16:07:41.216882+0000 mgr.y (mgr.24428) 689 : cluster 0 pgmap v1156: 204 pgs: 204 active+clean; 4.5 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 7.4 KiB/s wr, 2 op/s 2024-03-04T16:07:42.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:42 smithi117 bash[21480]: cluster 2024-03-04T16:07:41.300322+0000 mon.a (mon.0) 3448 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:42.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:42 smithi117 bash[21480]: cluster 2024-03-04T16:07:41.318093+0000 mon.a (mon.0) 3449 : cluster 0 osdmap e715: 8 total, 8 up, 8 in 2024-03-04T16:07:42.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:42 smithi117 bash[28843]: cluster 2024-03-04T16:07:41.216882+0000 mgr.y (mgr.24428) 689 : cluster 0 pgmap v1156: 204 pgs: 204 active+clean; 4.5 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 7.4 KiB/s wr, 2 op/s 2024-03-04T16:07:42.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:42 smithi117 bash[28843]: cluster 2024-03-04T16:07:41.300322+0000 mon.a (mon.0) 3448 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:42.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:42 smithi117 bash[28843]: cluster 2024-03-04T16:07:41.318093+0000 mon.a (mon.0) 3449 : cluster 0 osdmap e715: 8 total, 8 up, 8 in 2024-03-04T16:07:42.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:42 smithi151 bash[23756]: cluster 2024-03-04T16:07:41.216882+0000 mgr.y (mgr.24428) 689 : cluster 0 pgmap v1156: 204 pgs: 204 active+clean; 4.5 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 7.4 KiB/s wr, 2 op/s 2024-03-04T16:07:42.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:42 smithi151 bash[23756]: cluster 2024-03-04T16:07:41.300322+0000 mon.a (mon.0) 3448 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:42.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:42 smithi151 bash[23756]: cluster 2024-03-04T16:07:41.318093+0000 mon.a (mon.0) 3449 : cluster 0 osdmap e715: 8 total, 8 up, 8 in 2024-03-04T16:07:43.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:43 smithi151 bash[23756]: cluster 2024-03-04T16:07:42.328736+0000 mon.a (mon.0) 3450 : cluster 0 osdmap e716: 8 total, 8 up, 8 in 2024-03-04T16:07:43.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:43 smithi151 bash[23756]: audit 2024-03-04T16:07:42.334856+0000 mon.a (mon.0) 3451 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:07:43.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:43 smithi117 bash[21480]: cluster 2024-03-04T16:07:42.328736+0000 mon.a (mon.0) 3450 : cluster 0 osdmap e716: 8 total, 8 up, 8 in 2024-03-04T16:07:43.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:43 smithi117 bash[21480]: audit 2024-03-04T16:07:42.334856+0000 mon.a (mon.0) 3451 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:07:43.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:43 smithi117 bash[28843]: cluster 2024-03-04T16:07:42.328736+0000 mon.a (mon.0) 3450 : cluster 0 osdmap e716: 8 total, 8 up, 8 in 2024-03-04T16:07:43.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:43 smithi117 bash[28843]: audit 2024-03-04T16:07:42.334856+0000 mon.a (mon.0) 3451 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:07:44.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:44 smithi151 bash[23756]: cluster 2024-03-04T16:07:43.217742+0000 mgr.y (mgr.24428) 690 : cluster 0 pgmap v1159: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:07:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:44 smithi151 bash[23756]: audit 2024-03-04T16:07:43.333725+0000 mon.a (mon.0) 3452 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:07:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:44 smithi151 bash[23756]: cluster 2024-03-04T16:07:43.346859+0000 mon.a (mon.0) 3453 : cluster 0 osdmap e717: 8 total, 8 up, 8 in 2024-03-04T16:07:44.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:44 smithi151 bash[23756]: audit 2024-03-04T16:07:43.441463+0000 mon.a (mon.0) 3454 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:07:44.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:44 smithi117 bash[28843]: cluster 2024-03-04T16:07:43.217742+0000 mgr.y (mgr.24428) 690 : cluster 0 pgmap v1159: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:07:44.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:44 smithi117 bash[28843]: audit 2024-03-04T16:07:43.333725+0000 mon.a (mon.0) 3452 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:07:44.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:44 smithi117 bash[28843]: cluster 2024-03-04T16:07:43.346859+0000 mon.a (mon.0) 3453 : cluster 0 osdmap e717: 8 total, 8 up, 8 in 2024-03-04T16:07:44.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:44 smithi117 bash[28843]: audit 2024-03-04T16:07:43.441463+0000 mon.a (mon.0) 3454 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:07:44.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:44 smithi117 bash[21480]: cluster 2024-03-04T16:07:43.217742+0000 mgr.y (mgr.24428) 690 : cluster 0 pgmap v1159: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:07:44.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:44 smithi117 bash[21480]: audit 2024-03-04T16:07:43.333725+0000 mon.a (mon.0) 3452 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:07:44.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:44 smithi117 bash[21480]: cluster 2024-03-04T16:07:43.346859+0000 mon.a (mon.0) 3453 : cluster 0 osdmap e717: 8 total, 8 up, 8 in 2024-03-04T16:07:44.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:44 smithi117 bash[21480]: audit 2024-03-04T16:07:43.441463+0000 mon.a (mon.0) 3454 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:07:45.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:45 smithi151 bash[23756]: audit 2024-03-04T16:07:44.336358+0000 mon.a (mon.0) 3455 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-142", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:07:45.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:45 smithi151 bash[23756]: cluster 2024-03-04T16:07:44.342681+0000 mon.a (mon.0) 3456 : cluster 0 osdmap e718: 8 total, 8 up, 8 in 2024-03-04T16:07:45.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:45 smithi151 bash[23756]: audit 2024-03-04T16:07:44.344484+0000 mon.a (mon.0) 3457 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-142"}]: dispatch 2024-03-04T16:07:45.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:45 smithi151 bash[23756]: audit 2024-03-04T16:07:45.217474+0000 mon.c (mon.2) 814 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:07:45.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:45 smithi151 bash[23756]: audit 2024-03-04T16:07:45.341339+0000 mon.a (mon.0) 3458 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-142"}]': finished 2024-03-04T16:07:45.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:45 smithi117 bash[21480]: audit 2024-03-04T16:07:44.336358+0000 mon.a (mon.0) 3455 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-142", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:07:45.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:45 smithi117 bash[21480]: cluster 2024-03-04T16:07:44.342681+0000 mon.a (mon.0) 3456 : cluster 0 osdmap e718: 8 total, 8 up, 8 in 2024-03-04T16:07:45.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:45 smithi117 bash[21480]: audit 2024-03-04T16:07:44.344484+0000 mon.a (mon.0) 3457 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-142"}]: dispatch 2024-03-04T16:07:45.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:45 smithi117 bash[21480]: audit 2024-03-04T16:07:45.217474+0000 mon.c (mon.2) 814 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:07:45.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:45 smithi117 bash[21480]: audit 2024-03-04T16:07:45.341339+0000 mon.a (mon.0) 3458 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-142"}]': finished 2024-03-04T16:07:45.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:45 smithi117 bash[28843]: audit 2024-03-04T16:07:44.336358+0000 mon.a (mon.0) 3455 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-142", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:07:45.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:45 smithi117 bash[28843]: cluster 2024-03-04T16:07:44.342681+0000 mon.a (mon.0) 3456 : cluster 0 osdmap e718: 8 total, 8 up, 8 in 2024-03-04T16:07:45.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:45 smithi117 bash[28843]: audit 2024-03-04T16:07:44.344484+0000 mon.a (mon.0) 3457 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-142"}]: dispatch 2024-03-04T16:07:45.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:45 smithi117 bash[28843]: audit 2024-03-04T16:07:45.217474+0000 mon.c (mon.2) 814 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:07:45.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:45 smithi117 bash[28843]: audit 2024-03-04T16:07:45.341339+0000 mon.a (mon.0) 3458 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-142"}]': finished 2024-03-04T16:07:46.363 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:07:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:07:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:07:46.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:46 smithi151 bash[23756]: cluster 2024-03-04T16:07:45.218579+0000 mgr.y (mgr.24428) 691 : cluster 0 pgmap v1162: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:46.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:46 smithi151 bash[23756]: cluster 2024-03-04T16:07:45.347733+0000 mon.a (mon.0) 3459 : cluster 0 osdmap e719: 8 total, 8 up, 8 in 2024-03-04T16:07:46.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:46 smithi151 bash[23756]: audit 2024-03-04T16:07:45.350562+0000 mon.a (mon.0) 3460 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-142", "mode": "writeback"}]: dispatch 2024-03-04T16:07:46.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:46 smithi151 bash[23756]: cluster 2024-03-04T16:07:46.341605+0000 mon.a (mon.0) 3461 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:07:46.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:46 smithi151 bash[23756]: audit 2024-03-04T16:07:46.344043+0000 mon.a (mon.0) 3462 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-142", "mode": "writeback"}]': finished 2024-03-04T16:07:46.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:46 smithi151 bash[23756]: cluster 2024-03-04T16:07:46.351155+0000 mon.a (mon.0) 3463 : cluster 0 osdmap e720: 8 total, 8 up, 8 in 2024-03-04T16:07:46.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:46 smithi151 bash[23756]: audit 2024-03-04T16:07:46.353005+0000 mon.a (mon.0) 3464 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:07:46.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:46 smithi117 bash[21480]: cluster 2024-03-04T16:07:45.218579+0000 mgr.y (mgr.24428) 691 : cluster 0 pgmap v1162: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:46.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:46 smithi117 bash[21480]: cluster 2024-03-04T16:07:45.347733+0000 mon.a (mon.0) 3459 : cluster 0 osdmap e719: 8 total, 8 up, 8 in 2024-03-04T16:07:46.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:46 smithi117 bash[21480]: audit 2024-03-04T16:07:45.350562+0000 mon.a (mon.0) 3460 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-142", "mode": "writeback"}]: dispatch 2024-03-04T16:07:46.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:46 smithi117 bash[21480]: cluster 2024-03-04T16:07:46.341605+0000 mon.a (mon.0) 3461 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:07:46.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:46 smithi117 bash[21480]: audit 2024-03-04T16:07:46.344043+0000 mon.a (mon.0) 3462 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-142", "mode": "writeback"}]': finished 2024-03-04T16:07:46.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:46 smithi117 bash[21480]: cluster 2024-03-04T16:07:46.351155+0000 mon.a (mon.0) 3463 : cluster 0 osdmap e720: 8 total, 8 up, 8 in 2024-03-04T16:07:46.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:46 smithi117 bash[21480]: audit 2024-03-04T16:07:46.353005+0000 mon.a (mon.0) 3464 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:07:46.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:46 smithi117 bash[28843]: cluster 2024-03-04T16:07:45.218579+0000 mgr.y (mgr.24428) 691 : cluster 0 pgmap v1162: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 762 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:46.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:46 smithi117 bash[28843]: cluster 2024-03-04T16:07:45.347733+0000 mon.a (mon.0) 3459 : cluster 0 osdmap e719: 8 total, 8 up, 8 in 2024-03-04T16:07:46.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:46 smithi117 bash[28843]: audit 2024-03-04T16:07:45.350562+0000 mon.a (mon.0) 3460 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-142", "mode": "writeback"}]: dispatch 2024-03-04T16:07:46.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:46 smithi117 bash[28843]: cluster 2024-03-04T16:07:46.341605+0000 mon.a (mon.0) 3461 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:07:46.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:46 smithi117 bash[28843]: audit 2024-03-04T16:07:46.344043+0000 mon.a (mon.0) 3462 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-142", "mode": "writeback"}]': finished 2024-03-04T16:07:46.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:46 smithi117 bash[28843]: cluster 2024-03-04T16:07:46.351155+0000 mon.a (mon.0) 3463 : cluster 0 osdmap e720: 8 total, 8 up, 8 in 2024-03-04T16:07:46.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:46 smithi117 bash[28843]: audit 2024-03-04T16:07:46.353005+0000 mon.a (mon.0) 3464 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:07:47.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:07:47 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:07:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:48 smithi151 bash[23756]: cluster 2024-03-04T16:07:47.220580+0000 mgr.y (mgr.24428) 692 : cluster 0 pgmap v1165: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-04T16:07:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:48 smithi151 bash[23756]: audit 2024-03-04T16:07:47.226960+0000 mgr.y (mgr.24428) 693 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:07:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:48 smithi151 bash[23756]: audit 2024-03-04T16:07:47.347006+0000 mon.a (mon.0) 3465 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_count","val": "2"}]': finished 2024-03-04T16:07:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:48 smithi151 bash[23756]: cluster 2024-03-04T16:07:47.354366+0000 mon.a (mon.0) 3466 : cluster 0 osdmap e721: 8 total, 8 up, 8 in 2024-03-04T16:07:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:48 smithi151 bash[23756]: audit 2024-03-04T16:07:47.355797+0000 mon.a (mon.0) 3467 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:07:48.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:48 smithi117 bash[21480]: cluster 2024-03-04T16:07:47.220580+0000 mgr.y (mgr.24428) 692 : cluster 0 pgmap v1165: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-04T16:07:48.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:48 smithi117 bash[21480]: audit 2024-03-04T16:07:47.226960+0000 mgr.y (mgr.24428) 693 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:07:48.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:48 smithi117 bash[21480]: audit 2024-03-04T16:07:47.347006+0000 mon.a (mon.0) 3465 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_count","val": "2"}]': finished 2024-03-04T16:07:48.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:48 smithi117 bash[21480]: cluster 2024-03-04T16:07:47.354366+0000 mon.a (mon.0) 3466 : cluster 0 osdmap e721: 8 total, 8 up, 8 in 2024-03-04T16:07:48.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:48 smithi117 bash[21480]: audit 2024-03-04T16:07:47.355797+0000 mon.a (mon.0) 3467 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:07:48.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:48 smithi117 bash[28843]: cluster 2024-03-04T16:07:47.220580+0000 mgr.y (mgr.24428) 692 : cluster 0 pgmap v1165: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-04T16:07:48.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:48 smithi117 bash[28843]: audit 2024-03-04T16:07:47.226960+0000 mgr.y (mgr.24428) 693 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:07:48.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:48 smithi117 bash[28843]: audit 2024-03-04T16:07:47.347006+0000 mon.a (mon.0) 3465 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_count","val": "2"}]': finished 2024-03-04T16:07:48.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:48 smithi117 bash[28843]: cluster 2024-03-04T16:07:47.354366+0000 mon.a (mon.0) 3466 : cluster 0 osdmap e721: 8 total, 8 up, 8 in 2024-03-04T16:07:48.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:48 smithi117 bash[28843]: audit 2024-03-04T16:07:47.355797+0000 mon.a (mon.0) 3467 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:07:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:49 smithi151 bash[23756]: audit 2024-03-04T16:07:48.349350+0000 mon.a (mon.0) 3468 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_period","val": "600"}]': finished 2024-03-04T16:07:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:49 smithi151 bash[23756]: cluster 2024-03-04T16:07:48.356788+0000 mon.a (mon.0) 3469 : cluster 0 osdmap e722: 8 total, 8 up, 8 in 2024-03-04T16:07:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:49 smithi151 bash[23756]: audit 2024-03-04T16:07:48.358988+0000 mon.a (mon.0) 3470 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T16:07:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:49 smithi151 bash[23756]: cluster 2024-03-04T16:07:49.349687+0000 mon.a (mon.0) 3471 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:07:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:49 smithi151 bash[23756]: audit 2024-03-04T16:07:49.351985+0000 mon.a (mon.0) 3472 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T16:07:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:49 smithi151 bash[23756]: cluster 2024-03-04T16:07:49.359194+0000 mon.a (mon.0) 3473 : cluster 0 osdmap e723: 8 total, 8 up, 8 in 2024-03-04T16:07:49.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:49 smithi151 bash[23756]: audit 2024-03-04T16:07:49.360724+0000 mon.a (mon.0) 3474 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-04T16:07:49.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:49 smithi117 bash[21480]: audit 2024-03-04T16:07:48.349350+0000 mon.a (mon.0) 3468 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_period","val": "600"}]': finished 2024-03-04T16:07:49.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:49 smithi117 bash[21480]: cluster 2024-03-04T16:07:48.356788+0000 mon.a (mon.0) 3469 : cluster 0 osdmap e722: 8 total, 8 up, 8 in 2024-03-04T16:07:49.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:49 smithi117 bash[21480]: audit 2024-03-04T16:07:48.358988+0000 mon.a (mon.0) 3470 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T16:07:49.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:49 smithi117 bash[21480]: cluster 2024-03-04T16:07:49.349687+0000 mon.a (mon.0) 3471 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:07:49.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:49 smithi117 bash[21480]: audit 2024-03-04T16:07:49.351985+0000 mon.a (mon.0) 3472 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T16:07:49.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:49 smithi117 bash[21480]: cluster 2024-03-04T16:07:49.359194+0000 mon.a (mon.0) 3473 : cluster 0 osdmap e723: 8 total, 8 up, 8 in 2024-03-04T16:07:49.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:49 smithi117 bash[21480]: audit 2024-03-04T16:07:49.360724+0000 mon.a (mon.0) 3474 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-04T16:07:49.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:49 smithi117 bash[28843]: audit 2024-03-04T16:07:48.349350+0000 mon.a (mon.0) 3468 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_period","val": "600"}]': finished 2024-03-04T16:07:49.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:49 smithi117 bash[28843]: cluster 2024-03-04T16:07:48.356788+0000 mon.a (mon.0) 3469 : cluster 0 osdmap e722: 8 total, 8 up, 8 in 2024-03-04T16:07:49.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:49 smithi117 bash[28843]: audit 2024-03-04T16:07:48.358988+0000 mon.a (mon.0) 3470 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T16:07:49.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:49 smithi117 bash[28843]: cluster 2024-03-04T16:07:49.349687+0000 mon.a (mon.0) 3471 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:07:49.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:49 smithi117 bash[28843]: audit 2024-03-04T16:07:49.351985+0000 mon.a (mon.0) 3472 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T16:07:49.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:49 smithi117 bash[28843]: cluster 2024-03-04T16:07:49.359194+0000 mon.a (mon.0) 3473 : cluster 0 osdmap e723: 8 total, 8 up, 8 in 2024-03-04T16:07:49.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:49 smithi117 bash[28843]: audit 2024-03-04T16:07:49.360724+0000 mon.a (mon.0) 3474 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-04T16:07:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:50 smithi151 bash[23756]: cluster 2024-03-04T16:07:49.221545+0000 mgr.y (mgr.24428) 694 : cluster 0 pgmap v1168: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-04T16:07:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:50 smithi151 bash[23756]: cluster 2024-03-04T16:07:49.799491+0000 mon.a (mon.0) 3475 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:50 smithi151 bash[23756]: audit 2024-03-04T16:07:50.354827+0000 mon.a (mon.0) 3476 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-04T16:07:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:50 smithi151 bash[23756]: cluster 2024-03-04T16:07:50.361200+0000 mon.a (mon.0) 3477 : cluster 0 osdmap e724: 8 total, 8 up, 8 in 2024-03-04T16:07:50.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:50 smithi151 bash[23756]: audit 2024-03-04T16:07:50.361802+0000 mon.a (mon.0) 3478 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-04T16:07:50.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:50 smithi117 bash[21480]: cluster 2024-03-04T16:07:49.221545+0000 mgr.y (mgr.24428) 694 : cluster 0 pgmap v1168: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-04T16:07:50.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:50 smithi117 bash[21480]: cluster 2024-03-04T16:07:49.799491+0000 mon.a (mon.0) 3475 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:50.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:50 smithi117 bash[21480]: audit 2024-03-04T16:07:50.354827+0000 mon.a (mon.0) 3476 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-04T16:07:50.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:50 smithi117 bash[21480]: cluster 2024-03-04T16:07:50.361200+0000 mon.a (mon.0) 3477 : cluster 0 osdmap e724: 8 total, 8 up, 8 in 2024-03-04T16:07:50.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:50 smithi117 bash[21480]: audit 2024-03-04T16:07:50.361802+0000 mon.a (mon.0) 3478 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-04T16:07:50.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:50 smithi117 bash[28843]: cluster 2024-03-04T16:07:49.221545+0000 mgr.y (mgr.24428) 694 : cluster 0 pgmap v1168: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-03-04T16:07:50.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:50 smithi117 bash[28843]: cluster 2024-03-04T16:07:49.799491+0000 mon.a (mon.0) 3475 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:50.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:50 smithi117 bash[28843]: audit 2024-03-04T16:07:50.354827+0000 mon.a (mon.0) 3476 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-04T16:07:50.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:50 smithi117 bash[28843]: cluster 2024-03-04T16:07:50.361200+0000 mon.a (mon.0) 3477 : cluster 0 osdmap e724: 8 total, 8 up, 8 in 2024-03-04T16:07:50.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:50 smithi117 bash[28843]: audit 2024-03-04T16:07:50.361802+0000 mon.a (mon.0) 3478 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-03-04T16:07:52.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:52 smithi151 bash[23756]: cluster 2024-03-04T16:07:51.222411+0000 mgr.y (mgr.24428) 695 : cluster 0 pgmap v1171: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:07:52.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:52 smithi151 bash[23756]: audit 2024-03-04T16:07:51.357759+0000 mon.a (mon.0) 3479 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-04T16:07:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:52 smithi151 bash[23756]: cluster 2024-03-04T16:07:51.366833+0000 mon.a (mon.0) 3480 : cluster 0 osdmap e725: 8 total, 8 up, 8 in 2024-03-04T16:07:52.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:52 smithi151 bash[23756]: audit 2024-03-04T16:07:51.368346+0000 mon.a (mon.0) 3481 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-04T16:07:52.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:52 smithi117 bash[21480]: cluster 2024-03-04T16:07:51.222411+0000 mgr.y (mgr.24428) 695 : cluster 0 pgmap v1171: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:07:52.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:52 smithi117 bash[21480]: audit 2024-03-04T16:07:51.357759+0000 mon.a (mon.0) 3479 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-04T16:07:52.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:52 smithi117 bash[21480]: cluster 2024-03-04T16:07:51.366833+0000 mon.a (mon.0) 3480 : cluster 0 osdmap e725: 8 total, 8 up, 8 in 2024-03-04T16:07:52.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:52 smithi117 bash[21480]: audit 2024-03-04T16:07:51.368346+0000 mon.a (mon.0) 3481 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-04T16:07:52.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:52 smithi117 bash[28843]: cluster 2024-03-04T16:07:51.222411+0000 mgr.y (mgr.24428) 695 : cluster 0 pgmap v1171: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:07:52.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:52 smithi117 bash[28843]: audit 2024-03-04T16:07:51.357759+0000 mon.a (mon.0) 3479 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-03-04T16:07:52.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:52 smithi117 bash[28843]: cluster 2024-03-04T16:07:51.366833+0000 mon.a (mon.0) 3480 : cluster 0 osdmap e725: 8 total, 8 up, 8 in 2024-03-04T16:07:52.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:52 smithi117 bash[28843]: audit 2024-03-04T16:07:51.368346+0000 mon.a (mon.0) 3481 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-03-04T16:07:53.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:53 smithi151 bash[23756]: audit 2024-03-04T16:07:52.360278+0000 mon.a (mon.0) 3482 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-04T16:07:53.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:53 smithi151 bash[23756]: cluster 2024-03-04T16:07:52.368623+0000 mon.a (mon.0) 3483 : cluster 0 osdmap e726: 8 total, 8 up, 8 in 2024-03-04T16:07:53.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:53 smithi151 bash[23756]: audit 2024-03-04T16:07:52.470905+0000 mon.a (mon.0) 3484 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:07:53.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:53 smithi117 bash[28843]: audit 2024-03-04T16:07:52.360278+0000 mon.a (mon.0) 3482 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-04T16:07:53.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:53 smithi117 bash[28843]: cluster 2024-03-04T16:07:52.368623+0000 mon.a (mon.0) 3483 : cluster 0 osdmap e726: 8 total, 8 up, 8 in 2024-03-04T16:07:53.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:53 smithi117 bash[28843]: audit 2024-03-04T16:07:52.470905+0000 mon.a (mon.0) 3484 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:07:53.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:53 smithi117 bash[21480]: audit 2024-03-04T16:07:52.360278+0000 mon.a (mon.0) 3482 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-03-04T16:07:53.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:53 smithi117 bash[21480]: cluster 2024-03-04T16:07:52.368623+0000 mon.a (mon.0) 3483 : cluster 0 osdmap e726: 8 total, 8 up, 8 in 2024-03-04T16:07:53.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:53 smithi117 bash[21480]: audit 2024-03-04T16:07:52.470905+0000 mon.a (mon.0) 3484 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:07:54.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:54 smithi151 bash[23756]: cluster 2024-03-04T16:07:53.223327+0000 mgr.y (mgr.24428) 696 : cluster 0 pgmap v1174: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:54 smithi151 bash[23756]: audit 2024-03-04T16:07:53.387076+0000 mon.a (mon.0) 3485 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:07:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:54 smithi151 bash[23756]: cluster 2024-03-04T16:07:53.400387+0000 mon.a (mon.0) 3486 : cluster 0 osdmap e727: 8 total, 8 up, 8 in 2024-03-04T16:07:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:54 smithi151 bash[23756]: audit 2024-03-04T16:07:53.402253+0000 mon.a (mon.0) 3487 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-142"}]: dispatch 2024-03-04T16:07:54.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:54 smithi151 bash[23756]: audit 2024-03-04T16:07:54.055953+0000 mon.c (mon.2) 815 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:07:54.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:54 smithi117 bash[21480]: cluster 2024-03-04T16:07:53.223327+0000 mgr.y (mgr.24428) 696 : cluster 0 pgmap v1174: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:54.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:54 smithi117 bash[21480]: audit 2024-03-04T16:07:53.387076+0000 mon.a (mon.0) 3485 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:07:54.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:54 smithi117 bash[21480]: cluster 2024-03-04T16:07:53.400387+0000 mon.a (mon.0) 3486 : cluster 0 osdmap e727: 8 total, 8 up, 8 in 2024-03-04T16:07:54.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:54 smithi117 bash[21480]: audit 2024-03-04T16:07:53.402253+0000 mon.a (mon.0) 3487 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-142"}]: dispatch 2024-03-04T16:07:54.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:54 smithi117 bash[21480]: audit 2024-03-04T16:07:54.055953+0000 mon.c (mon.2) 815 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:07:54.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:54 smithi117 bash[28843]: cluster 2024-03-04T16:07:53.223327+0000 mgr.y (mgr.24428) 696 : cluster 0 pgmap v1174: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:54.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:54 smithi117 bash[28843]: audit 2024-03-04T16:07:53.387076+0000 mon.a (mon.0) 3485 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:07:54.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:54 smithi117 bash[28843]: cluster 2024-03-04T16:07:53.400387+0000 mon.a (mon.0) 3486 : cluster 0 osdmap e727: 8 total, 8 up, 8 in 2024-03-04T16:07:54.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:54 smithi117 bash[28843]: audit 2024-03-04T16:07:53.402253+0000 mon.a (mon.0) 3487 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-142"}]: dispatch 2024-03-04T16:07:54.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:54 smithi117 bash[28843]: audit 2024-03-04T16:07:54.055953+0000 mon.c (mon.2) 815 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:07:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:55 smithi151 bash[23756]: audit 2024-03-04T16:07:54.389782+0000 mon.a (mon.0) 3488 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-142"}]': finished 2024-03-04T16:07:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:55 smithi151 bash[23756]: cluster 2024-03-04T16:07:54.399195+0000 mon.a (mon.0) 3489 : cluster 0 osdmap e728: 8 total, 8 up, 8 in 2024-03-04T16:07:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:55 smithi151 bash[23756]: audit 2024-03-04T16:07:54.458330+0000 mon.a (mon.0) 3490 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:07:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:55 smithi151 bash[23756]: audit 2024-03-04T16:07:54.459196+0000 mon.a (mon.0) 3491 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-142"}]: dispatch 2024-03-04T16:07:56.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:55 smithi151 bash[23756]: audit 2024-03-04T16:07:54.742155+0000 mon.c (mon.2) 816 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:07:56.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:55 smithi151 bash[23756]: audit 2024-03-04T16:07:54.743512+0000 mon.c (mon.2) 817 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:07:56.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:55 smithi151 bash[23756]: audit 2024-03-04T16:07:54.750891+0000 mon.a (mon.0) 3492 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:07:56.183 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:55 smithi151 bash[23756]: cluster 2024-03-04T16:07:55.224254+0000 mgr.y (mgr.24428) 697 : cluster 0 pgmap v1177: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:56.317 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:07:56 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:07:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:07:56.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:55 smithi117 bash[21480]: audit 2024-03-04T16:07:54.389782+0000 mon.a (mon.0) 3488 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-142"}]': finished 2024-03-04T16:07:56.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:55 smithi117 bash[21480]: cluster 2024-03-04T16:07:54.399195+0000 mon.a (mon.0) 3489 : cluster 0 osdmap e728: 8 total, 8 up, 8 in 2024-03-04T16:07:56.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:55 smithi117 bash[21480]: audit 2024-03-04T16:07:54.458330+0000 mon.a (mon.0) 3490 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:07:56.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:55 smithi117 bash[21480]: audit 2024-03-04T16:07:54.459196+0000 mon.a (mon.0) 3491 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-142"}]: dispatch 2024-03-04T16:07:56.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:55 smithi117 bash[21480]: audit 2024-03-04T16:07:54.742155+0000 mon.c (mon.2) 816 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:07:56.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:55 smithi117 bash[21480]: audit 2024-03-04T16:07:54.743512+0000 mon.c (mon.2) 817 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:07:56.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:55 smithi117 bash[21480]: audit 2024-03-04T16:07:54.750891+0000 mon.a (mon.0) 3492 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:07:56.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:55 smithi117 bash[21480]: cluster 2024-03-04T16:07:55.224254+0000 mgr.y (mgr.24428) 697 : cluster 0 pgmap v1177: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:56.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:55 smithi117 bash[28843]: audit 2024-03-04T16:07:54.389782+0000 mon.a (mon.0) 3488 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-142"}]': finished 2024-03-04T16:07:56.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:55 smithi117 bash[28843]: cluster 2024-03-04T16:07:54.399195+0000 mon.a (mon.0) 3489 : cluster 0 osdmap e728: 8 total, 8 up, 8 in 2024-03-04T16:07:56.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:55 smithi117 bash[28843]: audit 2024-03-04T16:07:54.458330+0000 mon.a (mon.0) 3490 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:07:56.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:55 smithi117 bash[28843]: audit 2024-03-04T16:07:54.459196+0000 mon.a (mon.0) 3491 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-142"}]: dispatch 2024-03-04T16:07:56.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:55 smithi117 bash[28843]: audit 2024-03-04T16:07:54.742155+0000 mon.c (mon.2) 816 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:07:56.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:55 smithi117 bash[28843]: audit 2024-03-04T16:07:54.743512+0000 mon.c (mon.2) 817 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:07:56.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:55 smithi117 bash[28843]: audit 2024-03-04T16:07:54.750891+0000 mon.a (mon.0) 3492 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:07:56.320 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:55 smithi117 bash[28843]: cluster 2024-03-04T16:07:55.224254+0000 mgr.y (mgr.24428) 697 : cluster 0 pgmap v1177: 204 pgs: 204 active+clean; 4.4 MiB data, 763 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:07:57.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:56 smithi151 bash[23756]: cluster 2024-03-04T16:07:55.418044+0000 mon.a (mon.0) 3493 : cluster 0 osdmap e729: 8 total, 8 up, 8 in 2024-03-04T16:07:57.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:56 smithi151 bash[23756]: cluster 2024-03-04T16:07:55.880654+0000 mon.a (mon.0) 3494 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:57.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:56 smithi151 bash[23756]: cluster 2024-03-04T16:07:56.419280+0000 mon.a (mon.0) 3495 : cluster 0 osdmap e730: 8 total, 8 up, 8 in 2024-03-04T16:07:57.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:56 smithi151 bash[23756]: audit 2024-03-04T16:07:56.428066+0000 mon.a (mon.0) 3496 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:07:57.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:56 smithi117 bash[21480]: cluster 2024-03-04T16:07:55.418044+0000 mon.a (mon.0) 3493 : cluster 0 osdmap e729: 8 total, 8 up, 8 in 2024-03-04T16:07:57.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:56 smithi117 bash[21480]: cluster 2024-03-04T16:07:55.880654+0000 mon.a (mon.0) 3494 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:57.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:56 smithi117 bash[21480]: cluster 2024-03-04T16:07:56.419280+0000 mon.a (mon.0) 3495 : cluster 0 osdmap e730: 8 total, 8 up, 8 in 2024-03-04T16:07:57.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:56 smithi117 bash[21480]: audit 2024-03-04T16:07:56.428066+0000 mon.a (mon.0) 3496 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:07:57.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:56 smithi117 bash[28843]: cluster 2024-03-04T16:07:55.418044+0000 mon.a (mon.0) 3493 : cluster 0 osdmap e729: 8 total, 8 up, 8 in 2024-03-04T16:07:57.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:56 smithi117 bash[28843]: cluster 2024-03-04T16:07:55.880654+0000 mon.a (mon.0) 3494 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:07:57.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:56 smithi117 bash[28843]: cluster 2024-03-04T16:07:56.419280+0000 mon.a (mon.0) 3495 : cluster 0 osdmap e730: 8 total, 8 up, 8 in 2024-03-04T16:07:57.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:56 smithi117 bash[28843]: audit 2024-03-04T16:07:56.428066+0000 mon.a (mon.0) 3496 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:07:57.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:07:57 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:07:58.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:57 smithi151 bash[23756]: cluster 2024-03-04T16:07:57.226359+0000 mgr.y (mgr.24428) 698 : cluster 0 pgmap v1180: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-04T16:07:58.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:57 smithi151 bash[23756]: audit 2024-03-04T16:07:57.235007+0000 mgr.y (mgr.24428) 699 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:07:58.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:57 smithi151 bash[23756]: audit 2024-03-04T16:07:57.414785+0000 mon.a (mon.0) 3497 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:07:58.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:57 smithi151 bash[23756]: cluster 2024-03-04T16:07:57.423014+0000 mon.a (mon.0) 3498 : cluster 0 osdmap e731: 8 total, 8 up, 8 in 2024-03-04T16:07:58.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:57 smithi151 bash[23756]: audit 2024-03-04T16:07:57.477377+0000 mon.a (mon.0) 3499 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:07:58.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:57 smithi117 bash[21480]: cluster 2024-03-04T16:07:57.226359+0000 mgr.y (mgr.24428) 698 : cluster 0 pgmap v1180: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-04T16:07:58.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:57 smithi117 bash[21480]: audit 2024-03-04T16:07:57.235007+0000 mgr.y (mgr.24428) 699 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:07:58.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:57 smithi117 bash[21480]: audit 2024-03-04T16:07:57.414785+0000 mon.a (mon.0) 3497 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:07:58.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:57 smithi117 bash[21480]: cluster 2024-03-04T16:07:57.423014+0000 mon.a (mon.0) 3498 : cluster 0 osdmap e731: 8 total, 8 up, 8 in 2024-03-04T16:07:58.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:57 smithi117 bash[21480]: audit 2024-03-04T16:07:57.477377+0000 mon.a (mon.0) 3499 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:07:58.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:57 smithi117 bash[28843]: cluster 2024-03-04T16:07:57.226359+0000 mgr.y (mgr.24428) 698 : cluster 0 pgmap v1180: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-04T16:07:58.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:57 smithi117 bash[28843]: audit 2024-03-04T16:07:57.235007+0000 mgr.y (mgr.24428) 699 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:07:58.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:57 smithi117 bash[28843]: audit 2024-03-04T16:07:57.414785+0000 mon.a (mon.0) 3497 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:07:58.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:57 smithi117 bash[28843]: cluster 2024-03-04T16:07:57.423014+0000 mon.a (mon.0) 3498 : cluster 0 osdmap e731: 8 total, 8 up, 8 in 2024-03-04T16:07:58.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:57 smithi117 bash[28843]: audit 2024-03-04T16:07:57.477377+0000 mon.a (mon.0) 3499 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:07:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:59 smithi151 bash[23756]: audit 2024-03-04T16:07:58.417460+0000 mon.a (mon.0) 3500 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-144", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:07:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:59 smithi151 bash[23756]: cluster 2024-03-04T16:07:58.425111+0000 mon.a (mon.0) 3501 : cluster 0 osdmap e732: 8 total, 8 up, 8 in 2024-03-04T16:07:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:59 smithi151 bash[23756]: audit 2024-03-04T16:07:58.426732+0000 mon.a (mon.0) 3502 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-144"}]: dispatch 2024-03-04T16:07:59.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:07:59 smithi151 bash[23756]: cluster 2024-03-04T16:07:59.227211+0000 mgr.y (mgr.24428) 700 : cluster 0 pgmap v1183: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-04T16:07:59.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:59 smithi117 bash[21480]: audit 2024-03-04T16:07:58.417460+0000 mon.a (mon.0) 3500 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-144", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:07:59.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:59 smithi117 bash[21480]: cluster 2024-03-04T16:07:58.425111+0000 mon.a (mon.0) 3501 : cluster 0 osdmap e732: 8 total, 8 up, 8 in 2024-03-04T16:07:59.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:59 smithi117 bash[21480]: audit 2024-03-04T16:07:58.426732+0000 mon.a (mon.0) 3502 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-144"}]: dispatch 2024-03-04T16:07:59.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:07:59 smithi117 bash[21480]: cluster 2024-03-04T16:07:59.227211+0000 mgr.y (mgr.24428) 700 : cluster 0 pgmap v1183: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-04T16:07:59.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:59 smithi117 bash[28843]: audit 2024-03-04T16:07:58.417460+0000 mon.a (mon.0) 3500 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-144", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:07:59.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:59 smithi117 bash[28843]: cluster 2024-03-04T16:07:58.425111+0000 mon.a (mon.0) 3501 : cluster 0 osdmap e732: 8 total, 8 up, 8 in 2024-03-04T16:07:59.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:59 smithi117 bash[28843]: audit 2024-03-04T16:07:58.426732+0000 mon.a (mon.0) 3502 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-144"}]: dispatch 2024-03-04T16:07:59.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:07:59 smithi117 bash[28843]: cluster 2024-03-04T16:07:59.227211+0000 mgr.y (mgr.24428) 700 : cluster 0 pgmap v1183: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-03-04T16:08:00.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:00 smithi117 bash[21480]: audit 2024-03-04T16:07:59.429685+0000 mon.a (mon.0) 3503 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-144"}]': finished 2024-03-04T16:08:00.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:00 smithi117 bash[21480]: cluster 2024-03-04T16:07:59.440566+0000 mon.a (mon.0) 3504 : cluster 0 osdmap e733: 8 total, 8 up, 8 in 2024-03-04T16:08:00.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:00 smithi117 bash[21480]: audit 2024-03-04T16:07:59.443836+0000 mon.a (mon.0) 3505 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-144", "mode": "readproxy"}]: dispatch 2024-03-04T16:08:00.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:00 smithi117 bash[21480]: audit 2024-03-04T16:08:00.250550+0000 mon.c (mon.2) 818 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:08:00.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:00 smithi117 bash[28843]: audit 2024-03-04T16:07:59.429685+0000 mon.a (mon.0) 3503 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-144"}]': finished 2024-03-04T16:08:00.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:00 smithi117 bash[28843]: cluster 2024-03-04T16:07:59.440566+0000 mon.a (mon.0) 3504 : cluster 0 osdmap e733: 8 total, 8 up, 8 in 2024-03-04T16:08:00.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:00 smithi117 bash[28843]: audit 2024-03-04T16:07:59.443836+0000 mon.a (mon.0) 3505 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-144", "mode": "readproxy"}]: dispatch 2024-03-04T16:08:00.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:00 smithi117 bash[28843]: audit 2024-03-04T16:08:00.250550+0000 mon.c (mon.2) 818 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:08:00.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:00 smithi151 bash[23756]: audit 2024-03-04T16:07:59.429685+0000 mon.a (mon.0) 3503 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-144"}]': finished 2024-03-04T16:08:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:00 smithi151 bash[23756]: cluster 2024-03-04T16:07:59.440566+0000 mon.a (mon.0) 3504 : cluster 0 osdmap e733: 8 total, 8 up, 8 in 2024-03-04T16:08:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:00 smithi151 bash[23756]: audit 2024-03-04T16:07:59.443836+0000 mon.a (mon.0) 3505 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-144", "mode": "readproxy"}]: dispatch 2024-03-04T16:08:00.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:00 smithi151 bash[23756]: audit 2024-03-04T16:08:00.250550+0000 mon.c (mon.2) 818 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:08:01.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:01 smithi117 bash[21480]: cluster 2024-03-04T16:08:00.430053+0000 mon.a (mon.0) 3506 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:08:01.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:01 smithi117 bash[21480]: audit 2024-03-04T16:08:00.432546+0000 mon.a (mon.0) 3507 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-144", "mode": "readproxy"}]': finished 2024-03-04T16:08:01.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:01 smithi117 bash[21480]: cluster 2024-03-04T16:08:00.439308+0000 mon.a (mon.0) 3508 : cluster 0 osdmap e734: 8 total, 8 up, 8 in 2024-03-04T16:08:01.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:01 smithi117 bash[21480]: cluster 2024-03-04T16:08:01.228170+0000 mgr.y (mgr.24428) 701 : cluster 0 pgmap v1186: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:08:01.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:01 smithi117 bash[28843]: cluster 2024-03-04T16:08:00.430053+0000 mon.a (mon.0) 3506 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:08:01.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:01 smithi117 bash[28843]: audit 2024-03-04T16:08:00.432546+0000 mon.a (mon.0) 3507 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-144", "mode": "readproxy"}]': finished 2024-03-04T16:08:01.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:01 smithi117 bash[28843]: cluster 2024-03-04T16:08:00.439308+0000 mon.a (mon.0) 3508 : cluster 0 osdmap e734: 8 total, 8 up, 8 in 2024-03-04T16:08:01.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:01 smithi117 bash[28843]: cluster 2024-03-04T16:08:01.228170+0000 mgr.y (mgr.24428) 701 : cluster 0 pgmap v1186: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:08:01.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:01 smithi151 bash[23756]: cluster 2024-03-04T16:08:00.430053+0000 mon.a (mon.0) 3506 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:08:01.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:01 smithi151 bash[23756]: audit 2024-03-04T16:08:00.432546+0000 mon.a (mon.0) 3507 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-144", "mode": "readproxy"}]': finished 2024-03-04T16:08:01.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:01 smithi151 bash[23756]: cluster 2024-03-04T16:08:00.439308+0000 mon.a (mon.0) 3508 : cluster 0 osdmap e734: 8 total, 8 up, 8 in 2024-03-04T16:08:01.933 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:01 smithi151 bash[23756]: cluster 2024-03-04T16:08:01.228170+0000 mgr.y (mgr.24428) 701 : cluster 0 pgmap v1186: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 731 MiB used, 715 GiB / 715 GiB avail 2024-03-04T16:08:04.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:04 smithi117 bash[21480]: cluster 2024-03-04T16:08:03.230382+0000 mgr.y (mgr.24428) 702 : cluster 0 pgmap v1187: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 176 B/s wr, 1 op/s 2024-03-04T16:08:04.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:04 smithi117 bash[28843]: cluster 2024-03-04T16:08:03.230382+0000 mgr.y (mgr.24428) 702 : cluster 0 pgmap v1187: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 176 B/s wr, 1 op/s 2024-03-04T16:08:04.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:04 smithi151 bash[23756]: cluster 2024-03-04T16:08:03.230382+0000 mgr.y (mgr.24428) 702 : cluster 0 pgmap v1187: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 176 B/s wr, 1 op/s 2024-03-04T16:08:05.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:05 smithi117 bash[21480]: cluster 2024-03-04T16:08:04.802605+0000 mon.a (mon.0) 3509 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:08:05.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:05 smithi117 bash[28843]: cluster 2024-03-04T16:08:04.802605+0000 mon.a (mon.0) 3509 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:08:05.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:05 smithi151 bash[23756]: cluster 2024-03-04T16:08:04.802605+0000 mon.a (mon.0) 3509 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:08:06.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:06 smithi117 bash[21480]: cluster 2024-03-04T16:08:05.231327+0000 mgr.y (mgr.24428) 703 : cluster 0 pgmap v1188: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 901 B/s rd, 150 B/s wr, 1 op/s 2024-03-04T16:08:06.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:08:06 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:08:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:08:06.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:06 smithi117 bash[28843]: cluster 2024-03-04T16:08:05.231327+0000 mgr.y (mgr.24428) 703 : cluster 0 pgmap v1188: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 901 B/s rd, 150 B/s wr, 1 op/s 2024-03-04T16:08:06.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:06 smithi151 bash[23756]: cluster 2024-03-04T16:08:05.231327+0000 mgr.y (mgr.24428) 703 : cluster 0 pgmap v1188: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 901 B/s rd, 150 B/s wr, 1 op/s 2024-03-04T16:08:07.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:08:07 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:08:08.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:08 smithi117 bash[21480]: cluster 2024-03-04T16:08:07.233574+0000 mgr.y (mgr.24428) 704 : cluster 0 pgmap v1189: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-03-04T16:08:08.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:08 smithi117 bash[21480]: audit 2024-03-04T16:08:07.237915+0000 mgr.y (mgr.24428) 705 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:08:08.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:08 smithi117 bash[28843]: cluster 2024-03-04T16:08:07.233574+0000 mgr.y (mgr.24428) 704 : cluster 0 pgmap v1189: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-03-04T16:08:08.569 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:08 smithi117 bash[28843]: audit 2024-03-04T16:08:07.237915+0000 mgr.y (mgr.24428) 705 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:08:08.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:08 smithi151 bash[23756]: cluster 2024-03-04T16:08:07.233574+0000 mgr.y (mgr.24428) 704 : cluster 0 pgmap v1189: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-03-04T16:08:08.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:08 smithi151 bash[23756]: audit 2024-03-04T16:08:07.237915+0000 mgr.y (mgr.24428) 705 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:08:10.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:10 smithi117 bash[21480]: cluster 2024-03-04T16:08:09.234542+0000 mgr.y (mgr.24428) 706 : cluster 0 pgmap v1190: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 104 B/s wr, 1 op/s 2024-03-04T16:08:10.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:10 smithi117 bash[28843]: cluster 2024-03-04T16:08:09.234542+0000 mgr.y (mgr.24428) 706 : cluster 0 pgmap v1190: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 104 B/s wr, 1 op/s 2024-03-04T16:08:10.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:10 smithi151 bash[23756]: cluster 2024-03-04T16:08:09.234542+0000 mgr.y (mgr.24428) 706 : cluster 0 pgmap v1190: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 104 B/s wr, 1 op/s 2024-03-04T16:08:11.567 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:11 smithi117 bash[21480]: audit 2024-03-04T16:08:10.755858+0000 mon.a (mon.0) 3510 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:11.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:11 smithi117 bash[28843]: audit 2024-03-04T16:08:10.755858+0000 mon.a (mon.0) 3510 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:11.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:11 smithi151 bash[23756]: audit 2024-03-04T16:08:10.755858+0000 mon.a (mon.0) 3510 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:12.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:12 smithi151 bash[23756]: cluster 2024-03-04T16:08:11.235506+0000 mgr.y (mgr.24428) 707 : cluster 0 pgmap v1191: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 94 B/s wr, 1 op/s 2024-03-04T16:08:12.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:12 smithi151 bash[23756]: audit 2024-03-04T16:08:11.326162+0000 mon.a (mon.0) 3511 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:08:12.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:12 smithi151 bash[23756]: cluster 2024-03-04T16:08:11.332979+0000 mon.a (mon.0) 3512 : cluster 0 osdmap e735: 8 total, 8 up, 8 in 2024-03-04T16:08:12.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:12 smithi151 bash[23756]: audit 2024-03-04T16:08:11.334033+0000 mon.a (mon.0) 3513 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-144"}]: dispatch 2024-03-04T16:08:12.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:12 smithi117 bash[21480]: cluster 2024-03-04T16:08:11.235506+0000 mgr.y (mgr.24428) 707 : cluster 0 pgmap v1191: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 94 B/s wr, 1 op/s 2024-03-04T16:08:12.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:12 smithi117 bash[21480]: audit 2024-03-04T16:08:11.326162+0000 mon.a (mon.0) 3511 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:08:12.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:12 smithi117 bash[21480]: cluster 2024-03-04T16:08:11.332979+0000 mon.a (mon.0) 3512 : cluster 0 osdmap e735: 8 total, 8 up, 8 in 2024-03-04T16:08:12.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:12 smithi117 bash[21480]: audit 2024-03-04T16:08:11.334033+0000 mon.a (mon.0) 3513 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-144"}]: dispatch 2024-03-04T16:08:12.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:12 smithi117 bash[28843]: cluster 2024-03-04T16:08:11.235506+0000 mgr.y (mgr.24428) 707 : cluster 0 pgmap v1191: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 94 B/s wr, 1 op/s 2024-03-04T16:08:12.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:12 smithi117 bash[28843]: audit 2024-03-04T16:08:11.326162+0000 mon.a (mon.0) 3511 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:08:12.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:12 smithi117 bash[28843]: cluster 2024-03-04T16:08:11.332979+0000 mon.a (mon.0) 3512 : cluster 0 osdmap e735: 8 total, 8 up, 8 in 2024-03-04T16:08:12.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:12 smithi117 bash[28843]: audit 2024-03-04T16:08:11.334033+0000 mon.a (mon.0) 3513 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-144"}]: dispatch 2024-03-04T16:08:13.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:13 smithi151 bash[23756]: cluster 2024-03-04T16:08:12.326671+0000 mon.a (mon.0) 3514 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:08:13.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:13 smithi151 bash[23756]: audit 2024-03-04T16:08:12.329472+0000 mon.a (mon.0) 3515 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-144"}]': finished 2024-03-04T16:08:13.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:13 smithi151 bash[23756]: cluster 2024-03-04T16:08:12.336276+0000 mon.a (mon.0) 3516 : cluster 0 osdmap e736: 8 total, 8 up, 8 in 2024-03-04T16:08:13.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:13 smithi151 bash[23756]: audit 2024-03-04T16:08:12.397876+0000 mon.a (mon.0) 3517 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:13.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:13 smithi151 bash[23756]: audit 2024-03-04T16:08:12.398864+0000 mon.a (mon.0) 3518 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-144"}]: dispatch 2024-03-04T16:08:13.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:13 smithi117 bash[28843]: cluster 2024-03-04T16:08:12.326671+0000 mon.a (mon.0) 3514 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:08:13.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:13 smithi117 bash[28843]: audit 2024-03-04T16:08:12.329472+0000 mon.a (mon.0) 3515 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-144"}]': finished 2024-03-04T16:08:13.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:13 smithi117 bash[28843]: cluster 2024-03-04T16:08:12.336276+0000 mon.a (mon.0) 3516 : cluster 0 osdmap e736: 8 total, 8 up, 8 in 2024-03-04T16:08:13.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:13 smithi117 bash[28843]: audit 2024-03-04T16:08:12.397876+0000 mon.a (mon.0) 3517 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:13.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:13 smithi117 bash[28843]: audit 2024-03-04T16:08:12.398864+0000 mon.a (mon.0) 3518 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-144"}]: dispatch 2024-03-04T16:08:13.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:13 smithi117 bash[21480]: cluster 2024-03-04T16:08:12.326671+0000 mon.a (mon.0) 3514 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:08:13.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:13 smithi117 bash[21480]: audit 2024-03-04T16:08:12.329472+0000 mon.a (mon.0) 3515 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-144"}]': finished 2024-03-04T16:08:13.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:13 smithi117 bash[21480]: cluster 2024-03-04T16:08:12.336276+0000 mon.a (mon.0) 3516 : cluster 0 osdmap e736: 8 total, 8 up, 8 in 2024-03-04T16:08:13.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:13 smithi117 bash[21480]: audit 2024-03-04T16:08:12.397876+0000 mon.a (mon.0) 3517 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:13.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:13 smithi117 bash[21480]: audit 2024-03-04T16:08:12.398864+0000 mon.a (mon.0) 3518 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-144"}]: dispatch 2024-03-04T16:08:14.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:14 smithi151 bash[23756]: cluster 2024-03-04T16:08:13.236353+0000 mgr.y (mgr.24428) 708 : cluster 0 pgmap v1194: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:08:14.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:14 smithi151 bash[23756]: cluster 2024-03-04T16:08:13.341717+0000 mon.a (mon.0) 3519 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:08:14.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:14 smithi151 bash[23756]: cluster 2024-03-04T16:08:13.364197+0000 mon.a (mon.0) 3520 : cluster 0 osdmap e737: 8 total, 8 up, 8 in 2024-03-04T16:08:14.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:14 smithi117 bash[21480]: cluster 2024-03-04T16:08:13.236353+0000 mgr.y (mgr.24428) 708 : cluster 0 pgmap v1194: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:08:14.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:14 smithi117 bash[21480]: cluster 2024-03-04T16:08:13.341717+0000 mon.a (mon.0) 3519 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:08:14.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:14 smithi117 bash[21480]: cluster 2024-03-04T16:08:13.364197+0000 mon.a (mon.0) 3520 : cluster 0 osdmap e737: 8 total, 8 up, 8 in 2024-03-04T16:08:14.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:14 smithi117 bash[28843]: cluster 2024-03-04T16:08:13.236353+0000 mgr.y (mgr.24428) 708 : cluster 0 pgmap v1194: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:08:14.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:14 smithi117 bash[28843]: cluster 2024-03-04T16:08:13.341717+0000 mon.a (mon.0) 3519 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:08:14.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:14 smithi117 bash[28843]: cluster 2024-03-04T16:08:13.364197+0000 mon.a (mon.0) 3520 : cluster 0 osdmap e737: 8 total, 8 up, 8 in 2024-03-04T16:08:15.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:15 smithi151 bash[23756]: cluster 2024-03-04T16:08:14.376863+0000 mon.a (mon.0) 3521 : cluster 0 osdmap e738: 8 total, 8 up, 8 in 2024-03-04T16:08:15.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:15 smithi151 bash[23756]: audit 2024-03-04T16:08:14.382923+0000 mon.a (mon.0) 3522 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:08:15.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:15 smithi151 bash[23756]: audit 2024-03-04T16:08:15.284513+0000 mon.c (mon.2) 819 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:08:15.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:15 smithi117 bash[28843]: cluster 2024-03-04T16:08:14.376863+0000 mon.a (mon.0) 3521 : cluster 0 osdmap e738: 8 total, 8 up, 8 in 2024-03-04T16:08:15.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:15 smithi117 bash[28843]: audit 2024-03-04T16:08:14.382923+0000 mon.a (mon.0) 3522 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:08:15.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:15 smithi117 bash[28843]: audit 2024-03-04T16:08:15.284513+0000 mon.c (mon.2) 819 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:08:15.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:15 smithi117 bash[21480]: cluster 2024-03-04T16:08:14.376863+0000 mon.a (mon.0) 3521 : cluster 0 osdmap e738: 8 total, 8 up, 8 in 2024-03-04T16:08:15.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:15 smithi117 bash[21480]: audit 2024-03-04T16:08:14.382923+0000 mon.a (mon.0) 3522 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:08:15.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:15 smithi117 bash[21480]: audit 2024-03-04T16:08:15.284513+0000 mon.c (mon.2) 819 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:08:16.395 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:08:16 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:08:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:08:16.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:16 smithi151 bash[23756]: cluster 2024-03-04T16:08:15.237204+0000 mgr.y (mgr.24428) 709 : cluster 0 pgmap v1197: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:08:16.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:16 smithi151 bash[23756]: audit 2024-03-04T16:08:15.372683+0000 mon.a (mon.0) 3523 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:08:16.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:16 smithi151 bash[23756]: cluster 2024-03-04T16:08:15.383199+0000 mon.a (mon.0) 3524 : cluster 0 osdmap e739: 8 total, 8 up, 8 in 2024-03-04T16:08:16.684 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:16 smithi151 bash[23756]: audit 2024-03-04T16:08:15.446538+0000 mon.a (mon.0) 3525 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:08:16.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:16 smithi117 bash[28843]: cluster 2024-03-04T16:08:15.237204+0000 mgr.y (mgr.24428) 709 : cluster 0 pgmap v1197: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:08:16.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:16 smithi117 bash[28843]: audit 2024-03-04T16:08:15.372683+0000 mon.a (mon.0) 3523 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:08:16.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:16 smithi117 bash[28843]: cluster 2024-03-04T16:08:15.383199+0000 mon.a (mon.0) 3524 : cluster 0 osdmap e739: 8 total, 8 up, 8 in 2024-03-04T16:08:16.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:16 smithi117 bash[28843]: audit 2024-03-04T16:08:15.446538+0000 mon.a (mon.0) 3525 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:08:16.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:16 smithi117 bash[21480]: cluster 2024-03-04T16:08:15.237204+0000 mgr.y (mgr.24428) 709 : cluster 0 pgmap v1197: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:08:16.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:16 smithi117 bash[21480]: audit 2024-03-04T16:08:15.372683+0000 mon.a (mon.0) 3523 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:08:16.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:16 smithi117 bash[21480]: cluster 2024-03-04T16:08:15.383199+0000 mon.a (mon.0) 3524 : cluster 0 osdmap e739: 8 total, 8 up, 8 in 2024-03-04T16:08:16.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:16 smithi117 bash[21480]: audit 2024-03-04T16:08:15.446538+0000 mon.a (mon.0) 3525 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:08:17.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:08:17 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:08:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:17 smithi151 bash[23756]: audit 2024-03-04T16:08:16.408230+0000 mon.a (mon.0) 3526 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-146", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:08:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:17 smithi151 bash[23756]: cluster 2024-03-04T16:08:16.416464+0000 mon.a (mon.0) 3527 : cluster 0 osdmap e740: 8 total, 8 up, 8 in 2024-03-04T16:08:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:17 smithi151 bash[23756]: audit 2024-03-04T16:08:16.418570+0000 mon.a (mon.0) 3528 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-146"}]: dispatch 2024-03-04T16:08:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:17 smithi151 bash[23756]: cluster 2024-03-04T16:08:17.239343+0000 mgr.y (mgr.24428) 710 : cluster 0 pgmap v1200: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-04T16:08:17.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:17 smithi151 bash[23756]: audit 2024-03-04T16:08:17.249395+0000 mgr.y (mgr.24428) 711 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:08:17.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:17 smithi117 bash[21480]: audit 2024-03-04T16:08:16.408230+0000 mon.a (mon.0) 3526 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-146", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:08:17.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:17 smithi117 bash[21480]: cluster 2024-03-04T16:08:16.416464+0000 mon.a (mon.0) 3527 : cluster 0 osdmap e740: 8 total, 8 up, 8 in 2024-03-04T16:08:17.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:17 smithi117 bash[21480]: audit 2024-03-04T16:08:16.418570+0000 mon.a (mon.0) 3528 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-146"}]: dispatch 2024-03-04T16:08:17.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:17 smithi117 bash[21480]: cluster 2024-03-04T16:08:17.239343+0000 mgr.y (mgr.24428) 710 : cluster 0 pgmap v1200: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-04T16:08:17.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:17 smithi117 bash[21480]: audit 2024-03-04T16:08:17.249395+0000 mgr.y (mgr.24428) 711 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:08:17.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:17 smithi117 bash[28843]: audit 2024-03-04T16:08:16.408230+0000 mon.a (mon.0) 3526 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-146", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:08:17.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:17 smithi117 bash[28843]: cluster 2024-03-04T16:08:16.416464+0000 mon.a (mon.0) 3527 : cluster 0 osdmap e740: 8 total, 8 up, 8 in 2024-03-04T16:08:17.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:17 smithi117 bash[28843]: audit 2024-03-04T16:08:16.418570+0000 mon.a (mon.0) 3528 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-146"}]: dispatch 2024-03-04T16:08:17.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:17 smithi117 bash[28843]: cluster 2024-03-04T16:08:17.239343+0000 mgr.y (mgr.24428) 710 : cluster 0 pgmap v1200: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-04T16:08:17.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:17 smithi117 bash[28843]: audit 2024-03-04T16:08:17.249395+0000 mgr.y (mgr.24428) 711 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:08:18.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:18 smithi151 bash[23756]: audit 2024-03-04T16:08:17.417278+0000 mon.a (mon.0) 3529 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-146"}]': finished 2024-03-04T16:08:18.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:18 smithi151 bash[23756]: cluster 2024-03-04T16:08:17.426629+0000 mon.a (mon.0) 3530 : cluster 0 osdmap e741: 8 total, 8 up, 8 in 2024-03-04T16:08:18.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:18 smithi151 bash[23756]: audit 2024-03-04T16:08:17.428276+0000 mon.a (mon.0) 3531 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-146", "mode": "writeback"}]: dispatch 2024-03-04T16:08:18.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:18 smithi117 bash[21480]: audit 2024-03-04T16:08:17.417278+0000 mon.a (mon.0) 3529 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-146"}]': finished 2024-03-04T16:08:18.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:18 smithi117 bash[21480]: cluster 2024-03-04T16:08:17.426629+0000 mon.a (mon.0) 3530 : cluster 0 osdmap e741: 8 total, 8 up, 8 in 2024-03-04T16:08:18.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:18 smithi117 bash[21480]: audit 2024-03-04T16:08:17.428276+0000 mon.a (mon.0) 3531 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-146", "mode": "writeback"}]: dispatch 2024-03-04T16:08:18.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:18 smithi117 bash[28843]: audit 2024-03-04T16:08:17.417278+0000 mon.a (mon.0) 3529 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi117-64061-111", "overlaypool": "test-rados-api-smithi117-64061-146"}]': finished 2024-03-04T16:08:18.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:18 smithi117 bash[28843]: cluster 2024-03-04T16:08:17.426629+0000 mon.a (mon.0) 3530 : cluster 0 osdmap e741: 8 total, 8 up, 8 in 2024-03-04T16:08:18.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:18 smithi117 bash[28843]: audit 2024-03-04T16:08:17.428276+0000 mon.a (mon.0) 3531 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-146", "mode": "writeback"}]: dispatch 2024-03-04T16:08:19.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:19 smithi117 bash[21480]: cluster 2024-03-04T16:08:18.417843+0000 mon.a (mon.0) 3532 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:08:19.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:19 smithi117 bash[21480]: audit 2024-03-04T16:08:18.420385+0000 mon.a (mon.0) 3533 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-146", "mode": "writeback"}]': finished 2024-03-04T16:08:19.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:19 smithi117 bash[21480]: cluster 2024-03-04T16:08:18.427858+0000 mon.a (mon.0) 3534 : cluster 0 osdmap e742: 8 total, 8 up, 8 in 2024-03-04T16:08:19.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:19 smithi117 bash[21480]: audit 2024-03-04T16:08:18.515620+0000 mon.a (mon.0) 3535 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:08:19.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:19 smithi117 bash[21480]: cluster 2024-03-04T16:08:19.240328+0000 mgr.y (mgr.24428) 712 : cluster 0 pgmap v1203: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-04T16:08:19.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:19 smithi117 bash[28843]: cluster 2024-03-04T16:08:18.417843+0000 mon.a (mon.0) 3532 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:08:19.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:19 smithi117 bash[28843]: audit 2024-03-04T16:08:18.420385+0000 mon.a (mon.0) 3533 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-146", "mode": "writeback"}]': finished 2024-03-04T16:08:19.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:19 smithi117 bash[28843]: cluster 2024-03-04T16:08:18.427858+0000 mon.a (mon.0) 3534 : cluster 0 osdmap e742: 8 total, 8 up, 8 in 2024-03-04T16:08:19.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:19 smithi117 bash[28843]: audit 2024-03-04T16:08:18.515620+0000 mon.a (mon.0) 3535 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:08:19.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:19 smithi117 bash[28843]: cluster 2024-03-04T16:08:19.240328+0000 mgr.y (mgr.24428) 712 : cluster 0 pgmap v1203: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-04T16:08:19.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:19 smithi151 bash[23756]: cluster 2024-03-04T16:08:18.417843+0000 mon.a (mon.0) 3532 : cluster 3 Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-03-04T16:08:19.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:19 smithi151 bash[23756]: audit 2024-03-04T16:08:18.420385+0000 mon.a (mon.0) 3533 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi117-64061-146", "mode": "writeback"}]': finished 2024-03-04T16:08:19.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:19 smithi151 bash[23756]: cluster 2024-03-04T16:08:18.427858+0000 mon.a (mon.0) 3534 : cluster 0 osdmap e742: 8 total, 8 up, 8 in 2024-03-04T16:08:19.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:19 smithi151 bash[23756]: audit 2024-03-04T16:08:18.515620+0000 mon.a (mon.0) 3535 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "hit_set_count","val": "2"}]: dispatch 2024-03-04T16:08:19.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:19 smithi151 bash[23756]: cluster 2024-03-04T16:08:19.240328+0000 mgr.y (mgr.24428) 712 : cluster 0 pgmap v1203: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1023 B/s wr, 2 op/s 2024-03-04T16:08:20.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:20 smithi117 bash[28843]: audit 2024-03-04T16:08:19.440721+0000 mon.a (mon.0) 3536 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "hit_set_count","val": "2"}]': finished 2024-03-04T16:08:20.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:20 smithi117 bash[28843]: cluster 2024-03-04T16:08:19.450106+0000 mon.a (mon.0) 3537 : cluster 0 osdmap e743: 8 total, 8 up, 8 in 2024-03-04T16:08:20.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:20 smithi117 bash[28843]: audit 2024-03-04T16:08:19.450963+0000 mon.a (mon.0) 3538 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:08:20.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:20 smithi117 bash[28843]: cluster 2024-03-04T16:08:19.811343+0000 mon.a (mon.0) 3539 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:08:20.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:20 smithi117 bash[21480]: audit 2024-03-04T16:08:19.440721+0000 mon.a (mon.0) 3536 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "hit_set_count","val": "2"}]': finished 2024-03-04T16:08:20.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:20 smithi117 bash[21480]: cluster 2024-03-04T16:08:19.450106+0000 mon.a (mon.0) 3537 : cluster 0 osdmap e743: 8 total, 8 up, 8 in 2024-03-04T16:08:20.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:20 smithi117 bash[21480]: audit 2024-03-04T16:08:19.450963+0000 mon.a (mon.0) 3538 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:08:20.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:20 smithi117 bash[21480]: cluster 2024-03-04T16:08:19.811343+0000 mon.a (mon.0) 3539 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:08:20.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:20 smithi151 bash[23756]: audit 2024-03-04T16:08:19.440721+0000 mon.a (mon.0) 3536 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "hit_set_count","val": "2"}]': finished 2024-03-04T16:08:20.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:20 smithi151 bash[23756]: cluster 2024-03-04T16:08:19.450106+0000 mon.a (mon.0) 3537 : cluster 0 osdmap e743: 8 total, 8 up, 8 in 2024-03-04T16:08:20.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:20 smithi151 bash[23756]: audit 2024-03-04T16:08:19.450963+0000 mon.a (mon.0) 3538 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "hit_set_period","val": "600"}]: dispatch 2024-03-04T16:08:20.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:20 smithi151 bash[23756]: cluster 2024-03-04T16:08:19.811343+0000 mon.a (mon.0) 3539 : cluster 3 Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:08:21.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:21 smithi117 bash[21480]: audit 2024-03-04T16:08:20.452243+0000 mon.a (mon.0) 3540 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "hit_set_period","val": "600"}]': finished 2024-03-04T16:08:21.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:21 smithi117 bash[21480]: cluster 2024-03-04T16:08:20.460017+0000 mon.a (mon.0) 3541 : cluster 0 osdmap e744: 8 total, 8 up, 8 in 2024-03-04T16:08:21.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:21 smithi117 bash[21480]: audit 2024-03-04T16:08:20.463423+0000 mon.a (mon.0) 3542 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T16:08:21.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:21 smithi117 bash[21480]: cluster 2024-03-04T16:08:21.241316+0000 mgr.y (mgr.24428) 713 : cluster 0 pgmap v1206: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:08:21.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:21 smithi117 bash[28843]: audit 2024-03-04T16:08:20.452243+0000 mon.a (mon.0) 3540 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "hit_set_period","val": "600"}]': finished 2024-03-04T16:08:21.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:21 smithi117 bash[28843]: cluster 2024-03-04T16:08:20.460017+0000 mon.a (mon.0) 3541 : cluster 0 osdmap e744: 8 total, 8 up, 8 in 2024-03-04T16:08:21.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:21 smithi117 bash[28843]: audit 2024-03-04T16:08:20.463423+0000 mon.a (mon.0) 3542 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T16:08:21.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:21 smithi117 bash[28843]: cluster 2024-03-04T16:08:21.241316+0000 mgr.y (mgr.24428) 713 : cluster 0 pgmap v1206: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:08:21.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:21 smithi151 bash[23756]: audit 2024-03-04T16:08:20.452243+0000 mon.a (mon.0) 3540 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "hit_set_period","val": "600"}]': finished 2024-03-04T16:08:21.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:21 smithi151 bash[23756]: cluster 2024-03-04T16:08:20.460017+0000 mon.a (mon.0) 3541 : cluster 0 osdmap e744: 8 total, 8 up, 8 in 2024-03-04T16:08:21.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:21 smithi151 bash[23756]: audit 2024-03-04T16:08:20.463423+0000 mon.a (mon.0) 3542 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-03-04T16:08:21.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:21 smithi151 bash[23756]: cluster 2024-03-04T16:08:21.241316+0000 mgr.y (mgr.24428) 713 : cluster 0 pgmap v1206: 204 pgs: 204 active+clean; 4.4 MiB data, 750 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:08:22.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:22 smithi117 bash[21480]: cluster 2024-03-04T16:08:21.453174+0000 mon.a (mon.0) 3543 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:08:22.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:22 smithi117 bash[21480]: audit 2024-03-04T16:08:21.460680+0000 mon.a (mon.0) 3544 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T16:08:22.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:22 smithi117 bash[21480]: cluster 2024-03-04T16:08:21.471111+0000 mon.a (mon.0) 3545 : cluster 0 osdmap e745: 8 total, 8 up, 8 in 2024-03-04T16:08:22.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:22 smithi117 bash[21480]: audit 2024-03-04T16:08:21.472272+0000 mon.a (mon.0) 3546 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-04T16:08:22.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:22 smithi117 bash[28843]: cluster 2024-03-04T16:08:21.453174+0000 mon.a (mon.0) 3543 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:08:22.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:22 smithi117 bash[28843]: audit 2024-03-04T16:08:21.460680+0000 mon.a (mon.0) 3544 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T16:08:22.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:22 smithi117 bash[28843]: cluster 2024-03-04T16:08:21.471111+0000 mon.a (mon.0) 3545 : cluster 0 osdmap e745: 8 total, 8 up, 8 in 2024-03-04T16:08:22.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:22 smithi117 bash[28843]: audit 2024-03-04T16:08:21.472272+0000 mon.a (mon.0) 3546 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-04T16:08:22.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:22 smithi151 bash[23756]: cluster 2024-03-04T16:08:21.453174+0000 mon.a (mon.0) 3543 : cluster 1 Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-03-04T16:08:22.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:22 smithi151 bash[23756]: audit 2024-03-04T16:08:21.460680+0000 mon.a (mon.0) 3544 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "hit_set_type","val": "bloom"}]': finished 2024-03-04T16:08:22.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:22 smithi151 bash[23756]: cluster 2024-03-04T16:08:21.471111+0000 mon.a (mon.0) 3545 : cluster 0 osdmap e745: 8 total, 8 up, 8 in 2024-03-04T16:08:22.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:22 smithi151 bash[23756]: audit 2024-03-04T16:08:21.472272+0000 mon.a (mon.0) 3546 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-03-04T16:08:23.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:23 smithi117 bash[28843]: audit 2024-03-04T16:08:22.463230+0000 mon.a (mon.0) 3547 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-04T16:08:23.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:23 smithi117 bash[28843]: cluster 2024-03-04T16:08:22.471883+0000 mon.a (mon.0) 3548 : cluster 0 osdmap e746: 8 total, 8 up, 8 in 2024-03-04T16:08:23.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:23 smithi117 bash[28843]: audit 2024-03-04T16:08:22.473892+0000 mon.a (mon.0) 3549 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-04T16:08:23.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:23 smithi117 bash[28843]: cluster 2024-03-04T16:08:23.242124+0000 mgr.y (mgr.24428) 714 : cluster 0 pgmap v1209: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-04T16:08:23.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:23 smithi117 bash[28843]: audit 2024-03-04T16:08:23.466011+0000 mon.a (mon.0) 3550 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "target_max_objects","val": "1"}]': finished 2024-03-04T16:08:23.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:23 smithi117 bash[28843]: cluster 2024-03-04T16:08:23.473523+0000 mon.a (mon.0) 3551 : cluster 0 osdmap e747: 8 total, 8 up, 8 in 2024-03-04T16:08:23.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:23 smithi117 bash[21480]: audit 2024-03-04T16:08:22.463230+0000 mon.a (mon.0) 3547 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-04T16:08:23.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:23 smithi117 bash[21480]: cluster 2024-03-04T16:08:22.471883+0000 mon.a (mon.0) 3548 : cluster 0 osdmap e746: 8 total, 8 up, 8 in 2024-03-04T16:08:23.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:23 smithi117 bash[21480]: audit 2024-03-04T16:08:22.473892+0000 mon.a (mon.0) 3549 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-04T16:08:23.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:23 smithi117 bash[21480]: cluster 2024-03-04T16:08:23.242124+0000 mgr.y (mgr.24428) 714 : cluster 0 pgmap v1209: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-04T16:08:23.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:23 smithi117 bash[21480]: audit 2024-03-04T16:08:23.466011+0000 mon.a (mon.0) 3550 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "target_max_objects","val": "1"}]': finished 2024-03-04T16:08:23.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:23 smithi117 bash[21480]: cluster 2024-03-04T16:08:23.473523+0000 mon.a (mon.0) 3551 : cluster 0 osdmap e747: 8 total, 8 up, 8 in 2024-03-04T16:08:23.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:23 smithi151 bash[23756]: audit 2024-03-04T16:08:22.463230+0000 mon.a (mon.0) 3547 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-03-04T16:08:23.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:23 smithi151 bash[23756]: cluster 2024-03-04T16:08:22.471883+0000 mon.a (mon.0) 3548 : cluster 0 osdmap e746: 8 total, 8 up, 8 in 2024-03-04T16:08:23.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:23 smithi151 bash[23756]: audit 2024-03-04T16:08:22.473892+0000 mon.a (mon.0) 3549 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "target_max_objects","val": "1"}]: dispatch 2024-03-04T16:08:23.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:23 smithi151 bash[23756]: cluster 2024-03-04T16:08:23.242124+0000 mgr.y (mgr.24428) 714 : cluster 0 pgmap v1209: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 2.2 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-04T16:08:23.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:23 smithi151 bash[23756]: audit 2024-03-04T16:08:23.466011+0000 mon.a (mon.0) 3550 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-146","var": "target_max_objects","val": "1"}]': finished 2024-03-04T16:08:23.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:23 smithi151 bash[23756]: cluster 2024-03-04T16:08:23.473523+0000 mon.a (mon.0) 3551 : cluster 0 osdmap e747: 8 total, 8 up, 8 in 2024-03-04T16:08:25.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:25 smithi151 bash[23756]: cluster 2024-03-04T16:08:25.295921+0000 mon.a (mon.0) 3552 : cluster 3 Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-04T16:08:25.817 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:25 smithi117 bash[21480]: cluster 2024-03-04T16:08:25.295921+0000 mon.a (mon.0) 3552 : cluster 3 Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-04T16:08:25.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:25 smithi117 bash[28843]: cluster 2024-03-04T16:08:25.295921+0000 mon.a (mon.0) 3552 : cluster 3 Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-03-04T16:08:26.350 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:08:26 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:08:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:08:26.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:26 smithi151 bash[23756]: cluster 2024-03-04T16:08:25.243089+0000 mgr.y (mgr.24428) 715 : cluster 0 pgmap v1211: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-04T16:08:26.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:26 smithi117 bash[21480]: cluster 2024-03-04T16:08:25.243089+0000 mgr.y (mgr.24428) 715 : cluster 0 pgmap v1211: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-04T16:08:26.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:26 smithi117 bash[28843]: cluster 2024-03-04T16:08:25.243089+0000 mgr.y (mgr.24428) 715 : cluster 0 pgmap v1211: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.9 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-04T16:08:27.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:08:27 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:08:28.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:28 smithi151 bash[23756]: cluster 2024-03-04T16:08:27.245328+0000 mgr.y (mgr.24428) 716 : cluster 0 pgmap v1212: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-04T16:08:28.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:28 smithi151 bash[23756]: audit 2024-03-04T16:08:27.260822+0000 mgr.y (mgr.24428) 717 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:08:28.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:28 smithi117 bash[21480]: cluster 2024-03-04T16:08:27.245328+0000 mgr.y (mgr.24428) 716 : cluster 0 pgmap v1212: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-04T16:08:28.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:28 smithi117 bash[21480]: audit 2024-03-04T16:08:27.260822+0000 mgr.y (mgr.24428) 717 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:08:28.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:28 smithi117 bash[28843]: cluster 2024-03-04T16:08:27.245328+0000 mgr.y (mgr.24428) 716 : cluster 0 pgmap v1212: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 2.3 KiB/s rd, 0 B/s wr, 3 op/s 2024-03-04T16:08:28.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:28 smithi117 bash[28843]: audit 2024-03-04T16:08:27.260822+0000 mgr.y (mgr.24428) 717 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:08:30.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:30 smithi151 bash[23756]: cluster 2024-03-04T16:08:29.246191+0000 mgr.y (mgr.24428) 718 : cluster 0 pgmap v1213: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-04T16:08:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:30 smithi151 bash[23756]: audit 2024-03-04T16:08:30.325933+0000 mon.a (mon.0) 3553 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:08:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:30 smithi151 bash[23756]: audit 2024-03-04T16:08:30.331188+0000 mon.c (mon.2) 820 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:08:30.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:30 smithi117 bash[21480]: cluster 2024-03-04T16:08:29.246191+0000 mgr.y (mgr.24428) 718 : cluster 0 pgmap v1213: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-04T16:08:30.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:30 smithi117 bash[21480]: audit 2024-03-04T16:08:30.325933+0000 mon.a (mon.0) 3553 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:08:30.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:30 smithi117 bash[21480]: audit 2024-03-04T16:08:30.331188+0000 mon.c (mon.2) 820 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:08:30.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:30 smithi117 bash[28843]: cluster 2024-03-04T16:08:29.246191+0000 mgr.y (mgr.24428) 718 : cluster 0 pgmap v1213: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1.8 KiB/s rd, 0 B/s wr, 2 op/s 2024-03-04T16:08:30.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:30 smithi117 bash[28843]: audit 2024-03-04T16:08:30.325933+0000 mon.a (mon.0) 3553 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:08:30.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:30 smithi117 bash[28843]: audit 2024-03-04T16:08:30.331188+0000 mon.c (mon.2) 820 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:08:32.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:32 smithi151 bash[23756]: cluster 2024-03-04T16:08:31.247032+0000 mgr.y (mgr.24428) 719 : cluster 0 pgmap v1214: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 583 B/s rd, 0 B/s wr, 0 op/s 2024-03-04T16:08:32.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:32 smithi117 bash[21480]: cluster 2024-03-04T16:08:31.247032+0000 mgr.y (mgr.24428) 719 : cluster 0 pgmap v1214: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 583 B/s rd, 0 B/s wr, 0 op/s 2024-03-04T16:08:32.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:32 smithi117 bash[28843]: cluster 2024-03-04T16:08:31.247032+0000 mgr.y (mgr.24428) 719 : cluster 0 pgmap v1214: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 583 B/s rd, 0 B/s wr, 0 op/s 2024-03-04T16:08:34.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:34 smithi151 bash[23756]: cluster 2024-03-04T16:08:33.249264+0000 mgr.y (mgr.24428) 720 : cluster 0 pgmap v1215: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:08:34.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:34 smithi151 bash[23756]: audit 2024-03-04T16:08:33.496310+0000 mon.a (mon.0) 3554 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:34.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:34 smithi117 bash[21480]: cluster 2024-03-04T16:08:33.249264+0000 mgr.y (mgr.24428) 720 : cluster 0 pgmap v1215: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:08:34.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:34 smithi117 bash[21480]: audit 2024-03-04T16:08:33.496310+0000 mon.a (mon.0) 3554 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:34.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:34 smithi117 bash[28843]: cluster 2024-03-04T16:08:33.249264+0000 mgr.y (mgr.24428) 720 : cluster 0 pgmap v1215: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:08:34.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:34 smithi117 bash[28843]: audit 2024-03-04T16:08:33.496310+0000 mon.a (mon.0) 3554 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:35.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:35 smithi151 bash[23756]: audit 2024-03-04T16:08:34.371084+0000 mon.a (mon.0) 3555 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:08:35.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:35 smithi151 bash[23756]: cluster 2024-03-04T16:08:34.378659+0000 mon.a (mon.0) 3556 : cluster 0 osdmap e748: 8 total, 8 up, 8 in 2024-03-04T16:08:35.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:35 smithi151 bash[23756]: audit 2024-03-04T16:08:34.381200+0000 mon.a (mon.0) 3557 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-146"}]: dispatch 2024-03-04T16:08:35.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:35 smithi151 bash[23756]: audit 2024-03-04T16:08:35.373854+0000 mon.a (mon.0) 3558 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-146"}]': finished 2024-03-04T16:08:35.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:35 smithi117 bash[21480]: audit 2024-03-04T16:08:34.371084+0000 mon.a (mon.0) 3555 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:08:35.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:35 smithi117 bash[21480]: cluster 2024-03-04T16:08:34.378659+0000 mon.a (mon.0) 3556 : cluster 0 osdmap e748: 8 total, 8 up, 8 in 2024-03-04T16:08:35.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:35 smithi117 bash[21480]: audit 2024-03-04T16:08:34.381200+0000 mon.a (mon.0) 3557 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-146"}]: dispatch 2024-03-04T16:08:35.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:35 smithi117 bash[21480]: audit 2024-03-04T16:08:35.373854+0000 mon.a (mon.0) 3558 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-146"}]': finished 2024-03-04T16:08:35.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:35 smithi117 bash[28843]: audit 2024-03-04T16:08:34.371084+0000 mon.a (mon.0) 3555 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:08:35.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:35 smithi117 bash[28843]: cluster 2024-03-04T16:08:34.378659+0000 mon.a (mon.0) 3556 : cluster 0 osdmap e748: 8 total, 8 up, 8 in 2024-03-04T16:08:35.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:35 smithi117 bash[28843]: audit 2024-03-04T16:08:34.381200+0000 mon.a (mon.0) 3557 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-146"}]: dispatch 2024-03-04T16:08:35.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:35 smithi117 bash[28843]: audit 2024-03-04T16:08:35.373854+0000 mon.a (mon.0) 3558 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-146"}]': finished 2024-03-04T16:08:36.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:08:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:08:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:08:37.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:36 smithi151 bash[23756]: cluster 2024-03-04T16:08:35.250246+0000 mgr.y (mgr.24428) 721 : cluster 0 pgmap v1217: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:08:37.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:36 smithi151 bash[23756]: cluster 2024-03-04T16:08:35.381184+0000 mon.a (mon.0) 3559 : cluster 0 osdmap e749: 8 total, 8 up, 8 in 2024-03-04T16:08:37.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:36 smithi151 bash[23756]: audit 2024-03-04T16:08:35.455608+0000 mon.a (mon.0) 3560 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:37.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:36 smithi151 bash[23756]: audit 2024-03-04T16:08:35.456331+0000 mon.a (mon.0) 3561 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-146"}]: dispatch 2024-03-04T16:08:37.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:36 smithi117 bash[21480]: cluster 2024-03-04T16:08:35.250246+0000 mgr.y (mgr.24428) 721 : cluster 0 pgmap v1217: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:08:37.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:36 smithi117 bash[21480]: cluster 2024-03-04T16:08:35.381184+0000 mon.a (mon.0) 3559 : cluster 0 osdmap e749: 8 total, 8 up, 8 in 2024-03-04T16:08:37.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:36 smithi117 bash[21480]: audit 2024-03-04T16:08:35.455608+0000 mon.a (mon.0) 3560 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:37.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:36 smithi117 bash[21480]: audit 2024-03-04T16:08:35.456331+0000 mon.a (mon.0) 3561 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-146"}]: dispatch 2024-03-04T16:08:37.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:36 smithi117 bash[28843]: cluster 2024-03-04T16:08:35.250246+0000 mgr.y (mgr.24428) 721 : cluster 0 pgmap v1217: 204 pgs: 204 active+clean; 4.4 MiB data, 751 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:08:37.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:36 smithi117 bash[28843]: cluster 2024-03-04T16:08:35.381184+0000 mon.a (mon.0) 3559 : cluster 0 osdmap e749: 8 total, 8 up, 8 in 2024-03-04T16:08:37.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:36 smithi117 bash[28843]: audit 2024-03-04T16:08:35.455608+0000 mon.a (mon.0) 3560 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:37.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:36 smithi117 bash[28843]: audit 2024-03-04T16:08:35.456331+0000 mon.a (mon.0) 3561 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-146"}]: dispatch 2024-03-04T16:08:37.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:08:37 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:08:38.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:37 smithi151 bash[23756]: cluster 2024-03-04T16:08:36.405805+0000 mon.a (mon.0) 3562 : cluster 0 osdmap e750: 8 total, 8 up, 8 in 2024-03-04T16:08:38.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:37 smithi151 bash[23756]: cluster 2024-03-04T16:08:37.252133+0000 mgr.y (mgr.24428) 722 : cluster 0 pgmap v1220: 172 pgs: 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:08:38.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:37 smithi151 bash[23756]: audit 2024-03-04T16:08:37.270816+0000 mgr.y (mgr.24428) 723 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:08:38.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:37 smithi151 bash[23756]: cluster 2024-03-04T16:08:37.412584+0000 mon.a (mon.0) 3563 : cluster 0 osdmap e751: 8 total, 8 up, 8 in 2024-03-04T16:08:38.184 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:37 smithi151 bash[23756]: audit 2024-03-04T16:08:37.417577+0000 mon.a (mon.0) 3564 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:08:38.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:37 smithi117 bash[21480]: cluster 2024-03-04T16:08:36.405805+0000 mon.a (mon.0) 3562 : cluster 0 osdmap e750: 8 total, 8 up, 8 in 2024-03-04T16:08:38.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:37 smithi117 bash[21480]: cluster 2024-03-04T16:08:37.252133+0000 mgr.y (mgr.24428) 722 : cluster 0 pgmap v1220: 172 pgs: 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:08:38.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:37 smithi117 bash[21480]: audit 2024-03-04T16:08:37.270816+0000 mgr.y (mgr.24428) 723 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:08:38.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:37 smithi117 bash[21480]: cluster 2024-03-04T16:08:37.412584+0000 mon.a (mon.0) 3563 : cluster 0 osdmap e751: 8 total, 8 up, 8 in 2024-03-04T16:08:38.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:37 smithi117 bash[21480]: audit 2024-03-04T16:08:37.417577+0000 mon.a (mon.0) 3564 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:08:38.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:37 smithi117 bash[28843]: cluster 2024-03-04T16:08:36.405805+0000 mon.a (mon.0) 3562 : cluster 0 osdmap e750: 8 total, 8 up, 8 in 2024-03-04T16:08:38.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:37 smithi117 bash[28843]: cluster 2024-03-04T16:08:37.252133+0000 mgr.y (mgr.24428) 722 : cluster 0 pgmap v1220: 172 pgs: 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:08:38.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:37 smithi117 bash[28843]: audit 2024-03-04T16:08:37.270816+0000 mgr.y (mgr.24428) 723 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:08:38.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:37 smithi117 bash[28843]: cluster 2024-03-04T16:08:37.412584+0000 mon.a (mon.0) 3563 : cluster 0 osdmap e751: 8 total, 8 up, 8 in 2024-03-04T16:08:38.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:37 smithi117 bash[28843]: audit 2024-03-04T16:08:37.417577+0000 mon.a (mon.0) 3564 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:08:39.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:38 smithi151 bash[23756]: cluster 2024-03-04T16:08:37.861468+0000 mon.a (mon.0) 3565 : cluster 1 Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-04T16:08:39.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:38 smithi151 bash[23756]: audit 2024-03-04T16:08:38.407047+0000 mon.a (mon.0) 3566 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:08:39.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:38 smithi151 bash[23756]: cluster 2024-03-04T16:08:38.415401+0000 mon.a (mon.0) 3567 : cluster 0 osdmap e752: 8 total, 8 up, 8 in 2024-03-04T16:08:39.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:38 smithi151 bash[23756]: audit 2024-03-04T16:08:38.468672+0000 mon.a (mon.0) 3568 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:08:39.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:38 smithi117 bash[21480]: cluster 2024-03-04T16:08:37.861468+0000 mon.a (mon.0) 3565 : cluster 1 Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-04T16:08:39.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:38 smithi117 bash[21480]: audit 2024-03-04T16:08:38.407047+0000 mon.a (mon.0) 3566 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:08:39.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:38 smithi117 bash[21480]: cluster 2024-03-04T16:08:38.415401+0000 mon.a (mon.0) 3567 : cluster 0 osdmap e752: 8 total, 8 up, 8 in 2024-03-04T16:08:39.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:38 smithi117 bash[21480]: audit 2024-03-04T16:08:38.468672+0000 mon.a (mon.0) 3568 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:08:39.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:38 smithi117 bash[28843]: cluster 2024-03-04T16:08:37.861468+0000 mon.a (mon.0) 3565 : cluster 1 Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-03-04T16:08:39.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:38 smithi117 bash[28843]: audit 2024-03-04T16:08:38.407047+0000 mon.a (mon.0) 3566 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:08:39.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:38 smithi117 bash[28843]: cluster 2024-03-04T16:08:38.415401+0000 mon.a (mon.0) 3567 : cluster 0 osdmap e752: 8 total, 8 up, 8 in 2024-03-04T16:08:39.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:38 smithi117 bash[28843]: audit 2024-03-04T16:08:38.468672+0000 mon.a (mon.0) 3568 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-03-04T16:08:40.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:39 smithi151 bash[23756]: cluster 2024-03-04T16:08:39.252946+0000 mgr.y (mgr.24428) 724 : cluster 0 pgmap v1223: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-04T16:08:40.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:39 smithi151 bash[23756]: audit 2024-03-04T16:08:39.410081+0000 mon.a (mon.0) 3569 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-148", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:08:40.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:39 smithi151 bash[23756]: cluster 2024-03-04T16:08:39.418404+0000 mon.a (mon.0) 3570 : cluster 0 osdmap e753: 8 total, 8 up, 8 in 2024-03-04T16:08:40.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:39 smithi151 bash[23756]: audit 2024-03-04T16:08:39.442222+0000 mon.a (mon.0) 3571 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-148"}]: dispatch 2024-03-04T16:08:40.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:39 smithi117 bash[28843]: cluster 2024-03-04T16:08:39.252946+0000 mgr.y (mgr.24428) 724 : cluster 0 pgmap v1223: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-04T16:08:40.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:39 smithi117 bash[28843]: audit 2024-03-04T16:08:39.410081+0000 mon.a (mon.0) 3569 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-148", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:08:40.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:39 smithi117 bash[28843]: cluster 2024-03-04T16:08:39.418404+0000 mon.a (mon.0) 3570 : cluster 0 osdmap e753: 8 total, 8 up, 8 in 2024-03-04T16:08:40.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:39 smithi117 bash[28843]: audit 2024-03-04T16:08:39.442222+0000 mon.a (mon.0) 3571 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-148"}]: dispatch 2024-03-04T16:08:40.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:39 smithi117 bash[21480]: cluster 2024-03-04T16:08:39.252946+0000 mgr.y (mgr.24428) 724 : cluster 0 pgmap v1223: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail; 255 B/s rd, 0 op/s 2024-03-04T16:08:40.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:39 smithi117 bash[21480]: audit 2024-03-04T16:08:39.410081+0000 mon.a (mon.0) 3569 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-148", "force_nonempty": "--force-nonempty" }]': finished 2024-03-04T16:08:40.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:39 smithi117 bash[21480]: cluster 2024-03-04T16:08:39.418404+0000 mon.a (mon.0) 3570 : cluster 0 osdmap e753: 8 total, 8 up, 8 in 2024-03-04T16:08:40.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:39 smithi117 bash[21480]: audit 2024-03-04T16:08:39.442222+0000 mon.a (mon.0) 3571 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-148"}]: dispatch 2024-03-04T16:08:41.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:41 smithi151 bash[23756]: audit 2024-03-04T16:08:40.412722+0000 mon.a (mon.0) 3572 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-148"}]': finished 2024-03-04T16:08:41.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:41 smithi151 bash[23756]: cluster 2024-03-04T16:08:40.419487+0000 mon.a (mon.0) 3573 : cluster 0 osdmap e754: 8 total, 8 up, 8 in 2024-03-04T16:08:41.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:41 smithi151 bash[23756]: audit 2024-03-04T16:08:40.485476+0000 mon.a (mon.0) 3574 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:41.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:41 smithi151 bash[23756]: audit 2024-03-04T16:08:40.486501+0000 mon.a (mon.0) 3575 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-148"}]: dispatch 2024-03-04T16:08:41.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:41 smithi151 bash[23756]: cluster 2024-03-04T16:08:41.253784+0000 mgr.y (mgr.24428) 725 : cluster 0 pgmap v1226: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:08:41.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:41 smithi117 bash[21480]: audit 2024-03-04T16:08:40.412722+0000 mon.a (mon.0) 3572 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-148"}]': finished 2024-03-04T16:08:41.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:41 smithi117 bash[21480]: cluster 2024-03-04T16:08:40.419487+0000 mon.a (mon.0) 3573 : cluster 0 osdmap e754: 8 total, 8 up, 8 in 2024-03-04T16:08:41.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:41 smithi117 bash[21480]: audit 2024-03-04T16:08:40.485476+0000 mon.a (mon.0) 3574 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:41.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:41 smithi117 bash[21480]: audit 2024-03-04T16:08:40.486501+0000 mon.a (mon.0) 3575 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-148"}]: dispatch 2024-03-04T16:08:41.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:41 smithi117 bash[21480]: cluster 2024-03-04T16:08:41.253784+0000 mgr.y (mgr.24428) 725 : cluster 0 pgmap v1226: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:08:41.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:41 smithi117 bash[28843]: audit 2024-03-04T16:08:40.412722+0000 mon.a (mon.0) 3572 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-148"}]': finished 2024-03-04T16:08:41.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:41 smithi117 bash[28843]: cluster 2024-03-04T16:08:40.419487+0000 mon.a (mon.0) 3573 : cluster 0 osdmap e754: 8 total, 8 up, 8 in 2024-03-04T16:08:41.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:41 smithi117 bash[28843]: audit 2024-03-04T16:08:40.485476+0000 mon.a (mon.0) 3574 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:41.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:41 smithi117 bash[28843]: audit 2024-03-04T16:08:40.486501+0000 mon.a (mon.0) 3575 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-148"}]: dispatch 2024-03-04T16:08:41.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:41 smithi117 bash[28843]: cluster 2024-03-04T16:08:41.253784+0000 mgr.y (mgr.24428) 725 : cluster 0 pgmap v1226: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 752 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:08:42.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:42 smithi117 bash[21480]: cluster 2024-03-04T16:08:41.412759+0000 mon.a (mon.0) 3576 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:08:42.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:42 smithi117 bash[21480]: cluster 2024-03-04T16:08:41.431166+0000 mon.a (mon.0) 3577 : cluster 0 osdmap e755: 8 total, 8 up, 8 in 2024-03-04T16:08:42.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:42 smithi117 bash[28843]: cluster 2024-03-04T16:08:41.412759+0000 mon.a (mon.0) 3576 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:08:42.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:42 smithi117 bash[28843]: cluster 2024-03-04T16:08:41.431166+0000 mon.a (mon.0) 3577 : cluster 0 osdmap e755: 8 total, 8 up, 8 in 2024-03-04T16:08:42.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:42 smithi151 bash[23756]: cluster 2024-03-04T16:08:41.412759+0000 mon.a (mon.0) 3576 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:08:42.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:42 smithi151 bash[23756]: cluster 2024-03-04T16:08:41.431166+0000 mon.a (mon.0) 3577 : cluster 0 osdmap e755: 8 total, 8 up, 8 in 2024-03-04T16:08:43.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:43 smithi117 bash[21480]: cluster 2024-03-04T16:08:42.441601+0000 mon.a (mon.0) 3578 : cluster 0 osdmap e756: 8 total, 8 up, 8 in 2024-03-04T16:08:43.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:43 smithi117 bash[21480]: audit 2024-03-04T16:08:42.459354+0000 mon.a (mon.0) 3579 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:08:43.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:43 smithi117 bash[21480]: cluster 2024-03-04T16:08:43.254832+0000 mgr.y (mgr.24428) 726 : cluster 0 pgmap v1229: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 770 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-04T16:08:43.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:43 smithi117 bash[21480]: audit 2024-03-04T16:08:43.429432+0000 mon.a (mon.0) 3580 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:08:43.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:43 smithi117 bash[21480]: cluster 2024-03-04T16:08:43.437617+0000 mon.a (mon.0) 3581 : cluster 0 osdmap e757: 8 total, 8 up, 8 in 2024-03-04T16:08:43.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:43 smithi117 bash[28843]: cluster 2024-03-04T16:08:42.441601+0000 mon.a (mon.0) 3578 : cluster 0 osdmap e756: 8 total, 8 up, 8 in 2024-03-04T16:08:43.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:43 smithi117 bash[28843]: audit 2024-03-04T16:08:42.459354+0000 mon.a (mon.0) 3579 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:08:43.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:43 smithi117 bash[28843]: cluster 2024-03-04T16:08:43.254832+0000 mgr.y (mgr.24428) 726 : cluster 0 pgmap v1229: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 770 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-04T16:08:43.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:43 smithi117 bash[28843]: audit 2024-03-04T16:08:43.429432+0000 mon.a (mon.0) 3580 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:08:43.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:43 smithi117 bash[28843]: cluster 2024-03-04T16:08:43.437617+0000 mon.a (mon.0) 3581 : cluster 0 osdmap e757: 8 total, 8 up, 8 in 2024-03-04T16:08:43.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:43 smithi151 bash[23756]: cluster 2024-03-04T16:08:42.441601+0000 mon.a (mon.0) 3578 : cluster 0 osdmap e756: 8 total, 8 up, 8 in 2024-03-04T16:08:43.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:43 smithi151 bash[23756]: audit 2024-03-04T16:08:42.459354+0000 mon.a (mon.0) 3579 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:08:43.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:43 smithi151 bash[23756]: cluster 2024-03-04T16:08:43.254832+0000 mgr.y (mgr.24428) 726 : cluster 0 pgmap v1229: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 770 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-04T16:08:43.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:43 smithi151 bash[23756]: audit 2024-03-04T16:08:43.429432+0000 mon.a (mon.0) 3580 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:08:43.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:43 smithi151 bash[23756]: cluster 2024-03-04T16:08:43.437617+0000 mon.a (mon.0) 3581 : cluster 0 osdmap e757: 8 total, 8 up, 8 in 2024-03-04T16:08:45.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:44 smithi151 bash[23756]: audit 2024-03-04T16:08:43.569206+0000 mon.a (mon.0) 3582 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:45.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:44 smithi151 bash[23756]: audit 2024-03-04T16:08:43.570015+0000 mon.a (mon.0) 3583 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-150"}]: dispatch 2024-03-04T16:08:45.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:44 smithi151 bash[23756]: cluster 2024-03-04T16:08:44.439357+0000 mon.a (mon.0) 3584 : cluster 0 osdmap e758: 8 total, 8 up, 8 in 2024-03-04T16:08:45.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:44 smithi117 bash[21480]: audit 2024-03-04T16:08:43.569206+0000 mon.a (mon.0) 3582 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:45.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:44 smithi117 bash[21480]: audit 2024-03-04T16:08:43.570015+0000 mon.a (mon.0) 3583 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-150"}]: dispatch 2024-03-04T16:08:45.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:44 smithi117 bash[21480]: cluster 2024-03-04T16:08:44.439357+0000 mon.a (mon.0) 3584 : cluster 0 osdmap e758: 8 total, 8 up, 8 in 2024-03-04T16:08:45.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:44 smithi117 bash[28843]: audit 2024-03-04T16:08:43.569206+0000 mon.a (mon.0) 3582 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:45.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:44 smithi117 bash[28843]: audit 2024-03-04T16:08:43.570015+0000 mon.a (mon.0) 3583 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-150"}]: dispatch 2024-03-04T16:08:45.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:44 smithi117 bash[28843]: cluster 2024-03-04T16:08:44.439357+0000 mon.a (mon.0) 3584 : cluster 0 osdmap e758: 8 total, 8 up, 8 in 2024-03-04T16:08:46.388 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:08:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:08:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:08:46.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:46 smithi151 bash[23756]: cluster 2024-03-04T16:08:45.255709+0000 mgr.y (mgr.24428) 727 : cluster 0 pgmap v1232: 172 pgs: 172 active+clean; 4.4 MiB data, 770 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-04T16:08:46.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:46 smithi151 bash[23756]: audit 2024-03-04T16:08:45.392715+0000 mon.a (mon.0) 3585 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:08:46.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:46 smithi151 bash[23756]: audit 2024-03-04T16:08:45.395699+0000 mon.c (mon.2) 821 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:08:46.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:46 smithi151 bash[23756]: cluster 2024-03-04T16:08:45.444404+0000 mon.a (mon.0) 3586 : cluster 0 osdmap e759: 8 total, 8 up, 8 in 2024-03-04T16:08:46.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:46 smithi151 bash[23756]: audit 2024-03-04T16:08:45.452236+0000 mon.a (mon.0) 3587 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:08:46.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:46 smithi117 bash[21480]: cluster 2024-03-04T16:08:45.255709+0000 mgr.y (mgr.24428) 727 : cluster 0 pgmap v1232: 172 pgs: 172 active+clean; 4.4 MiB data, 770 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-04T16:08:46.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:46 smithi117 bash[21480]: audit 2024-03-04T16:08:45.392715+0000 mon.a (mon.0) 3585 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:08:46.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:46 smithi117 bash[21480]: audit 2024-03-04T16:08:45.395699+0000 mon.c (mon.2) 821 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:08:46.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:46 smithi117 bash[21480]: cluster 2024-03-04T16:08:45.444404+0000 mon.a (mon.0) 3586 : cluster 0 osdmap e759: 8 total, 8 up, 8 in 2024-03-04T16:08:46.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:46 smithi117 bash[21480]: audit 2024-03-04T16:08:45.452236+0000 mon.a (mon.0) 3587 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:08:46.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:46 smithi117 bash[28843]: cluster 2024-03-04T16:08:45.255709+0000 mgr.y (mgr.24428) 727 : cluster 0 pgmap v1232: 172 pgs: 172 active+clean; 4.4 MiB data, 770 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-03-04T16:08:46.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:46 smithi117 bash[28843]: audit 2024-03-04T16:08:45.392715+0000 mon.a (mon.0) 3585 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:08:46.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:46 smithi117 bash[28843]: audit 2024-03-04T16:08:45.395699+0000 mon.c (mon.2) 821 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:08:46.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:46 smithi117 bash[28843]: cluster 2024-03-04T16:08:45.444404+0000 mon.a (mon.0) 3586 : cluster 0 osdmap e759: 8 total, 8 up, 8 in 2024-03-04T16:08:46.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:46 smithi117 bash[28843]: audit 2024-03-04T16:08:45.452236+0000 mon.a (mon.0) 3587 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:08:47.682 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:08:47 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:08:47.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:47 smithi151 bash[23756]: audit 2024-03-04T16:08:46.438571+0000 mon.a (mon.0) 3588 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:08:47.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:47 smithi151 bash[23756]: cluster 2024-03-04T16:08:46.446899+0000 mon.a (mon.0) 3589 : cluster 0 osdmap e760: 8 total, 8 up, 8 in 2024-03-04T16:08:47.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:47 smithi151 bash[23756]: audit 2024-03-04T16:08:46.540489+0000 mon.a (mon.0) 3590 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:47.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:47 smithi151 bash[23756]: audit 2024-03-04T16:08:46.541048+0000 mon.a (mon.0) 3591 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-152"}]: dispatch 2024-03-04T16:08:47.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:47 smithi151 bash[23756]: cluster 2024-03-04T16:08:47.257812+0000 mgr.y (mgr.24428) 728 : cluster 0 pgmap v1235: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-04T16:08:47.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:47 smithi151 bash[23756]: audit 2024-03-04T16:08:47.282234+0000 mgr.y (mgr.24428) 729 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:08:47.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:47 smithi117 bash[28843]: audit 2024-03-04T16:08:46.438571+0000 mon.a (mon.0) 3588 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:08:47.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:47 smithi117 bash[28843]: cluster 2024-03-04T16:08:46.446899+0000 mon.a (mon.0) 3589 : cluster 0 osdmap e760: 8 total, 8 up, 8 in 2024-03-04T16:08:47.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:47 smithi117 bash[28843]: audit 2024-03-04T16:08:46.540489+0000 mon.a (mon.0) 3590 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:47.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:47 smithi117 bash[28843]: audit 2024-03-04T16:08:46.541048+0000 mon.a (mon.0) 3591 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-152"}]: dispatch 2024-03-04T16:08:47.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:47 smithi117 bash[28843]: cluster 2024-03-04T16:08:47.257812+0000 mgr.y (mgr.24428) 728 : cluster 0 pgmap v1235: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-04T16:08:47.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:47 smithi117 bash[28843]: audit 2024-03-04T16:08:47.282234+0000 mgr.y (mgr.24428) 729 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:08:47.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:47 smithi117 bash[21480]: audit 2024-03-04T16:08:46.438571+0000 mon.a (mon.0) 3588 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:08:47.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:47 smithi117 bash[21480]: cluster 2024-03-04T16:08:46.446899+0000 mon.a (mon.0) 3589 : cluster 0 osdmap e760: 8 total, 8 up, 8 in 2024-03-04T16:08:47.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:47 smithi117 bash[21480]: audit 2024-03-04T16:08:46.540489+0000 mon.a (mon.0) 3590 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:47.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:47 smithi117 bash[21480]: audit 2024-03-04T16:08:46.541048+0000 mon.a (mon.0) 3591 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-152"}]: dispatch 2024-03-04T16:08:47.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:47 smithi117 bash[21480]: cluster 2024-03-04T16:08:47.257812+0000 mgr.y (mgr.24428) 728 : cluster 0 pgmap v1235: 204 pgs: 204 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 3 op/s 2024-03-04T16:08:47.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:47 smithi117 bash[21480]: audit 2024-03-04T16:08:47.282234+0000 mgr.y (mgr.24428) 729 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:08:48.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:48 smithi117 bash[21480]: cluster 2024-03-04T16:08:47.460059+0000 mon.a (mon.0) 3592 : cluster 0 osdmap e761: 8 total, 8 up, 8 in 2024-03-04T16:08:48.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:48 smithi117 bash[28843]: cluster 2024-03-04T16:08:47.460059+0000 mon.a (mon.0) 3592 : cluster 0 osdmap e761: 8 total, 8 up, 8 in 2024-03-04T16:08:48.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:48 smithi151 bash[23756]: cluster 2024-03-04T16:08:47.460059+0000 mon.a (mon.0) 3592 : cluster 0 osdmap e761: 8 total, 8 up, 8 in 2024-03-04T16:08:49.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:49 smithi117 bash[21480]: cluster 2024-03-04T16:08:48.467711+0000 mon.a (mon.0) 3593 : cluster 0 osdmap e762: 8 total, 8 up, 8 in 2024-03-04T16:08:49.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:49 smithi117 bash[21480]: audit 2024-03-04T16:08:48.477980+0000 mon.a (mon.0) 3594 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:08:49.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:49 smithi117 bash[21480]: cluster 2024-03-04T16:08:49.258636+0000 mgr.y (mgr.24428) 730 : cluster 0 pgmap v1238: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T16:08:49.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:49 smithi117 bash[21480]: audit 2024-03-04T16:08:49.455312+0000 mon.a (mon.0) 3595 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:08:49.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:49 smithi117 bash[21480]: cluster 2024-03-04T16:08:49.464095+0000 mon.a (mon.0) 3596 : cluster 0 osdmap e763: 8 total, 8 up, 8 in 2024-03-04T16:08:49.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:49 smithi117 bash[21480]: audit 2024-03-04T16:08:49.468672+0000 mon.a (mon.0) 3597 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-111","var": "dedup_tier","val": "test-rados-api-smithi117-64061-154"}]: dispatch 2024-03-04T16:08:49.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:49 smithi117 bash[28843]: cluster 2024-03-04T16:08:48.467711+0000 mon.a (mon.0) 3593 : cluster 0 osdmap e762: 8 total, 8 up, 8 in 2024-03-04T16:08:49.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:49 smithi117 bash[28843]: audit 2024-03-04T16:08:48.477980+0000 mon.a (mon.0) 3594 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:08:49.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:49 smithi117 bash[28843]: cluster 2024-03-04T16:08:49.258636+0000 mgr.y (mgr.24428) 730 : cluster 0 pgmap v1238: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T16:08:49.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:49 smithi117 bash[28843]: audit 2024-03-04T16:08:49.455312+0000 mon.a (mon.0) 3595 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:08:49.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:49 smithi117 bash[28843]: cluster 2024-03-04T16:08:49.464095+0000 mon.a (mon.0) 3596 : cluster 0 osdmap e763: 8 total, 8 up, 8 in 2024-03-04T16:08:49.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:49 smithi117 bash[28843]: audit 2024-03-04T16:08:49.468672+0000 mon.a (mon.0) 3597 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-111","var": "dedup_tier","val": "test-rados-api-smithi117-64061-154"}]: dispatch 2024-03-04T16:08:49.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:49 smithi151 bash[23756]: cluster 2024-03-04T16:08:48.467711+0000 mon.a (mon.0) 3593 : cluster 0 osdmap e762: 8 total, 8 up, 8 in 2024-03-04T16:08:49.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:49 smithi151 bash[23756]: audit 2024-03-04T16:08:48.477980+0000 mon.a (mon.0) 3594 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-03-04T16:08:49.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:49 smithi151 bash[23756]: cluster 2024-03-04T16:08:49.258636+0000 mgr.y (mgr.24428) 730 : cluster 0 pgmap v1238: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-03-04T16:08:49.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:49 smithi151 bash[23756]: audit 2024-03-04T16:08:49.455312+0000 mon.a (mon.0) 3595 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi117-64061-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-03-04T16:08:49.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:49 smithi151 bash[23756]: cluster 2024-03-04T16:08:49.464095+0000 mon.a (mon.0) 3596 : cluster 0 osdmap e763: 8 total, 8 up, 8 in 2024-03-04T16:08:49.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:49 smithi151 bash[23756]: audit 2024-03-04T16:08:49.468672+0000 mon.a (mon.0) 3597 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi117-64061-111","var": "dedup_tier","val": "test-rados-api-smithi117-64061-154"}]: dispatch 2024-03-04T16:08:50.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:50 smithi117 bash[21480]: cluster 2024-03-04T16:08:49.479797+0000 mon.a (mon.0) 3598 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:08:50.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:50 smithi117 bash[21480]: audit 2024-03-04T16:08:49.536789+0000 mon.a (mon.0) 3599 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:50.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:50 smithi117 bash[21480]: audit 2024-03-04T16:08:49.537509+0000 mon.a (mon.0) 3600 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-154"}]: dispatch 2024-03-04T16:08:50.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:50 smithi117 bash[21480]: cluster 2024-03-04T16:08:50.467994+0000 mon.a (mon.0) 3601 : cluster 0 osdmap e764: 8 total, 8 up, 8 in 2024-03-04T16:08:50.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:50 smithi117 bash[28843]: cluster 2024-03-04T16:08:49.479797+0000 mon.a (mon.0) 3598 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:08:50.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:50 smithi117 bash[28843]: audit 2024-03-04T16:08:49.536789+0000 mon.a (mon.0) 3599 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:50.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:50 smithi117 bash[28843]: audit 2024-03-04T16:08:49.537509+0000 mon.a (mon.0) 3600 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-154"}]: dispatch 2024-03-04T16:08:50.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:50 smithi117 bash[28843]: cluster 2024-03-04T16:08:50.467994+0000 mon.a (mon.0) 3601 : cluster 0 osdmap e764: 8 total, 8 up, 8 in 2024-03-04T16:08:50.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:50 smithi151 bash[23756]: cluster 2024-03-04T16:08:49.479797+0000 mon.a (mon.0) 3598 : cluster 3 Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:08:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:50 smithi151 bash[23756]: audit 2024-03-04T16:08:49.536789+0000 mon.a (mon.0) 3599 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:50 smithi151 bash[23756]: audit 2024-03-04T16:08:49.537509+0000 mon.a (mon.0) 3600 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi117-64061-111", "tierpool": "test-rados-api-smithi117-64061-154"}]: dispatch 2024-03-04T16:08:50.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:50 smithi151 bash[23756]: cluster 2024-03-04T16:08:50.467994+0000 mon.a (mon.0) 3601 : cluster 0 osdmap e764: 8 total, 8 up, 8 in 2024-03-04T16:08:51.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:51 smithi117 bash[21480]: cluster 2024-03-04T16:08:51.259355+0000 mgr.y (mgr.24428) 731 : cluster 0 pgmap v1241: 172 pgs: 172 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:08:51.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:51 smithi117 bash[21480]: cluster 2024-03-04T16:08:51.471316+0000 mon.a (mon.0) 3602 : cluster 0 osdmap e765: 8 total, 8 up, 8 in 2024-03-04T16:08:51.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:51 smithi117 bash[21480]: audit 2024-03-04T16:08:51.474813+0000 mon.a (mon.0) 3603 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:51.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:51 smithi117 bash[28843]: cluster 2024-03-04T16:08:51.259355+0000 mgr.y (mgr.24428) 731 : cluster 0 pgmap v1241: 172 pgs: 172 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:08:51.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:51 smithi117 bash[28843]: cluster 2024-03-04T16:08:51.471316+0000 mon.a (mon.0) 3602 : cluster 0 osdmap e765: 8 total, 8 up, 8 in 2024-03-04T16:08:51.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:51 smithi117 bash[28843]: audit 2024-03-04T16:08:51.474813+0000 mon.a (mon.0) 3603 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:51.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:51 smithi151 bash[23756]: cluster 2024-03-04T16:08:51.259355+0000 mgr.y (mgr.24428) 731 : cluster 0 pgmap v1241: 172 pgs: 172 active+clean; 4.4 MiB data, 792 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:08:51.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:51 smithi151 bash[23756]: cluster 2024-03-04T16:08:51.471316+0000 mon.a (mon.0) 3602 : cluster 0 osdmap e765: 8 total, 8 up, 8 in 2024-03-04T16:08:51.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:51 smithi151 bash[23756]: audit 2024-03-04T16:08:51.474813+0000 mon.a (mon.0) 3603 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:53.490 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (7812 ms) 2024-03-04T16:08:53.491 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-03-04T16:08:53.491 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (12250 ms) 2024-03-04T16:08:53.491 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-03-04T16:08:53.491 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8074 ms) 2024-03-04T16:08:53.491 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-03-04T16:08:53.491 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13158 ms) 2024-03-04T16:08:53.491 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-03-04T16:08:53.491 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7521 ms) 2024-03-04T16:08:53.491 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-03-04T16:08:53.491 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8094 ms) 2024-03-04T16:08:53.491 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-03-04T16:08:53.491 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: hmm, no HitSet yet 2024-03-04T16:08:53.491 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: ok, hit_set contains 337:602f83fe:::foo:head 2024-03-04T16:08:53.492 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9034 ms) 2024-03-04T16:08:53.492 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-03-04T16:08:53.492 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709568449,0 2024-03-04T16:08:53.492 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: first is 1709568449 2024-03-04T16:08:53.492 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709568449,0 2024-03-04T16:08:53.492 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709568449,0 2024-03-04T16:08:53.492 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709568449,1709568451,1709568452,0 2024-03-04T16:08:53.492 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709568449,1709568451,1709568452,0 2024-03-04T16:08:53.492 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709568449,1709568451,1709568452,0 2024-03-04T16:08:53.492 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709568449,1709568451,1709568452,1709568454,1709568455,0 2024-03-04T16:08:53.492 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709568449,1709568451,1709568452,1709568454,1709568455,0 2024-03-04T16:08:53.492 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709568449,1709568451,1709568452,1709568454,1709568455,0 2024-03-04T16:08:53.492 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709568449,1709568451,1709568452,1709568454,1709568455,1709568457,1709568458,0 2024-03-04T16:08:53.492 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709568449,1709568451,1709568452,1709568454,1709568455,1709568457,1709568458,0 2024-03-04T16:08:53.493 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709568449,1709568451,1709568452,1709568454,1709568455,1709568457,1709568458,0 2024-03-04T16:08:53.493 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: got ls 1709568452,1709568454,1709568455,1709568457,1709568458,1709568460,1709568461,0 2024-03-04T16:08:53.493 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: first now 1709568452, trimmed 2024-03-04T16:08:53.493 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (20365 ms) 2024-03-04T16:08:53.493 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-03-04T16:08:53.493 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: foo0 2024-03-04T16:08:53.493 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-03-04T16:08:53.493 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14102 ms) 2024-03-04T16:08:53.493 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-03-04T16:08:53.493 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (17941 ms) 2024-03-04T16:08:53.493 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-03-04T16:08:53.493 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (23044 ms) 2024-03-04T16:08:53.493 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-03-04T16:08:53.493 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5023 ms) 2024-03-04T16:08:53.493 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-03-04T16:08:53.493 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3009 ms) 2024-03-04T16:08:53.494 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-03-04T16:08:53.494 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3019 ms) 2024-03-04T16:08:53.494 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-03-04T16:08:53.494 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3008 ms) 2024-03-04T16:08:53.494 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (230563 ms total) 2024-03-04T16:08:53.494 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: 2024-03-04T16:08:53.494 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-03-04T16:08:53.494 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (893517 ms total) 2024-03-04T16:08:53.494 INFO:tasks.workunit.client.0.smithi117.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-03-04T16:08:53.497 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.497 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=63942 2024-03-04T16:08:53.497 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 63942 2024-03-04T16:08:53.497 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.497 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64122 2024-03-04T16:08:53.497 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64122 2024-03-04T16:08:53.497 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.497 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64434 2024-03-04T16:08:53.497 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64434 2024-03-04T16:08:53.497 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.497 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64276 2024-03-04T16:08:53.497 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64276 2024-03-04T16:08:53.497 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.497 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64184 2024-03-04T16:08:53.498 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64184 2024-03-04T16:08:53.498 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.498 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=63971 2024-03-04T16:08:53.498 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 63971 2024-03-04T16:08:53.498 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.498 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64238 2024-03-04T16:08:53.498 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64238 2024-03-04T16:08:53.498 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.498 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64545 2024-03-04T16:08:53.498 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64545 2024-03-04T16:08:53.498 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.498 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64567 2024-03-04T16:08:53.498 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64567 2024-03-04T16:08:53.498 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.498 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=63976 2024-03-04T16:08:53.498 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 63976 2024-03-04T16:08:53.498 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.499 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64457 2024-03-04T16:08:53.499 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64457 2024-03-04T16:08:53.499 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.499 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64395 2024-03-04T16:08:53.499 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64395 2024-03-04T16:08:53.499 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.499 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=63926 2024-03-04T16:08:53.499 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 63926 2024-03-04T16:08:53.499 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.499 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=63958 2024-03-04T16:08:53.499 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 63958 2024-03-04T16:08:53.499 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.499 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64417 2024-03-04T16:08:53.499 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64417 2024-03-04T16:08:53.499 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.503 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=63918 2024-03-04T16:08:53.503 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 63918 2024-03-04T16:08:53.503 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.503 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=63934 2024-03-04T16:08:53.504 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 63934 2024-03-04T16:08:53.504 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.504 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64681 2024-03-04T16:08:53.504 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64681 2024-03-04T16:08:53.504 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.504 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64144 2024-03-04T16:08:53.504 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64144 2024-03-04T16:08:53.504 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.504 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64738 2024-03-04T16:08:53.504 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64738 2024-03-04T16:08:53.504 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.504 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64207 2024-03-04T16:08:53.504 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64207 2024-03-04T16:08:53.504 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.504 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64517 2024-03-04T16:08:53.504 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64517 2024-03-04T16:08:53.505 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.505 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64767 2024-03-04T16:08:53.505 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64767 2024-03-04T16:08:53.505 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.505 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64161 2024-03-04T16:08:53.505 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64161 2024-03-04T16:08:53.505 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.505 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64422 2024-03-04T16:08:53.505 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64422 2024-03-04T16:08:53.505 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.505 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64650 2024-03-04T16:08:53.505 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64650 2024-03-04T16:08:53.505 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.505 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64494 2024-03-04T16:08:53.505 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64494 2024-03-04T16:08:53.505 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.505 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64724 2024-03-04T16:08:53.506 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64724 2024-03-04T16:08:53.506 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.506 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64372 2024-03-04T16:08:53.506 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64372 2024-03-04T16:08:53.506 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.506 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64094 2024-03-04T16:08:53.506 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64094 2024-03-04T16:08:53.506 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.506 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64750 2024-03-04T16:08:53.506 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64750 2024-03-04T16:08:53.506 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.506 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64288 2024-03-04T16:08:53.506 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64288 2024-03-04T16:08:53.506 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.506 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=63910 2024-03-04T16:08:53.506 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 63910 2024-03-04T16:08:53.506 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.506 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=63950 2024-03-04T16:08:53.507 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 63950 2024-03-04T16:08:53.507 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.507 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64267 2024-03-04T16:08:53.507 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64267 2024-03-04T16:08:53.507 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.507 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=63966 2024-03-04T16:08:53.507 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 63966 2024-03-04T16:08:53.507 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.507 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64296 2024-03-04T16:08:53.507 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64296 2024-03-04T16:08:53.507 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.507 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64338 2024-03-04T16:08:53.507 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64338 2024-03-04T16:08:53.507 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.507 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64611 2024-03-04T16:08:53.508 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64611 2024-03-04T16:08:53.508 INFO:tasks.workunit.client.0.smithi117.stderr:+ for t in "${!pids[@]}" 2024-03-04T16:08:53.508 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=64793 2024-03-04T16:08:53.508 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 64793 2024-03-04T16:08:53.508 INFO:tasks.workunit.client.0.smithi117.stderr:+ exit 0 2024-03-04T16:08:53.508 INFO:tasks.workunit.client.0.smithi117.stderr:+ cleanup 2024-03-04T16:08:53.508 INFO:tasks.workunit.client.0.smithi117.stderr:+ pkill -P 63904 2024-03-04T16:08:53.567 INFO:tasks.workunit.client.0.smithi117.stderr:+ true 2024-03-04T16:08:53.568 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T16:08:53.569 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T16:08:53.583 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-03-04T16:08:53.584 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-03-04T16:08:53.584 DEBUG:teuthology.orchestra.run.smithi117:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=9ca0b5de35ed230e84387abdd8e619ed8e4f3282 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-03-04T16:08:53.634 INFO:tasks.workunit.client.0.smithi117.stderr:+ uuidgen 2024-03-04T16:08:53.635 INFO:tasks.workunit.client.0.smithi117.stderr:+ p=736f5599-4143-4934-af9d-08c0b0a4b1a3 2024-03-04T16:08:53.635 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph osd pool create 736f5599-4143-4934-af9d-08c0b0a4b1a3 12 2024-03-04T16:08:53.762 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf21c46640 1 -- 172.21.15.117:0/2477060361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf1c106610 msgr2=0x7fdf1c10e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:08:53.762 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf21c46640 1 --2- 172.21.15.117:0/2477060361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf1c106610 0x7fdf1c10e190 secure :-1 s=READY pgs=3095 cs=0 l=1 rev1=1 crypto rx=0x7fdf14009cb0 tx=0x7fdf1401cca0 comp rx=0 tx=0).stop 2024-03-04T16:08:53.762 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf21c46640 1 -- 172.21.15.117:0/2477060361 shutdown_connections 2024-03-04T16:08:53.762 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf21c46640 1 --2- 172.21.15.117:0/2477060361 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf1c106610 0x7fdf1c10e190 unknown :-1 s=CLOSED pgs=3095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:53.762 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf21c46640 1 --2- 172.21.15.117:0/2477060361 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf1c101e90 0x7fdf1c105ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:53.762 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf21c46640 1 --2- 172.21.15.117:0/2477060361 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf1c1014e0 0x7fdf1c1018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:53.762 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf21c46640 1 -- 172.21.15.117:0/2477060361 >> 172.21.15.117:0/2477060361 conn(0x7fdf1c0fcdd0 msgr2=0x7fdf1c0ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:08:53.762 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf21c46640 1 -- 172.21.15.117:0/2477060361 shutdown_connections 2024-03-04T16:08:53.762 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf21c46640 1 -- 172.21.15.117:0/2477060361 wait complete. 2024-03-04T16:08:53.764 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf21c46640 1 Processor -- start 2024-03-04T16:08:53.764 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf21c46640 1 -- start start 2024-03-04T16:08:53.764 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf21c46640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf1c1014e0 0x7fdf1c199e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:08:53.764 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf21c46640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf1c101e90 0x7fdf1c19a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:08:53.765 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf21c46640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf1c106610 0x7fdf1c19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:08:53.765 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf21c46640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c1138d0 con 0x7fdf1c101e90 2024-03-04T16:08:53.765 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf21c46640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c113750 con 0x7fdf1c106610 2024-03-04T16:08:53.765 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf21c46640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf1c113a50 con 0x7fdf1c1014e0 2024-03-04T16:08:53.765 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf1b7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf1c1014e0 0x7fdf1c199e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:08:53.765 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf1affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf1c101e90 0x7fdf1c19a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:08:53.765 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf1affd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf1c101e90 0x7fdf1c19a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56492/0 (socket says 172.21.15.117:56492) 2024-03-04T16:08:53.765 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf1b7fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf1c1014e0 0x7fdf1c199e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:41096/0 (socket says 172.21.15.117:41096) 2024-03-04T16:08:53.765 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf1affd640 1 -- 172.21.15.117:0/4038800558 learned_addr learned my addr 172.21.15.117:0/4038800558 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T16:08:53.765 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf1b7fe640 1 -- 172.21.15.117:0/4038800558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf1c106610 msgr2=0x7fdf1c19e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:08:53.765 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf1b7fe640 1 --2- 172.21.15.117:0/4038800558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf1c106610 0x7fdf1c19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:53.765 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf1b7fe640 1 -- 172.21.15.117:0/4038800558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf1c101e90 msgr2=0x7fdf1c19a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:08:53.765 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf1b7fe640 1 --2- 172.21.15.117:0/4038800558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf1c101e90 0x7fdf1c19a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:53.765 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf1b7fe640 1 -- 172.21.15.117:0/4038800558 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf1c19ee40 con 0x7fdf1c1014e0 2024-03-04T16:08:53.765 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf1b7fe640 1 --2- 172.21.15.117:0/4038800558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf1c1014e0 0x7fdf1c199e90 secure :-1 s=READY pgs=3096 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00e9f0 tx=0x7fdf0c00edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:08:53.765 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf18ff9640 1 -- 172.21.15.117:0/4038800558 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c00ccc0 con 0x7fdf1c1014e0 2024-03-04T16:08:53.765 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf21c46640 1 -- 172.21.15.117:0/4038800558 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf1c19f130 con 0x7fdf1c1014e0 2024-03-04T16:08:53.766 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf18ff9640 1 -- 172.21.15.117:0/4038800558 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fdf0c0044a0 con 0x7fdf1c1014e0 2024-03-04T16:08:53.766 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf18ff9640 1 -- 172.21.15.117:0/4038800558 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0c010430 con 0x7fdf1c1014e0 2024-03-04T16:08:53.766 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf21c46640 1 -- 172.21.15.117:0/4038800558 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf1c1a6a10 con 0x7fdf1c1014e0 2024-03-04T16:08:53.766 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf21c46640 1 -- 172.21.15.117:0/4038800558 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf1c068630 con 0x7fdf1c1014e0 2024-03-04T16:08:53.766 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.763+0000 7fdf18ff9640 1 -- 172.21.15.117:0/4038800558 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7fdf0c003e90 con 0x7fdf1c1014e0 2024-03-04T16:08:53.766 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.767+0000 7fdf18ff9640 1 --2- 172.21.15.117:0/4038800558 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fdf0406d700 0x7fdf0406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:08:53.767 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.767+0000 7fdf18ff9640 1 -- 172.21.15.117:0/4038800558 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(767..767 src has 1..767) v4 ==== 8680+0+0 (secure 0 0 0) 0x7fdf0c08e270 con 0x7fdf1c1014e0 2024-03-04T16:08:53.767 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.767+0000 7fdf1affd640 1 --2- 172.21.15.117:0/4038800558 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fdf0406d700 0x7fdf0406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:08:53.767 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.767+0000 7fdf1affd640 1 --2- 172.21.15.117:0/4038800558 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fdf0406d700 0x7fdf0406fbc0 secure :-1 s=READY pgs=3214 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c105f40 tx=0x7fdf1000a300 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:08:53.770 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.771+0000 7fdf18ff9640 1 -- 172.21.15.117:0/4038800558 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fdf0c05b600 con 0x7fdf1c1014e0 2024-03-04T16:08:53.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:53 smithi117 bash[28843]: audit 2024-03-04T16:08:52.463541+0000 mon.a (mon.0) 3604 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:08:53.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:53 smithi117 bash[28843]: cluster 2024-03-04T16:08:52.471406+0000 mon.a (mon.0) 3605 : cluster 0 osdmap e766: 8 total, 8 up, 8 in 2024-03-04T16:08:53.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:53 smithi117 bash[28843]: audit 2024-03-04T16:08:52.472668+0000 mon.a (mon.0) 3606 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:53.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:53 smithi117 bash[28843]: cluster 2024-03-04T16:08:53.260143+0000 mgr.y (mgr.24428) 732 : cluster 0 pgmap v1244: 164 pgs: 164 active+clean; 582 KiB data, 781 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:08:53.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:53 smithi117 bash[21480]: audit 2024-03-04T16:08:52.463541+0000 mon.a (mon.0) 3604 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:08:53.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:53 smithi117 bash[21480]: cluster 2024-03-04T16:08:52.471406+0000 mon.a (mon.0) 3605 : cluster 0 osdmap e766: 8 total, 8 up, 8 in 2024-03-04T16:08:53.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:53 smithi117 bash[21480]: audit 2024-03-04T16:08:52.472668+0000 mon.a (mon.0) 3606 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:53.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:53 smithi117 bash[21480]: cluster 2024-03-04T16:08:53.260143+0000 mgr.y (mgr.24428) 732 : cluster 0 pgmap v1244: 164 pgs: 164 active+clean; 582 KiB data, 781 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:08:53.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:53 smithi151 bash[23756]: audit 2024-03-04T16:08:52.463541+0000 mon.a (mon.0) 3604 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:08:53.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:53 smithi151 bash[23756]: cluster 2024-03-04T16:08:52.471406+0000 mon.a (mon.0) 3605 : cluster 0 osdmap e766: 8 total, 8 up, 8 in 2024-03-04T16:08:53.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:53 smithi151 bash[23756]: audit 2024-03-04T16:08:52.472668+0000 mon.a (mon.0) 3606 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-111"}]: dispatch 2024-03-04T16:08:53.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:53 smithi151 bash[23756]: cluster 2024-03-04T16:08:53.260143+0000 mgr.y (mgr.24428) 732 : cluster 0 pgmap v1244: 164 pgs: 164 active+clean; 582 KiB data, 781 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:08:53.946 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:53.947+0000 7fdf21c46640 1 -- 172.21.15.117:0/4038800558 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pg_num": 12} v 0) v1 -- 0x7fdf1c1a6ca0 con 0x7fdf1c1014e0 2024-03-04T16:08:54.494 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.495+0000 7fdf18ff9640 1 -- 172.21.15.117:0/4038800558 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pg_num": 12}]=0 pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' created v768) v1 ==== 176+0+0 (secure 0 0 0) 0x7fdf0c05f050 con 0x7fdf1c1014e0 2024-03-04T16:08:54.604 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.603+0000 7fdf21c46640 1 -- 172.21.15.117:0/4038800558 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pg_num": 12} v 0) v1 -- 0x7fdf1c1a6e70 con 0x7fdf1c1014e0 2024-03-04T16:08:54.606 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.607+0000 7fdf18ff9640 1 -- 172.21.15.117:0/4038800558 <== mon.2 v2:172.21.15.117:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pg_num": 12}]=0 pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' already exists v768) v1 ==== 183+0+0 (secure 0 0 0) 0x7fdf0c063610 con 0x7fdf1c1014e0 2024-03-04T16:08:54.606 INFO:tasks.workunit.client.0.smithi117.stderr:pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' already exists 2024-03-04T16:08:54.608 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.607+0000 7fdf21c46640 1 -- 172.21.15.117:0/4038800558 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fdf0406d700 msgr2=0x7fdf0406fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:08:54.608 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.607+0000 7fdf21c46640 1 --2- 172.21.15.117:0/4038800558 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fdf0406d700 0x7fdf0406fbc0 secure :-1 s=READY pgs=3214 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c105f40 tx=0x7fdf1000a300 comp rx=0 tx=0).stop 2024-03-04T16:08:54.609 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.607+0000 7fdf21c46640 1 -- 172.21.15.117:0/4038800558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf1c1014e0 msgr2=0x7fdf1c199e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:08:54.609 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.607+0000 7fdf21c46640 1 --2- 172.21.15.117:0/4038800558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf1c1014e0 0x7fdf1c199e90 secure :-1 s=READY pgs=3096 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c00e9f0 tx=0x7fdf0c00edf0 comp rx=0 tx=0).stop 2024-03-04T16:08:54.609 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.607+0000 7fdf21c46640 1 -- 172.21.15.117:0/4038800558 shutdown_connections 2024-03-04T16:08:54.609 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.607+0000 7fdf21c46640 1 --2- 172.21.15.117:0/4038800558 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fdf0406d700 0x7fdf0406fbc0 unknown :-1 s=CLOSED pgs=3214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:54.609 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.607+0000 7fdf21c46640 1 --2- 172.21.15.117:0/4038800558 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fdf1c101e90 0x7fdf1c19a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:54.609 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.607+0000 7fdf21c46640 1 --2- 172.21.15.117:0/4038800558 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fdf1c106610 0x7fdf1c19e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:54.609 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.607+0000 7fdf21c46640 1 --2- 172.21.15.117:0/4038800558 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fdf1c1014e0 0x7fdf1c199e90 unknown :-1 s=CLOSED pgs=3096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:54.609 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.607+0000 7fdf21c46640 1 -- 172.21.15.117:0/4038800558 >> 172.21.15.117:0/4038800558 conn(0x7fdf1c0fcdd0 msgr2=0x7fdf1c0fd2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:08:54.609 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.607+0000 7fdf21c46640 1 -- 172.21.15.117:0/4038800558 shutdown_connections 2024-03-04T16:08:54.609 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.607+0000 7fdf21c46640 1 -- 172.21.15.117:0/4038800558 wait complete. 2024-03-04T16:08:54.623 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph osd pool set-quota 736f5599-4143-4934-af9d-08c0b0a4b1a3 max_objects 10 2024-03-04T16:08:54.749 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e7421640 1 -- 172.21.15.117:0/1477170848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8e0103e50 msgr2=0x7fc8e0110310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:08:54.749 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e7421640 1 --2- 172.21.15.117:0/1477170848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8e0103e50 0x7fc8e0110310 secure :-1 s=READY pgs=3097 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009ce0 tx=0x7fc8d001cc80 comp rx=0 tx=0).stop 2024-03-04T16:08:54.749 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e7421640 1 -- 172.21.15.117:0/1477170848 shutdown_connections 2024-03-04T16:08:54.749 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e7421640 1 --2- 172.21.15.117:0/1477170848 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8e0103e50 0x7fc8e0110310 unknown :-1 s=CLOSED pgs=3097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:54.749 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e7421640 1 --2- 172.21.15.117:0/1477170848 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8e0103490 0x7fc8e0103910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:54.749 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e7421640 1 --2- 172.21.15.117:0/1477170848 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8e0109540 0x7fc8e0109920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:54.749 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e7421640 1 -- 172.21.15.117:0/1477170848 >> 172.21.15.117:0/1477170848 conn(0x7fc8e00fef20 msgr2=0x7fc8e0101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:08:54.750 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e7421640 1 -- 172.21.15.117:0/1477170848 shutdown_connections 2024-03-04T16:08:54.750 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e7421640 1 -- 172.21.15.117:0/1477170848 wait complete. 2024-03-04T16:08:54.750 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e7421640 1 Processor -- start 2024-03-04T16:08:54.750 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e7421640 1 -- start start 2024-03-04T16:08:54.750 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e7421640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8e0103490 0x7fc8e01a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:08:54.750 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e7421640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8e0103e50 0x7fc8e01a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:08:54.750 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e7421640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8e0109540 0x7fc8e019b870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:08:54.750 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e7421640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e01154e0 con 0x7fc8e0109540 2024-03-04T16:08:54.750 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e7421640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e0115360 con 0x7fc8e0103e50 2024-03-04T16:08:54.750 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e7421640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8e0115660 con 0x7fc8e0103490 2024-03-04T16:08:54.750 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e59be640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8e0109540 0x7fc8e019b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:08:54.750 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e51bd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8e0103490 0x7fc8e01a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:08:54.750 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e59be640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8e0109540 0x7fc8e019b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:56508/0 (socket says 172.21.15.117:56508) 2024-03-04T16:08:54.750 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e59be640 1 -- 172.21.15.117:0/13998417 learned_addr learned my addr 172.21.15.117:0/13998417 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T16:08:54.750 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e51bd640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8e0103490 0x7fc8e01a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:41112/0 (socket says 172.21.15.117:41112) 2024-03-04T16:08:54.751 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e49bc640 1 --2- 172.21.15.117:0/13998417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8e0103e50 0x7fc8e01a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:08:54.751 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e59be640 1 -- 172.21.15.117:0/13998417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8e0103490 msgr2=0x7fc8e01a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:08:54.751 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e59be640 1 --2- 172.21.15.117:0/13998417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8e0103490 0x7fc8e01a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:54.751 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e59be640 1 -- 172.21.15.117:0/13998417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8e0103e50 msgr2=0x7fc8e01a1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:08:54.751 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e59be640 1 --2- 172.21.15.117:0/13998417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8e0103e50 0x7fc8e01a1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:54.751 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e59be640 1 -- 172.21.15.117:0/13998417 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e019bfa0 con 0x7fc8e0109540 2024-03-04T16:08:54.751 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e59be640 1 --2- 172.21.15.117:0/13998417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8e0109540 0x7fc8e019b870 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009bd0 tx=0x7fc8d00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:08:54.751 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8de7fc640 1 -- 172.21.15.117:0/13998417 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d00b2040 con 0x7fc8e0109540 2024-03-04T16:08:54.751 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e7421640 1 -- 172.21.15.117:0/13998417 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8e019c230 con 0x7fc8e0109540 2024-03-04T16:08:54.751 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8de7fc640 1 -- 172.21.15.117:0/13998417 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fc8d0004780 con 0x7fc8e0109540 2024-03-04T16:08:54.751 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8de7fc640 1 -- 172.21.15.117:0/13998417 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d0005690 con 0x7fc8e0109540 2024-03-04T16:08:54.751 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.747+0000 7fc8e7421640 1 -- 172.21.15.117:0/13998417 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8e01a8580 con 0x7fc8e0109540 2024-03-04T16:08:54.752 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.751+0000 7fc8e7421640 1 -- 172.21.15.117:0/13998417 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8e0068630 con 0x7fc8e0109540 2024-03-04T16:08:54.752 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.751+0000 7fc8de7fc640 1 -- 172.21.15.117:0/13998417 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7fc8d0004a50 con 0x7fc8e0109540 2024-03-04T16:08:54.752 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.751+0000 7fc8de7fc640 1 --2- 172.21.15.117:0/13998417 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fc8c806d700 0x7fc8c806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:08:54.752 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.751+0000 7fc8de7fc640 1 -- 172.21.15.117:0/13998417 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(768..768 src has 1..768) v4 ==== 9055+0+0 (secure 0 0 0) 0x7fc8d01287d0 con 0x7fc8e0109540 2024-03-04T16:08:54.752 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.751+0000 7fc8e51bd640 1 --2- 172.21.15.117:0/13998417 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fc8c806d700 0x7fc8c806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:08:54.753 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.751+0000 7fc8e51bd640 1 --2- 172.21.15.117:0/13998417 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fc8c806d700 0x7fc8c806fbc0 secure :-1 s=READY pgs=3215 cs=0 l=1 rev1=1 crypto rx=0x7fc8e019c9b0 tx=0x7fc8d80091e0 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:08:54.756 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.755+0000 7fc8de7fc640 1 -- 172.21.15.117:0/13998417 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fc8d0016990 con 0x7fc8e0109540 2024-03-04T16:08:54.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:54 smithi117 bash[28843]: audit 2024-03-04T16:08:53.472641+0000 mon.a (mon.0) 3607 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:08:54.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:54 smithi117 bash[28843]: cluster 2024-03-04T16:08:53.482441+0000 mon.a (mon.0) 3608 : cluster 0 osdmap e767: 8 total, 8 up, 8 in 2024-03-04T16:08:54.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:54 smithi117 bash[28843]: audit 2024-03-04T16:08:53.950932+0000 mon.c (mon.2) 822 : audit 1 from='client.? 172.21.15.117:0/4038800558' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pg_num": 12}]: dispatch 2024-03-04T16:08:54.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:54 smithi117 bash[28843]: audit 2024-03-04T16:08:53.951466+0000 mon.a (mon.0) 3609 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pg_num": 12}]: dispatch 2024-03-04T16:08:54.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:54 smithi117 bash[21480]: audit 2024-03-04T16:08:53.472641+0000 mon.a (mon.0) 3607 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:08:54.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:54 smithi117 bash[21480]: cluster 2024-03-04T16:08:53.482441+0000 mon.a (mon.0) 3608 : cluster 0 osdmap e767: 8 total, 8 up, 8 in 2024-03-04T16:08:54.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:54 smithi117 bash[21480]: audit 2024-03-04T16:08:53.950932+0000 mon.c (mon.2) 822 : audit 1 from='client.? 172.21.15.117:0/4038800558' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pg_num": 12}]: dispatch 2024-03-04T16:08:54.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:54 smithi117 bash[21480]: audit 2024-03-04T16:08:53.951466+0000 mon.a (mon.0) 3609 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pg_num": 12}]: dispatch 2024-03-04T16:08:54.931 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:54.931+0000 7fc8e7421640 1 -- 172.21.15.117:0/13998417 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fc8e01a8750 con 0x7fc8e0109540 2024-03-04T16:08:54.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:54 smithi151 bash[23756]: audit 2024-03-04T16:08:53.472641+0000 mon.a (mon.0) 3607 : audit 1 from='client.? 172.21.15.117:0/879788534' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi117-64061-111"}]': finished 2024-03-04T16:08:54.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:54 smithi151 bash[23756]: cluster 2024-03-04T16:08:53.482441+0000 mon.a (mon.0) 3608 : cluster 0 osdmap e767: 8 total, 8 up, 8 in 2024-03-04T16:08:54.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:54 smithi151 bash[23756]: audit 2024-03-04T16:08:53.950932+0000 mon.c (mon.2) 822 : audit 1 from='client.? 172.21.15.117:0/4038800558' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pg_num": 12}]: dispatch 2024-03-04T16:08:54.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:54 smithi151 bash[23756]: audit 2024-03-04T16:08:53.951466+0000 mon.a (mon.0) 3609 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pg_num": 12}]: dispatch 2024-03-04T16:08:55.495 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:55.495+0000 7fc8de7fc640 1 -- 172.21.15.117:0/13998417 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 v769) v1 ==== 223+0+0 (secure 0 0 0) 0x7fc8d00f59f0 con 0x7fc8e0109540 2024-03-04T16:08:55.612 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:55.611+0000 7fc8e7421640 1 -- 172.21.15.117:0/13998417 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fc8e0061980 con 0x7fc8e0109540 2024-03-04T16:08:55.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:55 smithi117 bash[21480]: audit 2024-03-04T16:08:54.489624+0000 mon.a (mon.0) 3610 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pg_num": 12}]': finished 2024-03-04T16:08:55.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:55 smithi117 bash[21480]: cluster 2024-03-04T16:08:54.498164+0000 mon.a (mon.0) 3611 : cluster 0 osdmap e768: 8 total, 8 up, 8 in 2024-03-04T16:08:55.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:55 smithi117 bash[21480]: audit 2024-03-04T16:08:54.608751+0000 mon.c (mon.2) 823 : audit 1 from='client.? 172.21.15.117:0/4038800558' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pg_num": 12}]: dispatch 2024-03-04T16:08:55.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:55 smithi117 bash[21480]: audit 2024-03-04T16:08:54.609233+0000 mon.a (mon.0) 3612 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pg_num": 12}]: dispatch 2024-03-04T16:08:55.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:55 smithi117 bash[21480]: audit 2024-03-04T16:08:54.814076+0000 mon.c (mon.2) 824 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:08:55.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:55 smithi117 bash[21480]: cluster 2024-03-04T16:08:54.900505+0000 mon.a (mon.0) 3613 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:08:55.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:55 smithi117 bash[21480]: audit 2024-03-04T16:08:54.935080+0000 mon.a (mon.0) 3614 : audit 1 from='client.? 172.21.15.117:0/13998417' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "10"}]: dispatch 2024-03-04T16:08:55.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:55 smithi117 bash[21480]: cluster 2024-03-04T16:08:55.260766+0000 mgr.y (mgr.24428) 733 : cluster 0 pgmap v1247: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 781 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:08:55.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:55 smithi117 bash[28843]: audit 2024-03-04T16:08:54.489624+0000 mon.a (mon.0) 3610 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pg_num": 12}]': finished 2024-03-04T16:08:55.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:55 smithi117 bash[28843]: cluster 2024-03-04T16:08:54.498164+0000 mon.a (mon.0) 3611 : cluster 0 osdmap e768: 8 total, 8 up, 8 in 2024-03-04T16:08:55.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:55 smithi117 bash[28843]: audit 2024-03-04T16:08:54.608751+0000 mon.c (mon.2) 823 : audit 1 from='client.? 172.21.15.117:0/4038800558' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pg_num": 12}]: dispatch 2024-03-04T16:08:55.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:55 smithi117 bash[28843]: audit 2024-03-04T16:08:54.609233+0000 mon.a (mon.0) 3612 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pg_num": 12}]: dispatch 2024-03-04T16:08:55.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:55 smithi117 bash[28843]: audit 2024-03-04T16:08:54.814076+0000 mon.c (mon.2) 824 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:08:55.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:55 smithi117 bash[28843]: cluster 2024-03-04T16:08:54.900505+0000 mon.a (mon.0) 3613 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:08:55.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:55 smithi117 bash[28843]: audit 2024-03-04T16:08:54.935080+0000 mon.a (mon.0) 3614 : audit 1 from='client.? 172.21.15.117:0/13998417' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "10"}]: dispatch 2024-03-04T16:08:55.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:55 smithi117 bash[28843]: cluster 2024-03-04T16:08:55.260766+0000 mgr.y (mgr.24428) 733 : cluster 0 pgmap v1247: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 781 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:08:55.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:55 smithi151 bash[23756]: audit 2024-03-04T16:08:54.489624+0000 mon.a (mon.0) 3610 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pg_num": 12}]': finished 2024-03-04T16:08:55.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:55 smithi151 bash[23756]: cluster 2024-03-04T16:08:54.498164+0000 mon.a (mon.0) 3611 : cluster 0 osdmap e768: 8 total, 8 up, 8 in 2024-03-04T16:08:55.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:55 smithi151 bash[23756]: audit 2024-03-04T16:08:54.608751+0000 mon.c (mon.2) 823 : audit 1 from='client.? 172.21.15.117:0/4038800558' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pg_num": 12}]: dispatch 2024-03-04T16:08:55.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:55 smithi151 bash[23756]: audit 2024-03-04T16:08:54.609233+0000 mon.a (mon.0) 3612 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pg_num": 12}]: dispatch 2024-03-04T16:08:55.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:55 smithi151 bash[23756]: audit 2024-03-04T16:08:54.814076+0000 mon.c (mon.2) 824 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:08:55.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:55 smithi151 bash[23756]: cluster 2024-03-04T16:08:54.900505+0000 mon.a (mon.0) 3613 : cluster 3 Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-03-04T16:08:55.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:55 smithi151 bash[23756]: audit 2024-03-04T16:08:54.935080+0000 mon.a (mon.0) 3614 : audit 1 from='client.? 172.21.15.117:0/13998417' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "10"}]: dispatch 2024-03-04T16:08:55.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:55 smithi151 bash[23756]: cluster 2024-03-04T16:08:55.260766+0000 mgr.y (mgr.24428) 733 : cluster 0 pgmap v1247: 176 pgs: 12 unknown, 164 active+clean; 582 KiB data, 781 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:08:56.498 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.499+0000 7fc8de7fc640 1 -- 172.21.15.117:0/13998417 <== mon.0 v2:172.21.15.117:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 v770) v1 ==== 223+0+0 (secure 0 0 0) 0x7fc8d00f9440 con 0x7fc8e0109540 2024-03-04T16:08:56.498 INFO:tasks.workunit.client.0.smithi117.stderr:set-quota max_objects = 10 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 2024-03-04T16:08:56.504 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.503+0000 7fc8e7421640 1 -- 172.21.15.117:0/13998417 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fc8c806d700 msgr2=0x7fc8c806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:08:56.504 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.503+0000 7fc8e7421640 1 --2- 172.21.15.117:0/13998417 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fc8c806d700 0x7fc8c806fbc0 secure :-1 s=READY pgs=3215 cs=0 l=1 rev1=1 crypto rx=0x7fc8e019c9b0 tx=0x7fc8d80091e0 comp rx=0 tx=0).stop 2024-03-04T16:08:56.505 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.503+0000 7fc8e7421640 1 -- 172.21.15.117:0/13998417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8e0109540 msgr2=0x7fc8e019b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:08:56.505 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.503+0000 7fc8e7421640 1 --2- 172.21.15.117:0/13998417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8e0109540 0x7fc8e019b870 secure :-1 s=READY pgs=2479 cs=0 l=1 rev1=1 crypto rx=0x7fc8d0009bd0 tx=0x7fc8d00045e0 comp rx=0 tx=0).stop 2024-03-04T16:08:56.505 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.503+0000 7fc8e7421640 1 -- 172.21.15.117:0/13998417 shutdown_connections 2024-03-04T16:08:56.505 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.503+0000 7fc8e7421640 1 --2- 172.21.15.117:0/13998417 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fc8c806d700 0x7fc8c806fbc0 unknown :-1 s=CLOSED pgs=3215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:56.505 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.503+0000 7fc8e7421640 1 --2- 172.21.15.117:0/13998417 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fc8e0109540 0x7fc8e019b870 unknown :-1 s=CLOSED pgs=2479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:56.505 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.503+0000 7fc8e7421640 1 --2- 172.21.15.117:0/13998417 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fc8e0103e50 0x7fc8e01a1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:56.505 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.503+0000 7fc8e7421640 1 --2- 172.21.15.117:0/13998417 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fc8e0103490 0x7fc8e01a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:56.505 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.503+0000 7fc8e7421640 1 -- 172.21.15.117:0/13998417 >> 172.21.15.117:0/13998417 conn(0x7fc8e00fef20 msgr2=0x7fc8e0107040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:08:56.505 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.503+0000 7fc8e7421640 1 -- 172.21.15.117:0/13998417 shutdown_connections 2024-03-04T16:08:56.505 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.503+0000 7fc8e7421640 1 -- 172.21.15.117:0/13998417 wait complete. 2024-03-04T16:08:56.513 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:08:56 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:08:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:08:56.527 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph osd pool application enable 736f5599-4143-4934-af9d-08c0b0a4b1a3 rados 2024-03-04T16:08:56.654 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237229e640 1 -- 172.21.15.117:0/3247384469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f236c102430 msgr2=0x7f236c102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:08:56.654 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237229e640 1 --2- 172.21.15.117:0/3247384469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f236c102430 0x7f236c102810 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7f235c009930 tx=0x7f235c01c870 comp rx=0 tx=0).stop 2024-03-04T16:08:56.654 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237229e640 1 -- 172.21.15.117:0/3247384469 shutdown_connections 2024-03-04T16:08:56.654 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237229e640 1 --2- 172.21.15.117:0/3247384469 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f236c10b860 0x7f236c10dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:56.654 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237229e640 1 --2- 172.21.15.117:0/3247384469 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f236c102d50 0x7f236c10b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:56.654 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237229e640 1 --2- 172.21.15.117:0/3247384469 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f236c102430 0x7f236c102810 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:56.654 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237229e640 1 -- 172.21.15.117:0/3247384469 >> 172.21.15.117:0/3247384469 conn(0x7f236c0fbc60 msgr2=0x7f236c0fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:08:56.654 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237229e640 1 -- 172.21.15.117:0/3247384469 shutdown_connections 2024-03-04T16:08:56.654 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237229e640 1 -- 172.21.15.117:0/3247384469 wait complete. 2024-03-04T16:08:56.655 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237229e640 1 Processor -- start 2024-03-04T16:08:56.655 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237229e640 1 -- start start 2024-03-04T16:08:56.655 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237229e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f236c102430 0x7f236c199ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:08:56.655 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237229e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f236c102d50 0x7f236c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:08:56.655 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237229e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f236c10b860 0x7f236c19e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:08:56.655 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237229e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c113880 con 0x7f236c102d50 2024-03-04T16:08:56.655 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237229e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f236c113700 con 0x7f236c102430 2024-03-04T16:08:56.655 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237229e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f236c113a00 con 0x7f236c10b860 2024-03-04T16:08:56.655 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237083b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f236c10b860 0x7f236c19e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:08:56.655 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f236b7fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f236c102d50 0x7f236c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:08:56.655 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237083b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f236c10b860 0x7f236c19e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:41142/0 (socket says 172.21.15.117:41142) 2024-03-04T16:08:56.655 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237083b640 1 -- 172.21.15.117:0/1504644375 learned_addr learned my addr 172.21.15.117:0/1504644375 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T16:08:56.655 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237083b640 1 -- 172.21.15.117:0/1504644375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f236c102430 msgr2=0x7f236c199ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:08:56.655 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237083b640 1 --2- 172.21.15.117:0/1504644375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f236c102430 0x7f236c199ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:56.655 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237083b640 1 -- 172.21.15.117:0/1504644375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f236c102d50 msgr2=0x7f236c19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:08:56.655 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237083b640 1 --2- 172.21.15.117:0/1504644375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f236c102d50 0x7f236c19a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:56.655 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237083b640 1 -- 172.21.15.117:0/1504644375 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f236c19ee80 con 0x7f236c10b860 2024-03-04T16:08:56.655 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f237083b640 1 --2- 172.21.15.117:0/1504644375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f236c10b860 0x7f236c19e7a0 secure :-1 s=READY pgs=3098 cs=0 l=1 rev1=1 crypto rx=0x7f236400efd0 tx=0x7f236400c6a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:08:56.656 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.651+0000 7f23697fa640 1 -- 172.21.15.117:0/1504644375 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2364003b80 con 0x7f236c10b860 2024-03-04T16:08:56.656 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.655+0000 7f237229e640 1 -- 172.21.15.117:0/1504644375 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f236c19f170 con 0x7f236c10b860 2024-03-04T16:08:56.656 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.655+0000 7f23697fa640 1 -- 172.21.15.117:0/1504644375 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f2364003d20 con 0x7f236c10b860 2024-03-04T16:08:56.656 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.655+0000 7f23697fa640 1 -- 172.21.15.117:0/1504644375 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23640107e0 con 0x7f236c10b860 2024-03-04T16:08:56.656 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.655+0000 7f237229e640 1 -- 172.21.15.117:0/1504644375 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f236c1a6a50 con 0x7f236c10b860 2024-03-04T16:08:56.656 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.655+0000 7f237229e640 1 -- 172.21.15.117:0/1504644375 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236c068630 con 0x7f236c10b860 2024-03-04T16:08:56.657 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.655+0000 7f23697fa640 1 -- 172.21.15.117:0/1504644375 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f2364004020 con 0x7f236c10b860 2024-03-04T16:08:56.657 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.655+0000 7f23697fa640 1 --2- 172.21.15.117:0/1504644375 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f235406d700 0x7f235406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:08:56.657 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.655+0000 7f23697fa640 1 -- 172.21.15.117:0/1504644375 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(770..770 src has 1..770) v4 ==== 9055+0+0 (secure 0 0 0) 0x7f236408e880 con 0x7f236c10b860 2024-03-04T16:08:56.657 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.655+0000 7f236bfff640 1 --2- 172.21.15.117:0/1504644375 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f235406d700 0x7f235406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:08:56.657 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.655+0000 7f236bfff640 1 --2- 172.21.15.117:0/1504644375 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f235406d700 0x7f235406fbc0 secure :-1 s=READY pgs=3216 cs=0 l=1 rev1=1 crypto rx=0x7f235c0096c0 tx=0x7f235c019610 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:08:56.659 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.659+0000 7f23697fa640 1 -- 172.21.15.117:0/1504644375 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f236405bb20 con 0x7f236c10b860 2024-03-04T16:08:56.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:56 smithi117 bash[21480]: audit 2024-03-04T16:08:55.498339+0000 mon.a (mon.0) 3615 : audit 1 from='client.? 172.21.15.117:0/13998417' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "10"}]': finished 2024-03-04T16:08:56.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:56 smithi117 bash[21480]: cluster 2024-03-04T16:08:55.503687+0000 mon.a (mon.0) 3616 : cluster 0 osdmap e769: 8 total, 8 up, 8 in 2024-03-04T16:08:56.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:56 smithi117 bash[21480]: audit 2024-03-04T16:08:55.512980+0000 mon.c (mon.2) 825 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:08:56.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:56 smithi117 bash[21480]: audit 2024-03-04T16:08:55.514343+0000 mon.c (mon.2) 826 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:08:56.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:56 smithi117 bash[21480]: audit 2024-03-04T16:08:55.525618+0000 mon.a (mon.0) 3617 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:08:56.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:56 smithi117 bash[21480]: audit 2024-03-04T16:08:55.616208+0000 mon.a (mon.0) 3618 : audit 1 from='client.? 172.21.15.117:0/13998417' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "10"}]: dispatch 2024-03-04T16:08:56.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:56 smithi117 bash[28843]: audit 2024-03-04T16:08:55.498339+0000 mon.a (mon.0) 3615 : audit 1 from='client.? 172.21.15.117:0/13998417' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "10"}]': finished 2024-03-04T16:08:56.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:56 smithi117 bash[28843]: cluster 2024-03-04T16:08:55.503687+0000 mon.a (mon.0) 3616 : cluster 0 osdmap e769: 8 total, 8 up, 8 in 2024-03-04T16:08:56.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:56 smithi117 bash[28843]: audit 2024-03-04T16:08:55.512980+0000 mon.c (mon.2) 825 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:08:56.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:56 smithi117 bash[28843]: audit 2024-03-04T16:08:55.514343+0000 mon.c (mon.2) 826 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:08:56.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:56 smithi117 bash[28843]: audit 2024-03-04T16:08:55.525618+0000 mon.a (mon.0) 3617 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:08:56.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:56 smithi117 bash[28843]: audit 2024-03-04T16:08:55.616208+0000 mon.a (mon.0) 3618 : audit 1 from='client.? 172.21.15.117:0/13998417' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "10"}]: dispatch 2024-03-04T16:08:56.831 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:56.831+0000 7f237229e640 1 -- 172.21.15.117:0/1504644375 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"} v 0) v1 -- 0x7f236c1a6c80 con 0x7f236c10b860 2024-03-04T16:08:56.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:56 smithi151 bash[23756]: audit 2024-03-04T16:08:55.498339+0000 mon.a (mon.0) 3615 : audit 1 from='client.? 172.21.15.117:0/13998417' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "10"}]': finished 2024-03-04T16:08:56.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:56 smithi151 bash[23756]: cluster 2024-03-04T16:08:55.503687+0000 mon.a (mon.0) 3616 : cluster 0 osdmap e769: 8 total, 8 up, 8 in 2024-03-04T16:08:56.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:56 smithi151 bash[23756]: audit 2024-03-04T16:08:55.512980+0000 mon.c (mon.2) 825 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:08:56.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:56 smithi151 bash[23756]: audit 2024-03-04T16:08:55.514343+0000 mon.c (mon.2) 826 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:08:56.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:56 smithi151 bash[23756]: audit 2024-03-04T16:08:55.525618+0000 mon.a (mon.0) 3617 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:08:56.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:56 smithi151 bash[23756]: audit 2024-03-04T16:08:55.616208+0000 mon.a (mon.0) 3618 : audit 1 from='client.? 172.21.15.117:0/13998417' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "10"}]: dispatch 2024-03-04T16:08:57.540 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:57.539+0000 7f23697fa640 1 -- 172.21.15.117:0/1504644375 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"}]=0 enabled application 'rados' on pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' v771) v1 ==== 213+0+0 (secure 0 0 0) 0x7f236405f570 con 0x7f236c10b860 2024-03-04T16:08:57.647 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:57.647+0000 7f237229e640 1 -- 172.21.15.117:0/1504644375 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"} v 0) v1 -- 0x7f236c19b230 con 0x7f236c10b860 2024-03-04T16:08:57.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:08:57 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:08:57.697 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:57 smithi151 bash[23756]: audit 2024-03-04T16:08:56.501211+0000 mon.a (mon.0) 3619 : audit 1 from='client.? 172.21.15.117:0/13998417' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "10"}]': finished 2024-03-04T16:08:57.698 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:57 smithi151 bash[23756]: cluster 2024-03-04T16:08:56.506397+0000 mon.a (mon.0) 3620 : cluster 0 osdmap e770: 8 total, 8 up, 8 in 2024-03-04T16:08:57.698 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:57 smithi151 bash[23756]: audit 2024-03-04T16:08:56.835358+0000 mon.c (mon.2) 827 : audit 1 from='client.? 172.21.15.117:0/1504644375' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"}]: dispatch 2024-03-04T16:08:57.698 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:57 smithi151 bash[23756]: audit 2024-03-04T16:08:56.836215+0000 mon.a (mon.0) 3621 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"}]: dispatch 2024-03-04T16:08:57.698 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:57 smithi151 bash[23756]: cluster 2024-03-04T16:08:57.262313+0000 mgr.y (mgr.24428) 734 : cluster 0 pgmap v1250: 176 pgs: 176 active+clean; 582 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:08:57.698 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:57 smithi151 bash[23756]: audit 2024-03-04T16:08:57.289332+0000 mgr.y (mgr.24428) 735 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:08:57.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:57 smithi117 bash[21480]: audit 2024-03-04T16:08:56.501211+0000 mon.a (mon.0) 3619 : audit 1 from='client.? 172.21.15.117:0/13998417' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "10"}]': finished 2024-03-04T16:08:57.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:57 smithi117 bash[21480]: cluster 2024-03-04T16:08:56.506397+0000 mon.a (mon.0) 3620 : cluster 0 osdmap e770: 8 total, 8 up, 8 in 2024-03-04T16:08:57.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:57 smithi117 bash[21480]: audit 2024-03-04T16:08:56.835358+0000 mon.c (mon.2) 827 : audit 1 from='client.? 172.21.15.117:0/1504644375' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"}]: dispatch 2024-03-04T16:08:57.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:57 smithi117 bash[21480]: audit 2024-03-04T16:08:56.836215+0000 mon.a (mon.0) 3621 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"}]: dispatch 2024-03-04T16:08:57.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:57 smithi117 bash[21480]: cluster 2024-03-04T16:08:57.262313+0000 mgr.y (mgr.24428) 734 : cluster 0 pgmap v1250: 176 pgs: 176 active+clean; 582 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:08:57.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:57 smithi117 bash[21480]: audit 2024-03-04T16:08:57.289332+0000 mgr.y (mgr.24428) 735 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:08:57.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:57 smithi117 bash[28843]: audit 2024-03-04T16:08:56.501211+0000 mon.a (mon.0) 3619 : audit 1 from='client.? 172.21.15.117:0/13998417' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "10"}]': finished 2024-03-04T16:08:57.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:57 smithi117 bash[28843]: cluster 2024-03-04T16:08:56.506397+0000 mon.a (mon.0) 3620 : cluster 0 osdmap e770: 8 total, 8 up, 8 in 2024-03-04T16:08:57.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:57 smithi117 bash[28843]: audit 2024-03-04T16:08:56.835358+0000 mon.c (mon.2) 827 : audit 1 from='client.? 172.21.15.117:0/1504644375' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"}]: dispatch 2024-03-04T16:08:57.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:57 smithi117 bash[28843]: audit 2024-03-04T16:08:56.836215+0000 mon.a (mon.0) 3621 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"}]: dispatch 2024-03-04T16:08:57.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:57 smithi117 bash[28843]: cluster 2024-03-04T16:08:57.262313+0000 mgr.y (mgr.24428) 734 : cluster 0 pgmap v1250: 176 pgs: 176 active+clean; 582 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:08:57.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:57 smithi117 bash[28843]: audit 2024-03-04T16:08:57.289332+0000 mgr.y (mgr.24428) 735 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:08:58.539 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:58.539+0000 7f23697fa640 1 -- 172.21.15.117:0/1504644375 <== mon.2 v2:172.21.15.117:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"}]=0 enabled application 'rados' on pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' v772) v1 ==== 213+0+0 (secure 0 0 0) 0x7f2364063b30 con 0x7f236c10b860 2024-03-04T16:08:58.540 INFO:tasks.workunit.client.0.smithi117.stderr:enabled application 'rados' on pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' 2024-03-04T16:08:58.544 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:58.543+0000 7f237229e640 1 -- 172.21.15.117:0/1504644375 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f235406d700 msgr2=0x7f235406fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:08:58.544 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:58.543+0000 7f237229e640 1 --2- 172.21.15.117:0/1504644375 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f235406d700 0x7f235406fbc0 secure :-1 s=READY pgs=3216 cs=0 l=1 rev1=1 crypto rx=0x7f235c0096c0 tx=0x7f235c019610 comp rx=0 tx=0).stop 2024-03-04T16:08:58.544 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:58.543+0000 7f237229e640 1 -- 172.21.15.117:0/1504644375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f236c10b860 msgr2=0x7f236c19e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:08:58.545 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:58.543+0000 7f237229e640 1 --2- 172.21.15.117:0/1504644375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f236c10b860 0x7f236c19e7a0 secure :-1 s=READY pgs=3098 cs=0 l=1 rev1=1 crypto rx=0x7f236400efd0 tx=0x7f236400c6a0 comp rx=0 tx=0).stop 2024-03-04T16:08:58.545 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:58.543+0000 7f237229e640 1 -- 172.21.15.117:0/1504644375 shutdown_connections 2024-03-04T16:08:58.545 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:58.543+0000 7f237229e640 1 --2- 172.21.15.117:0/1504644375 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f235406d700 0x7f235406fbc0 unknown :-1 s=CLOSED pgs=3216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:58.545 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:58.543+0000 7f237229e640 1 --2- 172.21.15.117:0/1504644375 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f236c102d50 0x7f236c19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:58.545 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:58.543+0000 7f237229e640 1 --2- 172.21.15.117:0/1504644375 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f236c10b860 0x7f236c19e7a0 unknown :-1 s=CLOSED pgs=3098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:58.545 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:58.543+0000 7f237229e640 1 --2- 172.21.15.117:0/1504644375 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f236c102430 0x7f236c199ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:08:58.545 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:58.543+0000 7f237229e640 1 -- 172.21.15.117:0/1504644375 >> 172.21.15.117:0/1504644375 conn(0x7f236c0fbc60 msgr2=0x7f236c10cf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:08:58.545 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:58.543+0000 7f237229e640 1 -- 172.21.15.117:0/1504644375 shutdown_connections 2024-03-04T16:08:58.545 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:08:58.543+0000 7f237229e640 1 -- 172.21.15.117:0/1504644375 wait complete. 2024-03-04T16:08:58.565 INFO:tasks.workunit.client.0.smithi117.stderr:+ seq 1 10 2024-03-04T16:08:58.674 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p 736f5599-4143-4934-af9d-08c0b0a4b1a3 put obj1 /etc/passwd 2024-03-04T16:08:58.724 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p 736f5599-4143-4934-af9d-08c0b0a4b1a3 put obj2 /etc/passwd 2024-03-04T16:08:58.765 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p 736f5599-4143-4934-af9d-08c0b0a4b1a3 put obj3 /etc/passwd 2024-03-04T16:08:58.804 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p 736f5599-4143-4934-af9d-08c0b0a4b1a3 put obj4 /etc/passwd 2024-03-04T16:08:58.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:58 smithi117 bash[28843]: audit 2024-03-04T16:08:57.529990+0000 mon.a (mon.0) 3622 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"}]': finished 2024-03-04T16:08:58.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:58 smithi117 bash[28843]: cluster 2024-03-04T16:08:57.540040+0000 mon.a (mon.0) 3623 : cluster 0 osdmap e771: 8 total, 8 up, 8 in 2024-03-04T16:08:58.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:58 smithi117 bash[28843]: audit 2024-03-04T16:08:57.652107+0000 mon.c (mon.2) 828 : audit 1 from='client.? 172.21.15.117:0/1504644375' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"}]: dispatch 2024-03-04T16:08:58.817 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:58 smithi117 bash[28843]: audit 2024-03-04T16:08:57.652739+0000 mon.a (mon.0) 3624 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"}]: dispatch 2024-03-04T16:08:58.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:58 smithi117 bash[21480]: audit 2024-03-04T16:08:57.529990+0000 mon.a (mon.0) 3622 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"}]': finished 2024-03-04T16:08:58.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:58 smithi117 bash[21480]: cluster 2024-03-04T16:08:57.540040+0000 mon.a (mon.0) 3623 : cluster 0 osdmap e771: 8 total, 8 up, 8 in 2024-03-04T16:08:58.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:58 smithi117 bash[21480]: audit 2024-03-04T16:08:57.652107+0000 mon.c (mon.2) 828 : audit 1 from='client.? 172.21.15.117:0/1504644375' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"}]: dispatch 2024-03-04T16:08:58.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:58 smithi117 bash[21480]: audit 2024-03-04T16:08:57.652739+0000 mon.a (mon.0) 3624 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"}]: dispatch 2024-03-04T16:08:58.841 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p 736f5599-4143-4934-af9d-08c0b0a4b1a3 put obj5 /etc/passwd 2024-03-04T16:08:58.879 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p 736f5599-4143-4934-af9d-08c0b0a4b1a3 put obj6 /etc/passwd 2024-03-04T16:08:58.918 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p 736f5599-4143-4934-af9d-08c0b0a4b1a3 put obj7 /etc/passwd 2024-03-04T16:08:58.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:58 smithi151 bash[23756]: audit 2024-03-04T16:08:57.529990+0000 mon.a (mon.0) 3622 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"}]': finished 2024-03-04T16:08:58.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:58 smithi151 bash[23756]: cluster 2024-03-04T16:08:57.540040+0000 mon.a (mon.0) 3623 : cluster 0 osdmap e771: 8 total, 8 up, 8 in 2024-03-04T16:08:58.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:58 smithi151 bash[23756]: audit 2024-03-04T16:08:57.652107+0000 mon.c (mon.2) 828 : audit 1 from='client.? 172.21.15.117:0/1504644375' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"}]: dispatch 2024-03-04T16:08:58.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:58 smithi151 bash[23756]: audit 2024-03-04T16:08:57.652739+0000 mon.a (mon.0) 3624 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"}]: dispatch 2024-03-04T16:08:58.955 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p 736f5599-4143-4934-af9d-08c0b0a4b1a3 put obj8 /etc/passwd 2024-03-04T16:08:58.993 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p 736f5599-4143-4934-af9d-08c0b0a4b1a3 put obj9 /etc/passwd 2024-03-04T16:08:59.031 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p 736f5599-4143-4934-af9d-08c0b0a4b1a3 put obj10 /etc/passwd 2024-03-04T16:08:59.067 INFO:tasks.workunit.client.0.smithi117.stderr:+ sleep 30 2024-03-04T16:08:59.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:59 smithi117 bash[21480]: audit 2024-03-04T16:08:58.532556+0000 mon.a (mon.0) 3625 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"}]': finished 2024-03-04T16:08:59.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:59 smithi117 bash[21480]: cluster 2024-03-04T16:08:58.540065+0000 mon.a (mon.0) 3626 : cluster 0 osdmap e772: 8 total, 8 up, 8 in 2024-03-04T16:08:59.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:08:59 smithi117 bash[21480]: cluster 2024-03-04T16:08:59.263048+0000 mgr.y (mgr.24428) 736 : cluster 0 pgmap v1253: 176 pgs: 176 active+clean; 582 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:08:59.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:59 smithi117 bash[28843]: audit 2024-03-04T16:08:58.532556+0000 mon.a (mon.0) 3625 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"}]': finished 2024-03-04T16:08:59.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:59 smithi117 bash[28843]: cluster 2024-03-04T16:08:58.540065+0000 mon.a (mon.0) 3626 : cluster 0 osdmap e772: 8 total, 8 up, 8 in 2024-03-04T16:08:59.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:08:59 smithi117 bash[28843]: cluster 2024-03-04T16:08:59.263048+0000 mgr.y (mgr.24428) 736 : cluster 0 pgmap v1253: 176 pgs: 176 active+clean; 582 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:08:59.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:59 smithi151 bash[23756]: audit 2024-03-04T16:08:58.532556+0000 mon.a (mon.0) 3625 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "app": "rados"}]': finished 2024-03-04T16:08:59.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:59 smithi151 bash[23756]: cluster 2024-03-04T16:08:58.540065+0000 mon.a (mon.0) 3626 : cluster 0 osdmap e772: 8 total, 8 up, 8 in 2024-03-04T16:08:59.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:08:59 smithi151 bash[23756]: cluster 2024-03-04T16:08:59.263048+0000 mgr.y (mgr.24428) 736 : cluster 0 pgmap v1253: 176 pgs: 176 active+clean; 582 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:09:00.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:00 smithi117 bash[21480]: audit 2024-03-04T16:09:00.428714+0000 mon.c (mon.2) 829 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:09:00.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:00 smithi117 bash[28843]: audit 2024-03-04T16:09:00.428714+0000 mon.c (mon.2) 829 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:09:00.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:00 smithi151 bash[23756]: audit 2024-03-04T16:09:00.428714+0000 mon.c (mon.2) 829 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:09:01.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:01 smithi117 bash[21480]: cluster 2024-03-04T16:09:01.263847+0000 mgr.y (mgr.24428) 737 : cluster 0 pgmap v1254: 176 pgs: 176 active+clean; 582 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 888 B/s rd, 0 op/s 2024-03-04T16:09:01.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:01 smithi117 bash[28843]: cluster 2024-03-04T16:09:01.263847+0000 mgr.y (mgr.24428) 737 : cluster 0 pgmap v1254: 176 pgs: 176 active+clean; 582 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 888 B/s rd, 0 op/s 2024-03-04T16:09:01.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:01 smithi151 bash[23756]: cluster 2024-03-04T16:09:01.263847+0000 mgr.y (mgr.24428) 737 : cluster 0 pgmap v1254: 176 pgs: 176 active+clean; 582 KiB data, 793 MiB used, 714 GiB / 715 GiB avail; 888 B/s rd, 0 op/s 2024-03-04T16:09:04.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:04 smithi151 bash[23756]: cluster 2024-03-04T16:09:03.265720+0000 mgr.y (mgr.24428) 738 : cluster 0 pgmap v1255: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 757 B/s rd, 18 KiB/s wr, 2 op/s 2024-03-04T16:09:04.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:04 smithi117 bash[21480]: cluster 2024-03-04T16:09:03.265720+0000 mgr.y (mgr.24428) 738 : cluster 0 pgmap v1255: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 757 B/s rd, 18 KiB/s wr, 2 op/s 2024-03-04T16:09:04.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:04 smithi117 bash[28843]: cluster 2024-03-04T16:09:03.265720+0000 mgr.y (mgr.24428) 738 : cluster 0 pgmap v1255: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 757 B/s rd, 18 KiB/s wr, 2 op/s 2024-03-04T16:09:05.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:05 smithi151 bash[23756]: cluster 2024-03-04T16:09:04.903350+0000 mon.a (mon.0) 3627 : cluster 3 pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' is full (reached quota's max_objects: 10) 2024-03-04T16:09:05.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:05 smithi151 bash[23756]: cluster 2024-03-04T16:09:04.904005+0000 mon.a (mon.0) 3628 : cluster 3 Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-04T16:09:05.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:05 smithi151 bash[23756]: cluster 2024-03-04T16:09:04.913929+0000 mon.a (mon.0) 3629 : cluster 0 osdmap e773: 8 total, 8 up, 8 in 2024-03-04T16:09:05.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:05 smithi117 bash[21480]: cluster 2024-03-04T16:09:04.903350+0000 mon.a (mon.0) 3627 : cluster 3 pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' is full (reached quota's max_objects: 10) 2024-03-04T16:09:05.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:05 smithi117 bash[21480]: cluster 2024-03-04T16:09:04.904005+0000 mon.a (mon.0) 3628 : cluster 3 Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-04T16:09:05.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:05 smithi117 bash[21480]: cluster 2024-03-04T16:09:04.913929+0000 mon.a (mon.0) 3629 : cluster 0 osdmap e773: 8 total, 8 up, 8 in 2024-03-04T16:09:05.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:05 smithi117 bash[28843]: cluster 2024-03-04T16:09:04.903350+0000 mon.a (mon.0) 3627 : cluster 3 pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' is full (reached quota's max_objects: 10) 2024-03-04T16:09:05.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:05 smithi117 bash[28843]: cluster 2024-03-04T16:09:04.904005+0000 mon.a (mon.0) 3628 : cluster 3 Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-04T16:09:05.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:05 smithi117 bash[28843]: cluster 2024-03-04T16:09:04.913929+0000 mon.a (mon.0) 3629 : cluster 0 osdmap e773: 8 total, 8 up, 8 in 2024-03-04T16:09:06.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:06 smithi117 bash[21480]: cluster 2024-03-04T16:09:05.266543+0000 mgr.y (mgr.24428) 739 : cluster 0 pgmap v1257: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 662 B/s rd, 16 KiB/s wr, 1 op/s 2024-03-04T16:09:06.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:09:06 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:09:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:09:06.568 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:06 smithi117 bash[28843]: cluster 2024-03-04T16:09:05.266543+0000 mgr.y (mgr.24428) 739 : cluster 0 pgmap v1257: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 662 B/s rd, 16 KiB/s wr, 1 op/s 2024-03-04T16:09:06.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:06 smithi151 bash[23756]: cluster 2024-03-04T16:09:05.266543+0000 mgr.y (mgr.24428) 739 : cluster 0 pgmap v1257: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 662 B/s rd, 16 KiB/s wr, 1 op/s 2024-03-04T16:09:07.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:09:07 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:09:08.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:08 smithi151 bash[23756]: cluster 2024-03-04T16:09:07.268391+0000 mgr.y (mgr.24428) 740 : cluster 0 pgmap v1258: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-03-04T16:09:08.683 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:08 smithi151 bash[23756]: audit 2024-03-04T16:09:07.292910+0000 mgr.y (mgr.24428) 741 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:09:08.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:08 smithi117 bash[21480]: cluster 2024-03-04T16:09:07.268391+0000 mgr.y (mgr.24428) 740 : cluster 0 pgmap v1258: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-03-04T16:09:08.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:08 smithi117 bash[21480]: audit 2024-03-04T16:09:07.292910+0000 mgr.y (mgr.24428) 741 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:09:08.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:08 smithi117 bash[28843]: cluster 2024-03-04T16:09:07.268391+0000 mgr.y (mgr.24428) 740 : cluster 0 pgmap v1258: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 14 KiB/s wr, 2 op/s 2024-03-04T16:09:08.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:08 smithi117 bash[28843]: audit 2024-03-04T16:09:07.292910+0000 mgr.y (mgr.24428) 741 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:09:10.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:10 smithi151 bash[23756]: cluster 2024-03-04T16:09:09.269202+0000 mgr.y (mgr.24428) 742 : cluster 0 pgmap v1259: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-04T16:09:10.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:10 smithi117 bash[28843]: cluster 2024-03-04T16:09:09.269202+0000 mgr.y (mgr.24428) 742 : cluster 0 pgmap v1259: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-04T16:09:10.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:10 smithi117 bash[21480]: cluster 2024-03-04T16:09:09.269202+0000 mgr.y (mgr.24428) 742 : cluster 0 pgmap v1259: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-04T16:09:12.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:12 smithi151 bash[23756]: cluster 2024-03-04T16:09:11.269935+0000 mgr.y (mgr.24428) 743 : cluster 0 pgmap v1260: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-04T16:09:12.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:12 smithi117 bash[21480]: cluster 2024-03-04T16:09:11.269935+0000 mgr.y (mgr.24428) 743 : cluster 0 pgmap v1260: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-04T16:09:12.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:12 smithi117 bash[28843]: cluster 2024-03-04T16:09:11.269935+0000 mgr.y (mgr.24428) 743 : cluster 0 pgmap v1260: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-04T16:09:14.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:14 smithi151 bash[23756]: cluster 2024-03-04T16:09:13.271818+0000 mgr.y (mgr.24428) 744 : cluster 0 pgmap v1261: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:09:14.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:14 smithi117 bash[21480]: cluster 2024-03-04T16:09:13.271818+0000 mgr.y (mgr.24428) 744 : cluster 0 pgmap v1261: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:09:14.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:14 smithi117 bash[28843]: cluster 2024-03-04T16:09:13.271818+0000 mgr.y (mgr.24428) 744 : cluster 0 pgmap v1261: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:09:16.350 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:09:16 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:09:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:09:16.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:16 smithi151 bash[23756]: cluster 2024-03-04T16:09:15.272606+0000 mgr.y (mgr.24428) 745 : cluster 0 pgmap v1262: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 988 B/s rd, 0 op/s 2024-03-04T16:09:16.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:16 smithi151 bash[23756]: audit 2024-03-04T16:09:15.470448+0000 mon.a (mon.0) 3630 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:09:16.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:16 smithi151 bash[23756]: audit 2024-03-04T16:09:15.474921+0000 mon.c (mon.2) 830 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:09:16.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:16 smithi117 bash[21480]: cluster 2024-03-04T16:09:15.272606+0000 mgr.y (mgr.24428) 745 : cluster 0 pgmap v1262: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 988 B/s rd, 0 op/s 2024-03-04T16:09:16.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:16 smithi117 bash[21480]: audit 2024-03-04T16:09:15.470448+0000 mon.a (mon.0) 3630 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:09:16.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:16 smithi117 bash[21480]: audit 2024-03-04T16:09:15.474921+0000 mon.c (mon.2) 830 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:09:16.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:16 smithi117 bash[28843]: cluster 2024-03-04T16:09:15.272606+0000 mgr.y (mgr.24428) 745 : cluster 0 pgmap v1262: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 988 B/s rd, 0 op/s 2024-03-04T16:09:16.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:16 smithi117 bash[28843]: audit 2024-03-04T16:09:15.470448+0000 mon.a (mon.0) 3630 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:09:16.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:16 smithi117 bash[28843]: audit 2024-03-04T16:09:15.474921+0000 mon.c (mon.2) 830 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:09:17.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:09:17 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:09:18.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:18 smithi151 bash[23756]: cluster 2024-03-04T16:09:17.274513+0000 mgr.y (mgr.24428) 746 : cluster 0 pgmap v1263: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:09:18.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:18 smithi151 bash[23756]: audit 2024-03-04T16:09:17.304214+0000 mgr.y (mgr.24428) 747 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:09:18.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:18 smithi117 bash[21480]: cluster 2024-03-04T16:09:17.274513+0000 mgr.y (mgr.24428) 746 : cluster 0 pgmap v1263: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:09:18.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:18 smithi117 bash[21480]: audit 2024-03-04T16:09:17.304214+0000 mgr.y (mgr.24428) 747 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:09:18.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:18 smithi117 bash[28843]: cluster 2024-03-04T16:09:17.274513+0000 mgr.y (mgr.24428) 746 : cluster 0 pgmap v1263: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:09:18.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:18 smithi117 bash[28843]: audit 2024-03-04T16:09:17.304214+0000 mgr.y (mgr.24428) 747 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:09:20.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:20 smithi151 bash[23756]: cluster 2024-03-04T16:09:19.275221+0000 mgr.y (mgr.24428) 748 : cluster 0 pgmap v1264: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:09:20.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:20 smithi117 bash[21480]: cluster 2024-03-04T16:09:19.275221+0000 mgr.y (mgr.24428) 748 : cluster 0 pgmap v1264: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:09:20.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:20 smithi117 bash[28843]: cluster 2024-03-04T16:09:19.275221+0000 mgr.y (mgr.24428) 748 : cluster 0 pgmap v1264: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:09:22.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:22 smithi151 bash[23756]: cluster 2024-03-04T16:09:21.275960+0000 mgr.y (mgr.24428) 749 : cluster 0 pgmap v1265: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:09:22.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:22 smithi117 bash[21480]: cluster 2024-03-04T16:09:21.275960+0000 mgr.y (mgr.24428) 749 : cluster 0 pgmap v1265: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:09:22.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:22 smithi117 bash[28843]: cluster 2024-03-04T16:09:21.275960+0000 mgr.y (mgr.24428) 749 : cluster 0 pgmap v1265: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:09:24.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:24 smithi151 bash[23756]: cluster 2024-03-04T16:09:23.277899+0000 mgr.y (mgr.24428) 750 : cluster 0 pgmap v1266: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:09:24.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:24 smithi117 bash[21480]: cluster 2024-03-04T16:09:23.277899+0000 mgr.y (mgr.24428) 750 : cluster 0 pgmap v1266: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:09:24.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:24 smithi117 bash[28843]: cluster 2024-03-04T16:09:23.277899+0000 mgr.y (mgr.24428) 750 : cluster 0 pgmap v1266: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:09:26.374 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:09:26 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:09:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:09:26.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:26 smithi151 bash[23756]: cluster 2024-03-04T16:09:25.278669+0000 mgr.y (mgr.24428) 751 : cluster 0 pgmap v1267: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:09:26.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:26 smithi117 bash[28843]: cluster 2024-03-04T16:09:25.278669+0000 mgr.y (mgr.24428) 751 : cluster 0 pgmap v1267: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:09:26.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:26 smithi117 bash[21480]: cluster 2024-03-04T16:09:25.278669+0000 mgr.y (mgr.24428) 751 : cluster 0 pgmap v1267: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:09:27.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:09:27 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:09:28.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:28 smithi151 bash[23756]: cluster 2024-03-04T16:09:27.280544+0000 mgr.y (mgr.24428) 752 : cluster 0 pgmap v1268: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:09:28.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:28 smithi151 bash[23756]: audit 2024-03-04T16:09:27.314552+0000 mgr.y (mgr.24428) 753 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:09:28.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:28 smithi117 bash[21480]: cluster 2024-03-04T16:09:27.280544+0000 mgr.y (mgr.24428) 752 : cluster 0 pgmap v1268: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:09:28.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:28 smithi117 bash[21480]: audit 2024-03-04T16:09:27.314552+0000 mgr.y (mgr.24428) 753 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:09:28.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:28 smithi117 bash[28843]: cluster 2024-03-04T16:09:27.280544+0000 mgr.y (mgr.24428) 752 : cluster 0 pgmap v1268: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:09:28.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:28 smithi117 bash[28843]: audit 2024-03-04T16:09:27.314552+0000 mgr.y (mgr.24428) 753 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:09:29.069 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=131999 2024-03-04T16:09:29.069 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph osd pool set-quota 736f5599-4143-4934-af9d-08c0b0a4b1a3 max_objects 100 2024-03-04T16:09:29.069 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p 736f5599-4143-4934-af9d-08c0b0a4b1a3 put onemore /etc/passwd 2024-03-04T16:09:29.197 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd633855640 1 -- 172.21.15.117:0/1028440076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd62c101e80 msgr2=0x7fd62c105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:09:29.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd633855640 1 --2- 172.21.15.117:0/1028440076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd62c101e80 0x7fd62c105ed0 secure :-1 s=READY pgs=3121 cs=0 l=1 rev1=1 crypto rx=0x7fd628009930 tx=0x7fd62801c7e0 comp rx=0 tx=0).stop 2024-03-04T16:09:29.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd633855640 1 -- 172.21.15.117:0/1028440076 shutdown_connections 2024-03-04T16:09:29.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd633855640 1 --2- 172.21.15.117:0/1028440076 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd62c106600 0x7fd62c10e180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:09:29.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd633855640 1 --2- 172.21.15.117:0/1028440076 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd62c101e80 0x7fd62c105ed0 unknown :-1 s=CLOSED pgs=3121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:09:29.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd633855640 1 --2- 172.21.15.117:0/1028440076 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd62c1014d0 0x7fd62c1018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:09:29.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd633855640 1 -- 172.21.15.117:0/1028440076 >> 172.21.15.117:0/1028440076 conn(0x7fd62c0fcdc0 msgr2=0x7fd62c0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:09:29.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd633855640 1 -- 172.21.15.117:0/1028440076 shutdown_connections 2024-03-04T16:09:29.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd633855640 1 -- 172.21.15.117:0/1028440076 wait complete. 2024-03-04T16:09:29.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd633855640 1 Processor -- start 2024-03-04T16:09:29.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd633855640 1 -- start start 2024-03-04T16:09:29.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd633855640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd62c1014d0 0x7fd62c199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:09:29.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd633855640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd62c101e80 0x7fd62c19a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:09:29.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd633855640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd62c106600 0x7fd62c19e790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:09:29.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd633855640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c113880 con 0x7fd62c106600 2024-03-04T16:09:29.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd633855640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c113700 con 0x7fd62c101e80 2024-03-04T16:09:29.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd633855640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd62c113a00 con 0x7fd62c1014d0 2024-03-04T16:09:29.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd6315f1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd62c1014d0 0x7fd62c199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:09:29.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd631df2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd62c106600 0x7fd62c19e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:09:29.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd6315f1640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd62c1014d0 0x7fd62c199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:51752/0 (socket says 172.21.15.117:51752) 2024-03-04T16:09:29.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd6315f1640 1 -- 172.21.15.117:0/3439654890 learned_addr learned my addr 172.21.15.117:0/3439654890 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T16:09:29.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd630df0640 1 --2- 172.21.15.117:0/3439654890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd62c101e80 0x7fd62c19a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:09:29.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd631df2640 1 -- 172.21.15.117:0/3439654890 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd62c1014d0 msgr2=0x7fd62c199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:09:29.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd631df2640 1 --2- 172.21.15.117:0/3439654890 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd62c1014d0 0x7fd62c199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:09:29.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd631df2640 1 -- 172.21.15.117:0/3439654890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd62c101e80 msgr2=0x7fd62c19a400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:09:29.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd631df2640 1 --2- 172.21.15.117:0/3439654890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd62c101e80 0x7fd62c19a400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:09:29.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd631df2640 1 -- 172.21.15.117:0/3439654890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd62c19ee70 con 0x7fd62c106600 2024-03-04T16:09:29.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd6315f1640 1 --2- 172.21.15.117:0/3439654890 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd62c1014d0 0x7fd62c199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T16:09:29.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd630df0640 1 --2- 172.21.15.117:0/3439654890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd62c101e80 0x7fd62c19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T16:09:29.200 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd631df2640 1 --2- 172.21.15.117:0/3439654890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd62c106600 0x7fd62c19e790 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7fd62400ec80 tx=0x7fd62400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:09:29.200 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd6227fc640 1 -- 172.21.15.117:0/3439654890 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624003b80 con 0x7fd62c106600 2024-03-04T16:09:29.200 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd633855640 1 -- 172.21.15.117:0/3439654890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd62c19f160 con 0x7fd62c106600 2024-03-04T16:09:29.200 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd6227fc640 1 -- 172.21.15.117:0/3439654890 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fd62400ee90 con 0x7fd62c106600 2024-03-04T16:09:29.200 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd6227fc640 1 -- 172.21.15.117:0/3439654890 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd624010690 con 0x7fd62c106600 2024-03-04T16:09:29.200 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.194+0000 7fd633855640 1 -- 172.21.15.117:0/3439654890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd62c1a6a40 con 0x7fd62c106600 2024-03-04T16:09:29.200 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.198+0000 7fd633855640 1 -- 172.21.15.117:0/3439654890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd62c0681e0 con 0x7fd62c106600 2024-03-04T16:09:29.201 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.198+0000 7fd6227fc640 1 -- 172.21.15.117:0/3439654890 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7fd624004020 con 0x7fd62c106600 2024-03-04T16:09:29.201 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.198+0000 7fd6227fc640 1 --2- 172.21.15.117:0/3439654890 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fd61806d700 0x7fd61806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:09:29.201 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.198+0000 7fd6227fc640 1 -- 172.21.15.117:0/3439654890 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(773..773 src has 1..773) v4 ==== 9068+0+0 (secure 0 0 0) 0x7fd62408e5c0 con 0x7fd62c106600 2024-03-04T16:09:29.201 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.198+0000 7fd6315f1640 1 --2- 172.21.15.117:0/3439654890 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fd61806d700 0x7fd61806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:09:29.201 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.198+0000 7fd6227fc640 1 -- 172.21.15.117:0/3439654890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=774}) v3 -- 0x7fd618079190 con 0x7fd62c106600 2024-03-04T16:09:29.201 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.198+0000 7fd6315f1640 1 --2- 172.21.15.117:0/3439654890 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fd61806d700 0x7fd61806fbc0 secure :-1 s=READY pgs=3228 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0058d0 tx=0x7fd61c00a380 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:09:29.203 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.202+0000 7fd6227fc640 1 -- 172.21.15.117:0/3439654890 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fd62405b7d0 con 0x7fd62c106600 2024-03-04T16:09:29.376 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.374+0000 7fd633855640 1 -- 172.21.15.117:0/3439654890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7fd62c1a6c10 con 0x7fd62c106600 2024-03-04T16:09:29.388 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.386+0000 7fd6227fc640 1 -- 172.21.15.117:0/3439654890 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 v774) v1 ==== 225+0+0 (secure 0 0 0) 0x7fd62405f220 con 0x7fd62c106600 2024-03-04T16:09:29.391 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.390+0000 7fd6227fc640 1 -- 172.21.15.117:0/3439654890 <== mon.0 v2:172.21.15.117:3300/0 8 ==== osd_map(774..774 src has 1..774) v4 ==== 628+0+0 (secure 0 0 0) 0x7fd624059170 con 0x7fd62c106600 2024-03-04T16:09:29.391 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.390+0000 7fd6227fc640 1 -- 172.21.15.117:0/3439654890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=775}) v3 -- 0x7fd61807a0f0 con 0x7fd62c106600 2024-03-04T16:09:29.496 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.494+0000 7fd633855640 1 -- 172.21.15.117:0/3439654890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7fd62c1a6e60 con 0x7fd62c106600 2024-03-04T16:09:29.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:29 smithi151 bash[23756]: audit 2024-03-04T16:09:29.380016+0000 mon.a (mon.0) 3631 : audit 1 from='client.? 172.21.15.117:0/3439654890' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "100"}]: dispatch 2024-03-04T16:09:29.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:29 smithi117 bash[21480]: audit 2024-03-04T16:09:29.380016+0000 mon.a (mon.0) 3631 : audit 1 from='client.? 172.21.15.117:0/3439654890' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "100"}]: dispatch 2024-03-04T16:09:29.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:29 smithi117 bash[28843]: audit 2024-03-04T16:09:29.380016+0000 mon.a (mon.0) 3631 : audit 1 from='client.? 172.21.15.117:0/3439654890' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "100"}]: dispatch 2024-03-04T16:09:29.910 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.910+0000 7fd6227fc640 1 -- 172.21.15.117:0/3439654890 <== mon.0 v2:172.21.15.117:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 v775) v1 ==== 225+0+0 (secure 0 0 0) 0x7fd6240637e0 con 0x7fd62c106600 2024-03-04T16:09:29.910 INFO:tasks.workunit.client.0.smithi117.stderr:set-quota max_objects = 100 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 2024-03-04T16:09:29.911 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.910+0000 7fd6227fc640 1 -- 172.21.15.117:0/3439654890 <== mon.0 v2:172.21.15.117:3300/0 10 ==== osd_map(775..775 src has 1..775) v4 ==== 628+0+0 (secure 0 0 0) 0x7fd62408be60 con 0x7fd62c106600 2024-03-04T16:09:29.911 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.910+0000 7fd6227fc640 1 -- 172.21.15.117:0/3439654890 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=776}) v3 -- 0x7fd61807a370 con 0x7fd62c106600 2024-03-04T16:09:29.920 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.914+0000 7fd633855640 1 -- 172.21.15.117:0/3439654890 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fd61806d700 msgr2=0x7fd61806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:09:29.920 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.914+0000 7fd633855640 1 --2- 172.21.15.117:0/3439654890 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fd61806d700 0x7fd61806fbc0 secure :-1 s=READY pgs=3228 cs=0 l=1 rev1=1 crypto rx=0x7fd61c0058d0 tx=0x7fd61c00a380 comp rx=0 tx=0).stop 2024-03-04T16:09:29.920 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.914+0000 7fd633855640 1 -- 172.21.15.117:0/3439654890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd62c106600 msgr2=0x7fd62c19e790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:09:29.920 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.914+0000 7fd633855640 1 --2- 172.21.15.117:0/3439654890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd62c106600 0x7fd62c19e790 secure :-1 s=READY pgs=2492 cs=0 l=1 rev1=1 crypto rx=0x7fd62400ec80 tx=0x7fd62400c6a0 comp rx=0 tx=0).stop 2024-03-04T16:09:29.920 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.914+0000 7fd633855640 1 -- 172.21.15.117:0/3439654890 shutdown_connections 2024-03-04T16:09:29.920 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.914+0000 7fd633855640 1 --2- 172.21.15.117:0/3439654890 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fd61806d700 0x7fd61806fbc0 unknown :-1 s=CLOSED pgs=3228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:09:29.920 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.914+0000 7fd633855640 1 --2- 172.21.15.117:0/3439654890 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fd62c106600 0x7fd62c19e790 unknown :-1 s=CLOSED pgs=2492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:09:29.920 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.914+0000 7fd633855640 1 --2- 172.21.15.117:0/3439654890 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fd62c101e80 0x7fd62c19a400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:09:29.920 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.914+0000 7fd633855640 1 --2- 172.21.15.117:0/3439654890 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fd62c1014d0 0x7fd62c199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:09:29.920 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.914+0000 7fd633855640 1 -- 172.21.15.117:0/3439654890 >> 172.21.15.117:0/3439654890 conn(0x7fd62c0fcdc0 msgr2=0x7fd62c0fd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:09:29.921 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.918+0000 7fd633855640 1 -- 172.21.15.117:0/3439654890 shutdown_connections 2024-03-04T16:09:29.921 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:29.922+0000 7fd633855640 1 -- 172.21.15.117:0/3439654890 wait complete. 2024-03-04T16:09:29.941 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 131999 2024-03-04T16:09:29.941 INFO:tasks.workunit.client.0.smithi117.stderr:+ [ 0 -ne 0 ] 2024-03-04T16:09:29.941 INFO:tasks.workunit.client.0.smithi117.stderr:+ true 2024-03-04T16:09:29.941 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p 736f5599-4143-4934-af9d-08c0b0a4b1a3 put twomore /etc/passwd 2024-03-04T16:09:29.982 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph osd pool set-quota 736f5599-4143-4934-af9d-08c0b0a4b1a3 max_bytes 100 2024-03-04T16:09:30.102 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71daf6c640 1 -- 172.21.15.117:0/3940319026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d41034b0 msgr2=0x7f71d4103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:09:30.102 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71daf6c640 1 --2- 172.21.15.117:0/3940319026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d41034b0 0x7f71d4103930 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009930 tx=0x7f71c001c7e0 comp rx=0 tx=0).stop 2024-03-04T16:09:30.102 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71daf6c640 1 -- 172.21.15.117:0/3940319026 shutdown_connections 2024-03-04T16:09:30.102 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71daf6c640 1 --2- 172.21.15.117:0/3940319026 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d41034b0 0x7f71d4103930 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:09:30.102 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71daf6c640 1 --2- 172.21.15.117:0/3940319026 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71d4103e70 0x7f71d4110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:09:30.102 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71daf6c640 1 --2- 172.21.15.117:0/3940319026 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d4109560 0x7f71d4109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:09:30.102 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71daf6c640 1 -- 172.21.15.117:0/3940319026 >> 172.21.15.117:0/3940319026 conn(0x7f71d40fef40 msgr2=0x7f71d4101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:09:30.103 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71daf6c640 1 -- 172.21.15.117:0/3940319026 shutdown_connections 2024-03-04T16:09:30.103 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71daf6c640 1 -- 172.21.15.117:0/3940319026 wait complete. 2024-03-04T16:09:30.103 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71daf6c640 1 Processor -- start 2024-03-04T16:09:30.103 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71daf6c640 1 -- start start 2024-03-04T16:09:30.103 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71daf6c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71d41034b0 0x7f71d41a16b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:09:30.103 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71daf6c640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d4103e70 0x7f71d41a1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:09:30.103 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71daf6c640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d4109560 0x7f71d419b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:09:30.103 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71daf6c640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d4115540 con 0x7f71d4103e70 2024-03-04T16:09:30.103 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71daf6c640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71d41153c0 con 0x7f71d4109560 2024-03-04T16:09:30.103 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71daf6c640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71d41156c0 con 0x7f71d41034b0 2024-03-04T16:09:30.103 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71d8d08640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71d41034b0 0x7f71d41a16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:09:30.103 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71d3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d4103e70 0x7f71d41a1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:09:30.103 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71d8d08640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71d41034b0 0x7f71d41a16b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:51796/0 (socket says 172.21.15.117:51796) 2024-03-04T16:09:30.103 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71d3fff640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d4103e70 0x7f71d41a1bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:42700/0 (socket says 172.21.15.117:42700) 2024-03-04T16:09:30.103 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71d8d08640 1 -- 172.21.15.117:0/4086043450 learned_addr learned my addr 172.21.15.117:0/4086043450 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T16:09:30.104 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.098+0000 7f71d9509640 1 --2- 172.21.15.117:0/4086043450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d4109560 0x7f71d419b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:09:30.104 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.102+0000 7f71d3fff640 1 -- 172.21.15.117:0/4086043450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71d41034b0 msgr2=0x7f71d41a16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:09:30.104 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.102+0000 7f71d3fff640 1 --2- 172.21.15.117:0/4086043450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71d41034b0 0x7f71d41a16b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:09:30.104 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.102+0000 7f71d3fff640 1 -- 172.21.15.117:0/4086043450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d4109560 msgr2=0x7f71d419b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:09:30.104 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.102+0000 7f71d3fff640 1 --2- 172.21.15.117:0/4086043450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d4109560 0x7f71d419b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:09:30.104 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.102+0000 7f71d3fff640 1 -- 172.21.15.117:0/4086043450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d419bfd0 con 0x7f71d4103e70 2024-03-04T16:09:30.104 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.102+0000 7f71d9509640 1 --2- 172.21.15.117:0/4086043450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d4109560 0x7f71d419b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T16:09:30.104 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.102+0000 7f71d8d08640 1 --2- 172.21.15.117:0/4086043450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71d41034b0 0x7f71d41a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T16:09:30.104 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.102+0000 7f71d3fff640 1 --2- 172.21.15.117:0/4086043450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d4103e70 0x7f71d41a1bf0 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009850 tx=0x7f71c00adc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:09:30.104 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.102+0000 7f71d1ffb640 1 -- 172.21.15.117:0/4086043450 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c00b2040 con 0x7f71d4103e70 2024-03-04T16:09:30.104 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.102+0000 7f71d1ffb640 1 -- 172.21.15.117:0/4086043450 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f71c00ade50 con 0x7f71d4103e70 2024-03-04T16:09:30.104 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.102+0000 7f71d1ffb640 1 -- 172.21.15.117:0/4086043450 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71c0003860 con 0x7f71d4103e70 2024-03-04T16:09:30.104 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.102+0000 7f71daf6c640 1 -- 172.21.15.117:0/4086043450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71d419c260 con 0x7f71d4103e70 2024-03-04T16:09:30.104 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.102+0000 7f71daf6c640 1 -- 172.21.15.117:0/4086043450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71d419c660 con 0x7f71d4103e70 2024-03-04T16:09:30.105 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.102+0000 7f71daf6c640 1 -- 172.21.15.117:0/4086043450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71d4104bd0 con 0x7f71d4103e70 2024-03-04T16:09:30.105 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.102+0000 7f71d1ffb640 1 -- 172.21.15.117:0/4086043450 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f71c00bd020 con 0x7f71d4103e70 2024-03-04T16:09:30.105 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.102+0000 7f71d1ffb640 1 --2- 172.21.15.117:0/4086043450 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f71c406d750 0x7f71c406fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:09:30.106 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.102+0000 7f71d1ffb640 1 -- 172.21.15.117:0/4086043450 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(775..775 src has 1..775) v4 ==== 9068+0+0 (secure 0 0 0) 0x7f71c0127540 con 0x7f71d4103e70 2024-03-04T16:09:30.106 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.102+0000 7f71d8d08640 1 --2- 172.21.15.117:0/4086043450 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f71c406d750 0x7f71c406fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:09:30.106 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.102+0000 7f71d8d08640 1 --2- 172.21.15.117:0/4086043450 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f71c406d750 0x7f71c406fc10 secure :-1 s=READY pgs=3230 cs=0 l=1 rev1=1 crypto rx=0x7f71c80058d0 tx=0x7f71c800b480 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:09:30.109 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.106+0000 7f71d1ffb640 1 -- 172.21.15.117:0/4086043450 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f71c00c4060 con 0x7f71d4103e70 2024-03-04T16:09:30.283 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.282+0000 7f71daf6c640 1 -- 172.21.15.117:0/4086043450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f71d41a8570 con 0x7f71d4103e70 2024-03-04T16:09:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:30 smithi151 bash[23756]: cluster 2024-03-04T16:09:29.281126+0000 mgr.y (mgr.24428) 754 : cluster 0 pgmap v1269: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:09:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:30 smithi151 bash[23756]: audit 2024-03-04T16:09:29.392178+0000 mon.a (mon.0) 3632 : audit 1 from='client.? 172.21.15.117:0/3439654890' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "100"}]': finished 2024-03-04T16:09:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:30 smithi151 bash[23756]: cluster 2024-03-04T16:09:29.396332+0000 mon.a (mon.0) 3633 : cluster 0 osdmap e774: 8 total, 8 up, 8 in 2024-03-04T16:09:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:30 smithi151 bash[23756]: audit 2024-03-04T16:09:29.500773+0000 mon.a (mon.0) 3634 : audit 1 from='client.? 172.21.15.117:0/3439654890' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "100"}]: dispatch 2024-03-04T16:09:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:30 smithi151 bash[23756]: cluster 2024-03-04T16:09:29.910675+0000 mon.a (mon.0) 3635 : cluster 1 pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' no longer out of quota; removing NO_QUOTA flag 2024-03-04T16:09:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:30 smithi151 bash[23756]: cluster 2024-03-04T16:09:29.911200+0000 mon.a (mon.0) 3636 : cluster 1 Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-04T16:09:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:30 smithi151 bash[23756]: audit 2024-03-04T16:09:29.913544+0000 mon.a (mon.0) 3637 : audit 1 from='client.? 172.21.15.117:0/3439654890' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "100"}]': finished 2024-03-04T16:09:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:30 smithi151 bash[23756]: cluster 2024-03-04T16:09:29.920418+0000 mon.a (mon.0) 3638 : cluster 0 osdmap e775: 8 total, 8 up, 8 in 2024-03-04T16:09:30.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:30 smithi151 bash[23756]: audit 2024-03-04T16:09:30.287198+0000 mon.a (mon.0) 3639 : audit 1 from='client.? 172.21.15.117:0/4086043450' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-04T16:09:30.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:30 smithi117 bash[21480]: cluster 2024-03-04T16:09:29.281126+0000 mgr.y (mgr.24428) 754 : cluster 0 pgmap v1269: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:09:30.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:30 smithi117 bash[21480]: audit 2024-03-04T16:09:29.392178+0000 mon.a (mon.0) 3632 : audit 1 from='client.? 172.21.15.117:0/3439654890' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "100"}]': finished 2024-03-04T16:09:30.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:30 smithi117 bash[21480]: cluster 2024-03-04T16:09:29.396332+0000 mon.a (mon.0) 3633 : cluster 0 osdmap e774: 8 total, 8 up, 8 in 2024-03-04T16:09:30.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:30 smithi117 bash[21480]: audit 2024-03-04T16:09:29.500773+0000 mon.a (mon.0) 3634 : audit 1 from='client.? 172.21.15.117:0/3439654890' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "100"}]: dispatch 2024-03-04T16:09:30.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:30 smithi117 bash[21480]: cluster 2024-03-04T16:09:29.910675+0000 mon.a (mon.0) 3635 : cluster 1 pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' no longer out of quota; removing NO_QUOTA flag 2024-03-04T16:09:30.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:30 smithi117 bash[21480]: cluster 2024-03-04T16:09:29.911200+0000 mon.a (mon.0) 3636 : cluster 1 Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-04T16:09:30.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:30 smithi117 bash[21480]: audit 2024-03-04T16:09:29.913544+0000 mon.a (mon.0) 3637 : audit 1 from='client.? 172.21.15.117:0/3439654890' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "100"}]': finished 2024-03-04T16:09:30.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:30 smithi117 bash[21480]: cluster 2024-03-04T16:09:29.920418+0000 mon.a (mon.0) 3638 : cluster 0 osdmap e775: 8 total, 8 up, 8 in 2024-03-04T16:09:30.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:30 smithi117 bash[21480]: audit 2024-03-04T16:09:30.287198+0000 mon.a (mon.0) 3639 : audit 1 from='client.? 172.21.15.117:0/4086043450' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-04T16:09:30.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:30 smithi117 bash[28843]: cluster 2024-03-04T16:09:29.281126+0000 mgr.y (mgr.24428) 754 : cluster 0 pgmap v1269: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:09:30.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:30 smithi117 bash[28843]: audit 2024-03-04T16:09:29.392178+0000 mon.a (mon.0) 3632 : audit 1 from='client.? 172.21.15.117:0/3439654890' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "100"}]': finished 2024-03-04T16:09:30.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:30 smithi117 bash[28843]: cluster 2024-03-04T16:09:29.396332+0000 mon.a (mon.0) 3633 : cluster 0 osdmap e774: 8 total, 8 up, 8 in 2024-03-04T16:09:30.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:30 smithi117 bash[28843]: audit 2024-03-04T16:09:29.500773+0000 mon.a (mon.0) 3634 : audit 1 from='client.? 172.21.15.117:0/3439654890' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "100"}]: dispatch 2024-03-04T16:09:30.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:30 smithi117 bash[28843]: cluster 2024-03-04T16:09:29.910675+0000 mon.a (mon.0) 3635 : cluster 1 pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' no longer out of quota; removing NO_QUOTA flag 2024-03-04T16:09:30.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:30 smithi117 bash[28843]: cluster 2024-03-04T16:09:29.911200+0000 mon.a (mon.0) 3636 : cluster 1 Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-04T16:09:30.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:30 smithi117 bash[28843]: audit 2024-03-04T16:09:29.913544+0000 mon.a (mon.0) 3637 : audit 1 from='client.? 172.21.15.117:0/3439654890' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "100"}]': finished 2024-03-04T16:09:30.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:30 smithi117 bash[28843]: cluster 2024-03-04T16:09:29.920418+0000 mon.a (mon.0) 3638 : cluster 0 osdmap e775: 8 total, 8 up, 8 in 2024-03-04T16:09:30.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:30 smithi117 bash[28843]: audit 2024-03-04T16:09:30.287198+0000 mon.a (mon.0) 3639 : audit 1 from='client.? 172.21.15.117:0/4086043450' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-04T16:09:30.913 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:30.910+0000 7f71d1ffb640 1 -- 172.21.15.117:0/4086043450 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 v776) v1 ==== 221+0+0 (secure 0 0 0) 0x7f71c00164f0 con 0x7f71d4103e70 2024-03-04T16:09:31.027 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:31.026+0000 7f71daf6c640 1 -- 172.21.15.117:0/4086043450 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f71d41a87c0 con 0x7f71d4103e70 2024-03-04T16:09:31.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:31 smithi117 bash[21480]: audit 2024-03-04T16:09:30.516803+0000 mon.a (mon.0) 3640 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:09:31.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:31 smithi117 bash[21480]: audit 2024-03-04T16:09:30.519760+0000 mon.c (mon.2) 831 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:09:31.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:31 smithi117 bash[21480]: audit 2024-03-04T16:09:30.916325+0000 mon.a (mon.0) 3641 : audit 1 from='client.? 172.21.15.117:0/4086043450' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "100"}]': finished 2024-03-04T16:09:31.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:31 smithi117 bash[21480]: cluster 2024-03-04T16:09:30.926097+0000 mon.a (mon.0) 3642 : cluster 0 osdmap e776: 8 total, 8 up, 8 in 2024-03-04T16:09:31.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:31 smithi117 bash[21480]: audit 2024-03-04T16:09:31.030879+0000 mon.a (mon.0) 3643 : audit 1 from='client.? 172.21.15.117:0/4086043450' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-04T16:09:31.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:31 smithi117 bash[21480]: cluster 2024-03-04T16:09:31.281881+0000 mgr.y (mgr.24428) 755 : cluster 0 pgmap v1273: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:09:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:31 smithi117 bash[28843]: audit 2024-03-04T16:09:30.516803+0000 mon.a (mon.0) 3640 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:09:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:31 smithi117 bash[28843]: audit 2024-03-04T16:09:30.519760+0000 mon.c (mon.2) 831 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:09:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:31 smithi117 bash[28843]: audit 2024-03-04T16:09:30.916325+0000 mon.a (mon.0) 3641 : audit 1 from='client.? 172.21.15.117:0/4086043450' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "100"}]': finished 2024-03-04T16:09:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:31 smithi117 bash[28843]: cluster 2024-03-04T16:09:30.926097+0000 mon.a (mon.0) 3642 : cluster 0 osdmap e776: 8 total, 8 up, 8 in 2024-03-04T16:09:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:31 smithi117 bash[28843]: audit 2024-03-04T16:09:31.030879+0000 mon.a (mon.0) 3643 : audit 1 from='client.? 172.21.15.117:0/4086043450' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-04T16:09:31.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:31 smithi117 bash[28843]: cluster 2024-03-04T16:09:31.281881+0000 mgr.y (mgr.24428) 755 : cluster 0 pgmap v1273: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:09:31.916 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:31.914+0000 7f71d1ffb640 1 -- 172.21.15.117:0/4086043450 <== mon.0 v2:172.21.15.117:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 v777) v1 ==== 221+0+0 (secure 0 0 0) 0x7f71c00f4850 con 0x7f71d4103e70 2024-03-04T16:09:31.916 INFO:tasks.workunit.client.0.smithi117.stderr:set-quota max_bytes = 100 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 2024-03-04T16:09:31.921 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:31.918+0000 7f71daf6c640 1 -- 172.21.15.117:0/4086043450 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f71c406d750 msgr2=0x7f71c406fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:09:31.921 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:31.918+0000 7f71daf6c640 1 --2- 172.21.15.117:0/4086043450 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f71c406d750 0x7f71c406fc10 secure :-1 s=READY pgs=3230 cs=0 l=1 rev1=1 crypto rx=0x7f71c80058d0 tx=0x7f71c800b480 comp rx=0 tx=0).stop 2024-03-04T16:09:31.921 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:31.918+0000 7f71daf6c640 1 -- 172.21.15.117:0/4086043450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d4103e70 msgr2=0x7f71d41a1bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:09:31.921 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:31.918+0000 7f71daf6c640 1 --2- 172.21.15.117:0/4086043450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d4103e70 0x7f71d41a1bf0 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f71c0009850 tx=0x7f71c00adc60 comp rx=0 tx=0).stop 2024-03-04T16:09:31.921 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:31.918+0000 7f71daf6c640 1 -- 172.21.15.117:0/4086043450 shutdown_connections 2024-03-04T16:09:31.921 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:31.918+0000 7f71daf6c640 1 --2- 172.21.15.117:0/4086043450 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f71c406d750 0x7f71c406fc10 unknown :-1 s=CLOSED pgs=3230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:09:31.922 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:31.918+0000 7f71daf6c640 1 --2- 172.21.15.117:0/4086043450 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f71d4103e70 0x7f71d41a1bf0 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:09:31.922 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:31.918+0000 7f71daf6c640 1 --2- 172.21.15.117:0/4086043450 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f71d4109560 0x7f71d419b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:09:31.922 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:31.918+0000 7f71daf6c640 1 --2- 172.21.15.117:0/4086043450 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f71d41034b0 0x7f71d41a16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:09:31.922 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:31.918+0000 7f71daf6c640 1 -- 172.21.15.117:0/4086043450 >> 172.21.15.117:0/4086043450 conn(0x7f71d40fef40 msgr2=0x7f71d40fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:09:31.922 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:31.918+0000 7f71daf6c640 1 -- 172.21.15.117:0/4086043450 shutdown_connections 2024-03-04T16:09:31.922 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:09:31.918+0000 7f71daf6c640 1 -- 172.21.15.117:0/4086043450 wait complete. 2024-03-04T16:09:31.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:31 smithi151 bash[23756]: audit 2024-03-04T16:09:30.516803+0000 mon.a (mon.0) 3640 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:09:31.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:31 smithi151 bash[23756]: audit 2024-03-04T16:09:30.519760+0000 mon.c (mon.2) 831 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:09:31.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:31 smithi151 bash[23756]: audit 2024-03-04T16:09:30.916325+0000 mon.a (mon.0) 3641 : audit 1 from='client.? 172.21.15.117:0/4086043450' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "100"}]': finished 2024-03-04T16:09:31.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:31 smithi151 bash[23756]: cluster 2024-03-04T16:09:30.926097+0000 mon.a (mon.0) 3642 : cluster 0 osdmap e776: 8 total, 8 up, 8 in 2024-03-04T16:09:31.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:31 smithi151 bash[23756]: audit 2024-03-04T16:09:31.030879+0000 mon.a (mon.0) 3643 : audit 1 from='client.? 172.21.15.117:0/4086043450' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "100"}]: dispatch 2024-03-04T16:09:31.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:31 smithi151 bash[23756]: cluster 2024-03-04T16:09:31.281881+0000 mgr.y (mgr.24428) 755 : cluster 0 pgmap v1273: 176 pgs: 176 active+clean; 699 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:09:31.942 INFO:tasks.workunit.client.0.smithi117.stderr:+ sleep 30 2024-03-04T16:09:33.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:32 smithi151 bash[23756]: audit 2024-03-04T16:09:31.918816+0000 mon.a (mon.0) 3644 : audit 1 from='client.? 172.21.15.117:0/4086043450' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "100"}]': finished 2024-03-04T16:09:33.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:32 smithi151 bash[23756]: cluster 2024-03-04T16:09:31.928545+0000 mon.a (mon.0) 3645 : cluster 0 osdmap e777: 8 total, 8 up, 8 in 2024-03-04T16:09:33.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:32 smithi117 bash[21480]: audit 2024-03-04T16:09:31.918816+0000 mon.a (mon.0) 3644 : audit 1 from='client.? 172.21.15.117:0/4086043450' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "100"}]': finished 2024-03-04T16:09:33.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:32 smithi117 bash[21480]: cluster 2024-03-04T16:09:31.928545+0000 mon.a (mon.0) 3645 : cluster 0 osdmap e777: 8 total, 8 up, 8 in 2024-03-04T16:09:33.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:32 smithi117 bash[28843]: audit 2024-03-04T16:09:31.918816+0000 mon.a (mon.0) 3644 : audit 1 from='client.? 172.21.15.117:0/4086043450' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "100"}]': finished 2024-03-04T16:09:33.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:32 smithi117 bash[28843]: cluster 2024-03-04T16:09:31.928545+0000 mon.a (mon.0) 3645 : cluster 0 osdmap e777: 8 total, 8 up, 8 in 2024-03-04T16:09:34.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:33 smithi151 bash[23756]: cluster 2024-03-04T16:09:33.282589+0000 mgr.y (mgr.24428) 756 : cluster 0 pgmap v1275: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-04T16:09:34.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:33 smithi117 bash[21480]: cluster 2024-03-04T16:09:33.282589+0000 mgr.y (mgr.24428) 756 : cluster 0 pgmap v1275: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-04T16:09:34.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:33 smithi117 bash[28843]: cluster 2024-03-04T16:09:33.282589+0000 mgr.y (mgr.24428) 756 : cluster 0 pgmap v1275: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-04T16:09:35.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:34 smithi117 bash[28843]: cluster 2024-03-04T16:09:34.913091+0000 mon.a (mon.0) 3646 : cluster 3 pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' is full (reached quota's max_bytes: 100 B) 2024-03-04T16:09:35.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:34 smithi117 bash[28843]: cluster 2024-03-04T16:09:34.913628+0000 mon.a (mon.0) 3647 : cluster 3 Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-04T16:09:35.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:34 smithi117 bash[28843]: cluster 2024-03-04T16:09:34.928115+0000 mon.a (mon.0) 3648 : cluster 0 osdmap e778: 8 total, 8 up, 8 in 2024-03-04T16:09:35.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:34 smithi117 bash[21480]: cluster 2024-03-04T16:09:34.913091+0000 mon.a (mon.0) 3646 : cluster 3 pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' is full (reached quota's max_bytes: 100 B) 2024-03-04T16:09:35.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:34 smithi117 bash[21480]: cluster 2024-03-04T16:09:34.913628+0000 mon.a (mon.0) 3647 : cluster 3 Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-04T16:09:35.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:34 smithi117 bash[21480]: cluster 2024-03-04T16:09:34.928115+0000 mon.a (mon.0) 3648 : cluster 0 osdmap e778: 8 total, 8 up, 8 in 2024-03-04T16:09:35.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:34 smithi151 bash[23756]: cluster 2024-03-04T16:09:34.913091+0000 mon.a (mon.0) 3646 : cluster 3 pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' is full (reached quota's max_bytes: 100 B) 2024-03-04T16:09:35.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:34 smithi151 bash[23756]: cluster 2024-03-04T16:09:34.913628+0000 mon.a (mon.0) 3647 : cluster 3 Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-04T16:09:35.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:34 smithi151 bash[23756]: cluster 2024-03-04T16:09:34.928115+0000 mon.a (mon.0) 3648 : cluster 0 osdmap e778: 8 total, 8 up, 8 in 2024-03-04T16:09:36.318 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:09:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:09:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:09:36.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:35 smithi117 bash[28843]: cluster 2024-03-04T16:09:35.283338+0000 mgr.y (mgr.24428) 757 : cluster 0 pgmap v1277: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 953 B/s rd, 4.5 KiB/s wr, 1 op/s 2024-03-04T16:09:36.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:35 smithi117 bash[21480]: cluster 2024-03-04T16:09:35.283338+0000 mgr.y (mgr.24428) 757 : cluster 0 pgmap v1277: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 953 B/s rd, 4.5 KiB/s wr, 1 op/s 2024-03-04T16:09:36.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:35 smithi151 bash[23756]: cluster 2024-03-04T16:09:35.283338+0000 mgr.y (mgr.24428) 757 : cluster 0 pgmap v1277: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 953 B/s rd, 4.5 KiB/s wr, 1 op/s 2024-03-04T16:09:37.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:09:37 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:09:38.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:38 smithi151 bash[23756]: cluster 2024-03-04T16:09:37.285340+0000 mgr.y (mgr.24428) 758 : cluster 0 pgmap v1278: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 3.8 KiB/s wr, 1 op/s 2024-03-04T16:09:38.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:38 smithi151 bash[23756]: audit 2024-03-04T16:09:37.325985+0000 mgr.y (mgr.24428) 759 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:09:38.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:38 smithi117 bash[21480]: cluster 2024-03-04T16:09:37.285340+0000 mgr.y (mgr.24428) 758 : cluster 0 pgmap v1278: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 3.8 KiB/s wr, 1 op/s 2024-03-04T16:09:38.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:38 smithi117 bash[21480]: audit 2024-03-04T16:09:37.325985+0000 mgr.y (mgr.24428) 759 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:09:38.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:38 smithi117 bash[28843]: cluster 2024-03-04T16:09:37.285340+0000 mgr.y (mgr.24428) 758 : cluster 0 pgmap v1278: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.6 KiB/s rd, 3.8 KiB/s wr, 1 op/s 2024-03-04T16:09:38.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:38 smithi117 bash[28843]: audit 2024-03-04T16:09:37.325985+0000 mgr.y (mgr.24428) 759 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:09:40.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:40 smithi151 bash[23756]: cluster 2024-03-04T16:09:39.286054+0000 mgr.y (mgr.24428) 760 : cluster 0 pgmap v1279: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-04T16:09:40.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:40 smithi117 bash[21480]: cluster 2024-03-04T16:09:39.286054+0000 mgr.y (mgr.24428) 760 : cluster 0 pgmap v1279: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-04T16:09:40.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:40 smithi117 bash[28843]: cluster 2024-03-04T16:09:39.286054+0000 mgr.y (mgr.24428) 760 : cluster 0 pgmap v1279: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-04T16:09:42.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:42 smithi151 bash[23756]: cluster 2024-03-04T16:09:41.286844+0000 mgr.y (mgr.24428) 761 : cluster 0 pgmap v1280: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 546 B/s rd, 0 op/s 2024-03-04T16:09:42.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:42 smithi117 bash[21480]: cluster 2024-03-04T16:09:41.286844+0000 mgr.y (mgr.24428) 761 : cluster 0 pgmap v1280: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 546 B/s rd, 0 op/s 2024-03-04T16:09:42.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:42 smithi117 bash[28843]: cluster 2024-03-04T16:09:41.286844+0000 mgr.y (mgr.24428) 761 : cluster 0 pgmap v1280: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 546 B/s rd, 0 op/s 2024-03-04T16:09:44.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:44 smithi151 bash[23756]: cluster 2024-03-04T16:09:43.288765+0000 mgr.y (mgr.24428) 762 : cluster 0 pgmap v1281: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:09:44.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:44 smithi117 bash[21480]: cluster 2024-03-04T16:09:43.288765+0000 mgr.y (mgr.24428) 762 : cluster 0 pgmap v1281: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:09:44.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:44 smithi117 bash[28843]: cluster 2024-03-04T16:09:43.288765+0000 mgr.y (mgr.24428) 762 : cluster 0 pgmap v1281: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:09:46.355 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:09:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:09:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:09:46.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:46 smithi151 bash[23756]: cluster 2024-03-04T16:09:45.289583+0000 mgr.y (mgr.24428) 763 : cluster 0 pgmap v1282: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:09:46.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:46 smithi151 bash[23756]: audit 2024-03-04T16:09:45.560972+0000 mon.a (mon.0) 3649 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:09:46.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:46 smithi151 bash[23756]: audit 2024-03-04T16:09:45.565570+0000 mon.c (mon.2) 832 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:09:46.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:46 smithi117 bash[28843]: cluster 2024-03-04T16:09:45.289583+0000 mgr.y (mgr.24428) 763 : cluster 0 pgmap v1282: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:09:46.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:46 smithi117 bash[28843]: audit 2024-03-04T16:09:45.560972+0000 mon.a (mon.0) 3649 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:09:46.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:46 smithi117 bash[28843]: audit 2024-03-04T16:09:45.565570+0000 mon.c (mon.2) 832 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:09:46.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:46 smithi117 bash[21480]: cluster 2024-03-04T16:09:45.289583+0000 mgr.y (mgr.24428) 763 : cluster 0 pgmap v1282: 176 pgs: 176 active+clean; 722 KiB data, 794 MiB used, 714 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-03-04T16:09:46.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:46 smithi117 bash[21480]: audit 2024-03-04T16:09:45.560972+0000 mon.a (mon.0) 3649 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:09:46.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:46 smithi117 bash[21480]: audit 2024-03-04T16:09:45.565570+0000 mon.c (mon.2) 832 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:09:47.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:09:47 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:09:48.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:48 smithi151 bash[23756]: cluster 2024-03-04T16:09:47.291492+0000 mgr.y (mgr.24428) 764 : cluster 0 pgmap v1283: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-04T16:09:48.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:48 smithi151 bash[23756]: audit 2024-03-04T16:09:47.329495+0000 mgr.y (mgr.24428) 765 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:09:48.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:48 smithi117 bash[21480]: cluster 2024-03-04T16:09:47.291492+0000 mgr.y (mgr.24428) 764 : cluster 0 pgmap v1283: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-04T16:09:48.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:48 smithi117 bash[21480]: audit 2024-03-04T16:09:47.329495+0000 mgr.y (mgr.24428) 765 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:09:48.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:48 smithi117 bash[28843]: cluster 2024-03-04T16:09:47.291492+0000 mgr.y (mgr.24428) 764 : cluster 0 pgmap v1283: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-04T16:09:48.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:48 smithi117 bash[28843]: audit 2024-03-04T16:09:47.329495+0000 mgr.y (mgr.24428) 765 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:09:50.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:50 smithi151 bash[23756]: cluster 2024-03-04T16:09:49.292270+0000 mgr.y (mgr.24428) 766 : cluster 0 pgmap v1284: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-04T16:09:50.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:50 smithi117 bash[21480]: cluster 2024-03-04T16:09:49.292270+0000 mgr.y (mgr.24428) 766 : cluster 0 pgmap v1284: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-04T16:09:50.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:50 smithi117 bash[28843]: cluster 2024-03-04T16:09:49.292270+0000 mgr.y (mgr.24428) 766 : cluster 0 pgmap v1284: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-04T16:09:52.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:52 smithi151 bash[23756]: cluster 2024-03-04T16:09:51.293081+0000 mgr.y (mgr.24428) 767 : cluster 0 pgmap v1285: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-04T16:09:52.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:52 smithi117 bash[21480]: cluster 2024-03-04T16:09:51.293081+0000 mgr.y (mgr.24428) 767 : cluster 0 pgmap v1285: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-04T16:09:52.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:52 smithi117 bash[28843]: cluster 2024-03-04T16:09:51.293081+0000 mgr.y (mgr.24428) 767 : cluster 0 pgmap v1285: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-04T16:09:54.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:54 smithi151 bash[23756]: cluster 2024-03-04T16:09:53.294996+0000 mgr.y (mgr.24428) 768 : cluster 0 pgmap v1286: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-04T16:09:54.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:54 smithi117 bash[21480]: cluster 2024-03-04T16:09:53.294996+0000 mgr.y (mgr.24428) 768 : cluster 0 pgmap v1286: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-04T16:09:54.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:54 smithi117 bash[28843]: cluster 2024-03-04T16:09:53.294996+0000 mgr.y (mgr.24428) 768 : cluster 0 pgmap v1286: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-04T16:09:56.378 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:09:56 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:09:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:09:56.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:56 smithi151 bash[23756]: cluster 2024-03-04T16:09:55.295786+0000 mgr.y (mgr.24428) 769 : cluster 0 pgmap v1287: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-04T16:09:56.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:56 smithi151 bash[23756]: audit 2024-03-04T16:09:55.587111+0000 mon.c (mon.2) 833 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:09:56.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:56 smithi151 bash[23756]: audit 2024-03-04T16:09:56.269916+0000 mon.c (mon.2) 834 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:09:56.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:56 smithi151 bash[23756]: audit 2024-03-04T16:09:56.271390+0000 mon.c (mon.2) 835 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:09:56.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:56 smithi151 bash[23756]: audit 2024-03-04T16:09:56.280206+0000 mon.a (mon.0) 3650 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:09:56.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:56 smithi117 bash[21480]: cluster 2024-03-04T16:09:55.295786+0000 mgr.y (mgr.24428) 769 : cluster 0 pgmap v1287: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-04T16:09:56.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:56 smithi117 bash[21480]: audit 2024-03-04T16:09:55.587111+0000 mon.c (mon.2) 833 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:09:56.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:56 smithi117 bash[21480]: audit 2024-03-04T16:09:56.269916+0000 mon.c (mon.2) 834 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:09:56.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:56 smithi117 bash[21480]: audit 2024-03-04T16:09:56.271390+0000 mon.c (mon.2) 835 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:09:56.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:56 smithi117 bash[21480]: audit 2024-03-04T16:09:56.280206+0000 mon.a (mon.0) 3650 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:09:56.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:56 smithi117 bash[28843]: cluster 2024-03-04T16:09:55.295786+0000 mgr.y (mgr.24428) 769 : cluster 0 pgmap v1287: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 36 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-04T16:09:56.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:56 smithi117 bash[28843]: audit 2024-03-04T16:09:55.587111+0000 mon.c (mon.2) 833 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:09:56.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:56 smithi117 bash[28843]: audit 2024-03-04T16:09:56.269916+0000 mon.c (mon.2) 834 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:09:56.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:56 smithi117 bash[28843]: audit 2024-03-04T16:09:56.271390+0000 mon.c (mon.2) 835 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:09:56.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:56 smithi117 bash[28843]: audit 2024-03-04T16:09:56.280206+0000 mon.a (mon.0) 3650 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:09:57.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:09:57 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:09:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:58 smithi151 bash[23756]: cluster 2024-03-04T16:09:57.297693+0000 mgr.y (mgr.24428) 770 : cluster 0 pgmap v1288: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-04T16:09:58.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:09:58 smithi151 bash[23756]: audit 2024-03-04T16:09:57.340914+0000 mgr.y (mgr.24428) 771 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:09:58.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:58 smithi117 bash[28843]: cluster 2024-03-04T16:09:57.297693+0000 mgr.y (mgr.24428) 770 : cluster 0 pgmap v1288: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-04T16:09:58.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:09:58 smithi117 bash[28843]: audit 2024-03-04T16:09:57.340914+0000 mgr.y (mgr.24428) 771 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:09:58.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:58 smithi117 bash[21480]: cluster 2024-03-04T16:09:57.297693+0000 mgr.y (mgr.24428) 770 : cluster 0 pgmap v1288: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 37 KiB/s rd, 0 B/s wr, 60 op/s 2024-03-04T16:09:58.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:09:58 smithi117 bash[21480]: audit 2024-03-04T16:09:57.340914+0000 mgr.y (mgr.24428) 771 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:10:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:00 smithi151 bash[23756]: cluster 2024-03-04T16:09:59.298485+0000 mgr.y (mgr.24428) 772 : cluster 0 pgmap v1289: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:00 smithi151 bash[23756]: cluster 2024-03-04T16:10:00.000196+0000 mon.a (mon.0) 3651 : cluster 3 Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-03-04T16:10:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:00 smithi151 bash[23756]: cluster 2024-03-04T16:10:00.000265+0000 mon.a (mon.0) 3652 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-03-04T16:10:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:00 smithi151 bash[23756]: cluster 2024-03-04T16:10:00.000307+0000 mon.a (mon.0) 3653 : cluster 3 application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-04T16:10:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:00 smithi151 bash[23756]: cluster 2024-03-04T16:10:00.000352+0000 mon.a (mon.0) 3654 : cluster 3 use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-04T16:10:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:00 smithi151 bash[23756]: cluster 2024-03-04T16:10:00.000414+0000 mon.a (mon.0) 3655 : cluster 3 [WRN] POOL_FULL: 1 pool(s) full 2024-03-04T16:10:00.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:00 smithi151 bash[23756]: cluster 2024-03-04T16:10:00.000455+0000 mon.a (mon.0) 3656 : cluster 3 pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' is full (running out of quota) 2024-03-04T16:10:00.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:00 smithi117 bash[21480]: cluster 2024-03-04T16:09:59.298485+0000 mgr.y (mgr.24428) 772 : cluster 0 pgmap v1289: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:00.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:00 smithi117 bash[21480]: cluster 2024-03-04T16:10:00.000196+0000 mon.a (mon.0) 3651 : cluster 3 Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-03-04T16:10:00.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:00 smithi117 bash[21480]: cluster 2024-03-04T16:10:00.000265+0000 mon.a (mon.0) 3652 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-03-04T16:10:00.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:00 smithi117 bash[21480]: cluster 2024-03-04T16:10:00.000307+0000 mon.a (mon.0) 3653 : cluster 3 application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-04T16:10:00.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:00 smithi117 bash[21480]: cluster 2024-03-04T16:10:00.000352+0000 mon.a (mon.0) 3654 : cluster 3 use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-04T16:10:00.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:00 smithi117 bash[21480]: cluster 2024-03-04T16:10:00.000414+0000 mon.a (mon.0) 3655 : cluster 3 [WRN] POOL_FULL: 1 pool(s) full 2024-03-04T16:10:00.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:00 smithi117 bash[21480]: cluster 2024-03-04T16:10:00.000455+0000 mon.a (mon.0) 3656 : cluster 3 pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' is full (running out of quota) 2024-03-04T16:10:00.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:00 smithi117 bash[28843]: cluster 2024-03-04T16:09:59.298485+0000 mgr.y (mgr.24428) 772 : cluster 0 pgmap v1289: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:00.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:00 smithi117 bash[28843]: cluster 2024-03-04T16:10:00.000196+0000 mon.a (mon.0) 3651 : cluster 3 Health detail: HEALTH_WARN 1 pool(s) do not have an application enabled; 1 pool(s) full 2024-03-04T16:10:00.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:00 smithi117 bash[28843]: cluster 2024-03-04T16:10:00.000265+0000 mon.a (mon.0) 3652 : cluster 3 [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-03-04T16:10:00.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:00 smithi117 bash[28843]: cluster 2024-03-04T16:10:00.000307+0000 mon.a (mon.0) 3653 : cluster 3 application not enabled on pool 'ceph_test_rados_api_asio' 2024-03-04T16:10:00.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:00 smithi117 bash[28843]: cluster 2024-03-04T16:10:00.000352+0000 mon.a (mon.0) 3654 : cluster 3 use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-03-04T16:10:00.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:00 smithi117 bash[28843]: cluster 2024-03-04T16:10:00.000414+0000 mon.a (mon.0) 3655 : cluster 3 [WRN] POOL_FULL: 1 pool(s) full 2024-03-04T16:10:00.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:00 smithi117 bash[28843]: cluster 2024-03-04T16:10:00.000455+0000 mon.a (mon.0) 3656 : cluster 3 pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' is full (running out of quota) 2024-03-04T16:10:01.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:01 smithi151 bash[23756]: audit 2024-03-04T16:10:00.598188+0000 mon.c (mon.2) 836 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:10:01.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:01 smithi117 bash[21480]: audit 2024-03-04T16:10:00.598188+0000 mon.c (mon.2) 836 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:10:01.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:01 smithi117 bash[28843]: audit 2024-03-04T16:10:00.598188+0000 mon.c (mon.2) 836 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:10:01.944 INFO:tasks.workunit.client.0.smithi117.stderr:+ pid=132100 2024-03-04T16:10:01.944 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph osd pool set-quota 736f5599-4143-4934-af9d-08c0b0a4b1a3 max_bytes 0 2024-03-04T16:10:01.944 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p 736f5599-4143-4934-af9d-08c0b0a4b1a3 put two /etc/passwd 2024-03-04T16:10:02.076 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f804938e640 1 -- 172.21.15.117:0/3095578620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8044100d80 msgr2=0x7f8044104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:02.076 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f804938e640 1 --2- 172.21.15.117:0/3095578620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8044100d80 0x7f8044104dd0 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f802c009930 tx=0x7f802c01c7e0 comp rx=0 tx=0).stop 2024-03-04T16:10:02.076 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f804938e640 1 -- 172.21.15.117:0/3095578620 shutdown_connections 2024-03-04T16:10:02.077 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f804938e640 1 --2- 172.21.15.117:0/3095578620 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8044100d80 0x7f8044104dd0 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:02.077 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f804938e640 1 --2- 172.21.15.117:0/3095578620 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8044105500 0x7f804410d080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:02.077 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f804938e640 1 --2- 172.21.15.117:0/3095578620 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f80441003d0 0x7f80441007b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:02.077 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f804938e640 1 -- 172.21.15.117:0/3095578620 >> 172.21.15.117:0/3095578620 conn(0x7f80440fbc60 msgr2=0x7f80440fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:10:02.077 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f804938e640 1 -- 172.21.15.117:0/3095578620 shutdown_connections 2024-03-04T16:10:02.077 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f804938e640 1 -- 172.21.15.117:0/3095578620 wait complete. 2024-03-04T16:10:02.077 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f804938e640 1 Processor -- start 2024-03-04T16:10:02.077 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f804938e640 1 -- start start 2024-03-04T16:10:02.077 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f804938e640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80441003d0 0x7f80440fc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:10:02.077 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f804938e640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8044100d80 0x7f80440fc830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:10:02.077 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f804938e640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8044105500 0x7f80440fcd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:10:02.077 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f804938e640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044112d80 con 0x7f8044105500 2024-03-04T16:10:02.077 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f804938e640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044112c00 con 0x7f80441003d0 2024-03-04T16:10:02.077 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f804938e640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8044112f00 con 0x7f8044100d80 2024-03-04T16:10:02.078 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f80427fc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8044100d80 0x7f80440fc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:10:02.078 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f80437fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8044105500 0x7f80440fcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:10:02.078 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f80437fe640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8044105500 0x7f80440fcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38426/0 (socket says 172.21.15.117:38426) 2024-03-04T16:10:02.078 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f80427fc640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8044100d80 0x7f80440fc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:54942/0 (socket says 172.21.15.117:54942) 2024-03-04T16:10:02.078 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f80437fe640 1 -- 172.21.15.117:0/2745215032 learned_addr learned my addr 172.21.15.117:0/2745215032 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T16:10:02.078 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f8042ffd640 1 --2- 172.21.15.117:0/2745215032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80441003d0 0x7f80440fc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:10:02.078 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f80427fc640 1 -- 172.21.15.117:0/2745215032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80441003d0 msgr2=0x7f80440fc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:02.078 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f80427fc640 1 --2- 172.21.15.117:0/2745215032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80441003d0 0x7f80440fc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:02.078 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f80427fc640 1 -- 172.21.15.117:0/2745215032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8044105500 msgr2=0x7f80440fcd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:02.078 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f80427fc640 1 --2- 172.21.15.117:0/2745215032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8044105500 0x7f80440fcd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:02.078 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f80427fc640 1 -- 172.21.15.117:0/2745215032 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80441a8a10 con 0x7f8044100d80 2024-03-04T16:10:02.078 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f80437fe640 1 --2- 172.21.15.117:0/2745215032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8044105500 0x7f80440fcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T16:10:02.078 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f8042ffd640 1 --2- 172.21.15.117:0/2745215032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80441003d0 0x7f80440fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T16:10:02.078 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f80427fc640 1 --2- 172.21.15.117:0/2745215032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8044100d80 0x7f80440fc830 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 crypto rx=0x7f802c0b0040 tx=0x7f802c0ad6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:10:02.078 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f803bfff640 1 -- 172.21.15.117:0/2745215032 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c0b2040 con 0x7f8044100d80 2024-03-04T16:10:02.079 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f804938e640 1 -- 172.21.15.117:0/2745215032 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80441a8ca0 con 0x7f8044100d80 2024-03-04T16:10:02.079 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f803bfff640 1 -- 172.21.15.117:0/2745215032 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f802c0a5a60 con 0x7f8044100d80 2024-03-04T16:10:02.079 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f803bfff640 1 -- 172.21.15.117:0/2745215032 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f802c005360 con 0x7f8044100d80 2024-03-04T16:10:02.079 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f804938e640 1 -- 172.21.15.117:0/2745215032 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80441a9140 con 0x7f8044100d80 2024-03-04T16:10:02.080 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.073+0000 7f804938e640 1 -- 172.21.15.117:0/2745215032 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80440681e0 con 0x7f8044100d80 2024-03-04T16:10:02.080 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.077+0000 7f803bfff640 1 -- 172.21.15.117:0/2745215032 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f802c0bd060 con 0x7f8044100d80 2024-03-04T16:10:02.080 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.077+0000 7f803bfff640 1 --2- 172.21.15.117:0/2745215032 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f803006d700 0x7f803006fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:10:02.080 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.077+0000 7f803bfff640 1 -- 172.21.15.117:0/2745215032 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(778..778 src has 1..778) v4 ==== 9068+0+0 (secure 0 0 0) 0x7f802c127330 con 0x7f8044100d80 2024-03-04T16:10:02.080 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.077+0000 7f8042ffd640 1 --2- 172.21.15.117:0/2745215032 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f803006d700 0x7f803006fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:10:02.081 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.077+0000 7f803bfff640 1 -- 172.21.15.117:0/2745215032 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=779}) v3 -- 0x7f8030079190 con 0x7f8044100d80 2024-03-04T16:10:02.081 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.077+0000 7f8042ffd640 1 --2- 172.21.15.117:0/2745215032 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f803006d700 0x7f803006fbc0 secure :-1 s=READY pgs=3232 cs=0 l=1 rev1=1 crypto rx=0x7f803c009f50 tx=0x7f803c009500 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:10:02.083 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.081+0000 7f803bfff640 1 -- 172.21.15.117:0/2745215032 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f802c0164f0 con 0x7f8044100d80 2024-03-04T16:10:02.256 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.253+0000 7f804938e640 1 -- 172.21.15.117:0/2745215032 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f80441a93d0 con 0x7f8044100d80 2024-03-04T16:10:02.409 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.401+0000 7f803bfff640 1 -- 172.21.15.117:0/2745215032 <== mon.2 v2:172.21.15.117:3301/0 7 ==== osd_map(779..779 src has 1..779) v4 ==== 628+0+0 (secure 0 0 0) 0x7f802c15ad20 con 0x7f8044100d80 2024-03-04T16:10:02.409 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.401+0000 7f803bfff640 1 -- 172.21.15.117:0/2745215032 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=780}) v3 -- 0x7f8030079d00 con 0x7f8044100d80 2024-03-04T16:10:02.412 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.409+0000 7f803bfff640 1 -- 172.21.15.117:0/2745215032 <== mon.2 v2:172.21.15.117:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 v779) v1 ==== 217+0+0 (secure 0 0 0) 0x7f802c0f45e0 con 0x7f8044100d80 2024-03-04T16:10:02.514 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:02.513+0000 7f804938e640 1 -- 172.21.15.117:0/2745215032 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f8044061900 con 0x7f8044100d80 2024-03-04T16:10:02.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:02 smithi151 bash[23756]: cluster 2024-03-04T16:10:01.299266+0000 mgr.y (mgr.24428) 773 : cluster 0 pgmap v1290: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:02 smithi151 bash[23756]: audit 2024-03-04T16:10:02.260126+0000 mon.c (mon.2) 837 : audit 1 from='client.? 172.21.15.117:0/2745215032' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:10:02.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:02 smithi151 bash[23756]: audit 2024-03-04T16:10:02.260698+0000 mon.a (mon.0) 3657 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:10:02.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:02 smithi117 bash[21480]: cluster 2024-03-04T16:10:01.299266+0000 mgr.y (mgr.24428) 773 : cluster 0 pgmap v1290: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:02.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:02 smithi117 bash[21480]: audit 2024-03-04T16:10:02.260126+0000 mon.c (mon.2) 837 : audit 1 from='client.? 172.21.15.117:0/2745215032' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:10:02.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:02 smithi117 bash[21480]: audit 2024-03-04T16:10:02.260698+0000 mon.a (mon.0) 3657 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:10:02.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:02 smithi117 bash[28843]: cluster 2024-03-04T16:10:01.299266+0000 mgr.y (mgr.24428) 773 : cluster 0 pgmap v1290: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:02.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:02 smithi117 bash[28843]: audit 2024-03-04T16:10:02.260126+0000 mon.c (mon.2) 837 : audit 1 from='client.? 172.21.15.117:0/2745215032' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:10:02.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:02 smithi117 bash[28843]: audit 2024-03-04T16:10:02.260698+0000 mon.a (mon.0) 3657 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:10:03.431 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.429+0000 7f803bfff640 1 -- 172.21.15.117:0/2745215032 <== mon.2 v2:172.21.15.117:3301/0 9 ==== osd_map(780..780 src has 1..780) v4 ==== 628+0+0 (secure 0 0 0) 0x7f802c124ca0 con 0x7f8044100d80 2024-03-04T16:10:03.432 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.429+0000 7f803bfff640 1 -- 172.21.15.117:0/2745215032 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=781}) v3 -- 0x7f803007a0f0 con 0x7f8044100d80 2024-03-04T16:10:03.438 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.437+0000 7f803bfff640 1 -- 172.21.15.117:0/2745215032 <== mon.2 v2:172.21.15.117:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 v780) v1 ==== 217+0+0 (secure 0 0 0) 0x7f802c127150 con 0x7f8044100d80 2024-03-04T16:10:03.438 INFO:tasks.workunit.client.0.smithi117.stderr:set-quota max_bytes = 0 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 2024-03-04T16:10:03.442 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.441+0000 7f804938e640 1 -- 172.21.15.117:0/2745215032 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f803006d700 msgr2=0x7f803006fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:03.442 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.441+0000 7f804938e640 1 --2- 172.21.15.117:0/2745215032 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f803006d700 0x7f803006fbc0 secure :-1 s=READY pgs=3232 cs=0 l=1 rev1=1 crypto rx=0x7f803c009f50 tx=0x7f803c009500 comp rx=0 tx=0).stop 2024-03-04T16:10:03.442 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.441+0000 7f804938e640 1 -- 172.21.15.117:0/2745215032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8044100d80 msgr2=0x7f80440fc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:03.442 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.441+0000 7f804938e640 1 --2- 172.21.15.117:0/2745215032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8044100d80 0x7f80440fc830 secure :-1 s=READY pgs=3125 cs=0 l=1 rev1=1 crypto rx=0x7f802c0b0040 tx=0x7f802c0ad6f0 comp rx=0 tx=0).stop 2024-03-04T16:10:03.443 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.441+0000 7f804938e640 1 -- 172.21.15.117:0/2745215032 shutdown_connections 2024-03-04T16:10:03.443 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.441+0000 7f804938e640 1 --2- 172.21.15.117:0/2745215032 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f803006d700 0x7f803006fbc0 unknown :-1 s=CLOSED pgs=3232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:03.443 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.441+0000 7f804938e640 1 --2- 172.21.15.117:0/2745215032 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8044105500 0x7f80440fcd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:03.443 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.441+0000 7f804938e640 1 --2- 172.21.15.117:0/2745215032 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8044100d80 0x7f80440fc830 unknown :-1 s=CLOSED pgs=3125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:03.443 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.441+0000 7f804938e640 1 --2- 172.21.15.117:0/2745215032 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f80441003d0 0x7f80440fc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:03.444 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.441+0000 7f804938e640 1 -- 172.21.15.117:0/2745215032 >> 172.21.15.117:0/2745215032 conn(0x7f80440fbc60 msgr2=0x7f8044068630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:10:03.444 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.441+0000 7f804938e640 1 -- 172.21.15.117:0/2745215032 shutdown_connections 2024-03-04T16:10:03.444 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.441+0000 7f804938e640 1 -- 172.21.15.117:0/2745215032 wait complete. 2024-03-04T16:10:03.461 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph osd pool set-quota 736f5599-4143-4934-af9d-08c0b0a4b1a3 max_objects 0 2024-03-04T16:10:03.585 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2d138640 1 -- 172.21.15.117:0/3526487442 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca281034b0 msgr2=0x7fca28103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:03.586 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2d138640 1 --2- 172.21.15.117:0/3526487442 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca281034b0 0x7fca28103930 secure :-1 s=READY pgs=3126 cs=0 l=1 rev1=1 crypto rx=0x7fca18009960 tx=0x7fca1801c7c0 comp rx=0 tx=0).stop 2024-03-04T16:10:03.586 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2d138640 1 -- 172.21.15.117:0/3526487442 shutdown_connections 2024-03-04T16:10:03.586 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2d138640 1 --2- 172.21.15.117:0/3526487442 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca28103e70 0x7fca28110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:03.586 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2d138640 1 --2- 172.21.15.117:0/3526487442 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca281034b0 0x7fca28103930 unknown :-1 s=CLOSED pgs=3126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:03.586 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2d138640 1 --2- 172.21.15.117:0/3526487442 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca28109560 0x7fca28109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:03.586 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2d138640 1 -- 172.21.15.117:0/3526487442 >> 172.21.15.117:0/3526487442 conn(0x7fca280fef40 msgr2=0x7fca28101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:10:03.586 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2d138640 1 -- 172.21.15.117:0/3526487442 shutdown_connections 2024-03-04T16:10:03.586 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2d138640 1 -- 172.21.15.117:0/3526487442 wait complete. 2024-03-04T16:10:03.586 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2d138640 1 Processor -- start 2024-03-04T16:10:03.586 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2d138640 1 -- start start 2024-03-04T16:10:03.586 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2d138640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca281034b0 0x7fca281a1760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:10:03.586 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2d138640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca28103e70 0x7fca281a1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:10:03.587 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2d138640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca28109560 0x7fca2819b920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:10:03.587 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2d138640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca28115550 con 0x7fca281034b0 2024-03-04T16:10:03.587 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2d138640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca281153d0 con 0x7fca28109560 2024-03-04T16:10:03.587 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2d138640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca281156d0 con 0x7fca28103e70 2024-03-04T16:10:03.587 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca26d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca281034b0 0x7fca281a1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:10:03.587 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2659c640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca28103e70 0x7fca281a1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:10:03.588 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca26d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca281034b0 0x7fca281a1760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38436/0 (socket says 172.21.15.117:38436) 2024-03-04T16:10:03.588 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca26d9d640 1 -- 172.21.15.117:0/1684284265 learned_addr learned my addr 172.21.15.117:0/1684284265 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T16:10:03.588 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2659c640 1 -- 172.21.15.117:0/1684284265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca28109560 msgr2=0x7fca2819b920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:03.588 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2759e640 1 --2- 172.21.15.117:0/1684284265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca28109560 0x7fca2819b920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:10:03.588 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2659c640 1 --2- 172.21.15.117:0/1684284265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca28109560 0x7fca2819b920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:03.588 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2659c640 1 -- 172.21.15.117:0/1684284265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca281034b0 msgr2=0x7fca281a1760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:03.588 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2659c640 1 --2- 172.21.15.117:0/1684284265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca281034b0 0x7fca281a1760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:03.588 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2659c640 1 -- 172.21.15.117:0/1684284265 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2819c050 con 0x7fca28103e70 2024-03-04T16:10:03.588 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca26d9d640 1 --2- 172.21.15.117:0/1684284265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca281034b0 0x7fca281a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T16:10:03.588 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2659c640 1 --2- 172.21.15.117:0/1684284265 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca28103e70 0x7fca281a1ca0 secure :-1 s=READY pgs=3127 cs=0 l=1 rev1=1 crypto rx=0x7fca1801c5f0 tx=0x7fca180adcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:10:03.588 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca1ffff640 1 -- 172.21.15.117:0/1684284265 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca180bc020 con 0x7fca28103e70 2024-03-04T16:10:03.588 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca1ffff640 1 -- 172.21.15.117:0/1684284265 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fca180044c0 con 0x7fca28103e70 2024-03-04T16:10:03.588 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2d138640 1 -- 172.21.15.117:0/1684284265 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2819c2e0 con 0x7fca28103e70 2024-03-04T16:10:03.588 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca1ffff640 1 -- 172.21.15.117:0/1684284265 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca180b2040 con 0x7fca28103e70 2024-03-04T16:10:03.589 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.581+0000 7fca2d138640 1 -- 172.21.15.117:0/1684284265 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca281a8630 con 0x7fca28103e70 2024-03-04T16:10:03.589 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.585+0000 7fca2d138640 1 -- 172.21.15.117:0/1684284265 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca280681e0 con 0x7fca28103e70 2024-03-04T16:10:03.590 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.585+0000 7fca1ffff640 1 -- 172.21.15.117:0/1684284265 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7fca180bf070 con 0x7fca28103e70 2024-03-04T16:10:03.590 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.585+0000 7fca1ffff640 1 --2- 172.21.15.117:0/1684284265 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fca1406d700 0x7fca1406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:10:03.590 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.585+0000 7fca1ffff640 1 -- 172.21.15.117:0/1684284265 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(780..780 src has 1..780) v4 ==== 9068+0+0 (secure 0 0 0) 0x7fca18127a60 con 0x7fca28103e70 2024-03-04T16:10:03.590 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.585+0000 7fca26d9d640 1 --2- 172.21.15.117:0/1684284265 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fca1406d700 0x7fca1406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:10:03.590 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.585+0000 7fca1ffff640 1 -- 172.21.15.117:0/1684284265 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=781}) v3 -- 0x7fca140791f0 con 0x7fca28103e70 2024-03-04T16:10:03.590 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.585+0000 7fca26d9d640 1 --2- 172.21.15.117:0/1684284265 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fca1406d700 0x7fca1406fbc0 secure :-1 s=READY pgs=3233 cs=0 l=1 rev1=1 crypto rx=0x7fca2819ca60 tx=0x7fca100091e0 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:10:03.591 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.589+0000 7fca1ffff640 1 -- 172.21.15.117:0/1684284265 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fca180164d0 con 0x7fca28103e70 2024-03-04T16:10:03.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:03 smithi151 bash[23756]: audit 2024-03-04T16:10:02.403925+0000 mon.a (mon.0) 3658 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]': finished 2024-03-04T16:10:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:03 smithi151 bash[23756]: cluster 2024-03-04T16:10:02.411563+0000 mon.a (mon.0) 3659 : cluster 0 osdmap e779: 8 total, 8 up, 8 in 2024-03-04T16:10:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:03 smithi151 bash[23756]: audit 2024-03-04T16:10:02.517988+0000 mon.c (mon.2) 838 : audit 1 from='client.? 172.21.15.117:0/2745215032' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:10:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:03 smithi151 bash[23756]: audit 2024-03-04T16:10:02.518523+0000 mon.a (mon.0) 3660 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:10:03.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:03 smithi151 bash[23756]: cluster 2024-03-04T16:10:03.300041+0000 mgr.y (mgr.24428) 774 : cluster 0 pgmap v1292: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-03-04T16:10:03.761 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:03.757+0000 7fca2d138640 1 -- 172.21.15.117:0/1684284265 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fca281a8800 con 0x7fca28103e70 2024-03-04T16:10:03.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:03 smithi117 bash[21480]: audit 2024-03-04T16:10:02.403925+0000 mon.a (mon.0) 3658 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]': finished 2024-03-04T16:10:03.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:03 smithi117 bash[21480]: cluster 2024-03-04T16:10:02.411563+0000 mon.a (mon.0) 3659 : cluster 0 osdmap e779: 8 total, 8 up, 8 in 2024-03-04T16:10:03.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:03 smithi117 bash[21480]: audit 2024-03-04T16:10:02.517988+0000 mon.c (mon.2) 838 : audit 1 from='client.? 172.21.15.117:0/2745215032' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:10:03.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:03 smithi117 bash[21480]: audit 2024-03-04T16:10:02.518523+0000 mon.a (mon.0) 3660 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:10:03.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:03 smithi117 bash[21480]: cluster 2024-03-04T16:10:03.300041+0000 mgr.y (mgr.24428) 774 : cluster 0 pgmap v1292: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-03-04T16:10:03.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:03 smithi117 bash[28843]: audit 2024-03-04T16:10:02.403925+0000 mon.a (mon.0) 3658 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]': finished 2024-03-04T16:10:03.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:03 smithi117 bash[28843]: cluster 2024-03-04T16:10:02.411563+0000 mon.a (mon.0) 3659 : cluster 0 osdmap e779: 8 total, 8 up, 8 in 2024-03-04T16:10:03.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:03 smithi117 bash[28843]: audit 2024-03-04T16:10:02.517988+0000 mon.c (mon.2) 838 : audit 1 from='client.? 172.21.15.117:0/2745215032' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:10:03.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:03 smithi117 bash[28843]: audit 2024-03-04T16:10:02.518523+0000 mon.a (mon.0) 3660 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:10:03.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:03 smithi117 bash[28843]: cluster 2024-03-04T16:10:03.300041+0000 mgr.y (mgr.24428) 774 : cluster 0 pgmap v1292: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 921 B/s rd, 0 op/s 2024-03-04T16:10:04.434 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:04.433+0000 7fca1ffff640 1 -- 172.21.15.117:0/1684284265 <== mon.2 v2:172.21.15.117:3301/0 7 ==== osd_map(781..781 src has 1..781) v4 ==== 628+0+0 (secure 0 0 0) 0x7fca18129e00 con 0x7fca28103e70 2024-03-04T16:10:04.435 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:04.433+0000 7fca1ffff640 1 -- 172.21.15.117:0/1684284265 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=782}) v3 -- 0x7fca14079d60 con 0x7fca28103e70 2024-03-04T16:10:04.444 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:04.441+0000 7fca1ffff640 1 -- 172.21.15.117:0/1684284265 <== mon.2 v2:172.21.15.117:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 v781) v1 ==== 221+0+0 (secure 0 0 0) 0x7fca180f4c10 con 0x7fca28103e70 2024-03-04T16:10:04.552 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:04.549+0000 7fca2d138640 1 -- 172.21.15.117:0/1684284265 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fca280619c0 con 0x7fca28103e70 2024-03-04T16:10:04.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:04 smithi117 bash[21480]: audit 2024-03-04T16:10:03.433420+0000 mon.a (mon.0) 3661 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]': finished 2024-03-04T16:10:04.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:04 smithi117 bash[21480]: cluster 2024-03-04T16:10:03.440987+0000 mon.a (mon.0) 3662 : cluster 0 osdmap e780: 8 total, 8 up, 8 in 2024-03-04T16:10:04.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:04 smithi117 bash[21480]: audit 2024-03-04T16:10:03.765183+0000 mon.c (mon.2) 839 : audit 1 from='client.? 172.21.15.117:0/1684284265' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:10:04.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:04 smithi117 bash[21480]: audit 2024-03-04T16:10:03.765647+0000 mon.a (mon.0) 3663 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:10:04.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:04 smithi117 bash[28843]: audit 2024-03-04T16:10:03.433420+0000 mon.a (mon.0) 3661 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]': finished 2024-03-04T16:10:04.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:04 smithi117 bash[28843]: cluster 2024-03-04T16:10:03.440987+0000 mon.a (mon.0) 3662 : cluster 0 osdmap e780: 8 total, 8 up, 8 in 2024-03-04T16:10:04.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:04 smithi117 bash[28843]: audit 2024-03-04T16:10:03.765183+0000 mon.c (mon.2) 839 : audit 1 from='client.? 172.21.15.117:0/1684284265' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:10:04.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:04 smithi117 bash[28843]: audit 2024-03-04T16:10:03.765647+0000 mon.a (mon.0) 3663 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:10:04.924 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:04.921+0000 7fca1ffff640 1 -- 172.21.15.117:0/1684284265 <== mon.2 v2:172.21.15.117:3301/0 9 ==== osd_map(782..782 src has 1..782) v4 ==== 628+0+0 (secure 0 0 0) 0x7fca180f25b0 con 0x7fca28103e70 2024-03-04T16:10:04.924 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:04.921+0000 7fca1ffff640 1 -- 172.21.15.117:0/1684284265 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=783}) v3 -- 0x7fca1407a150 con 0x7fca28103e70 2024-03-04T16:10:04.930 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:04.929+0000 7fca1ffff640 1 -- 172.21.15.117:0/1684284265 <== mon.2 v2:172.21.15.117:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 v782) v1 ==== 221+0+0 (secure 0 0 0) 0x7fca180f8660 con 0x7fca28103e70 2024-03-04T16:10:04.930 INFO:tasks.workunit.client.0.smithi117.stderr:set-quota max_objects = 0 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 2024-03-04T16:10:04.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:04 smithi151 bash[23756]: audit 2024-03-04T16:10:03.433420+0000 mon.a (mon.0) 3661 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]': finished 2024-03-04T16:10:04.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:04 smithi151 bash[23756]: cluster 2024-03-04T16:10:03.440987+0000 mon.a (mon.0) 3662 : cluster 0 osdmap e780: 8 total, 8 up, 8 in 2024-03-04T16:10:04.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:04 smithi151 bash[23756]: audit 2024-03-04T16:10:03.765183+0000 mon.c (mon.2) 839 : audit 1 from='client.? 172.21.15.117:0/1684284265' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:10:04.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:04 smithi151 bash[23756]: audit 2024-03-04T16:10:03.765647+0000 mon.a (mon.0) 3663 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:10:04.935 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:04.933+0000 7fca2d138640 1 -- 172.21.15.117:0/1684284265 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fca1406d700 msgr2=0x7fca1406fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:04.935 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:04.933+0000 7fca2d138640 1 --2- 172.21.15.117:0/1684284265 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fca1406d700 0x7fca1406fbc0 secure :-1 s=READY pgs=3233 cs=0 l=1 rev1=1 crypto rx=0x7fca2819ca60 tx=0x7fca100091e0 comp rx=0 tx=0).stop 2024-03-04T16:10:04.935 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:04.933+0000 7fca2d138640 1 -- 172.21.15.117:0/1684284265 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca28103e70 msgr2=0x7fca281a1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:04.935 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:04.933+0000 7fca2d138640 1 --2- 172.21.15.117:0/1684284265 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca28103e70 0x7fca281a1ca0 secure :-1 s=READY pgs=3127 cs=0 l=1 rev1=1 crypto rx=0x7fca1801c5f0 tx=0x7fca180adcd0 comp rx=0 tx=0).stop 2024-03-04T16:10:04.936 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:04.933+0000 7fca2d138640 1 -- 172.21.15.117:0/1684284265 shutdown_connections 2024-03-04T16:10:04.936 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:04.933+0000 7fca2d138640 1 --2- 172.21.15.117:0/1684284265 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fca1406d700 0x7fca1406fbc0 unknown :-1 s=CLOSED pgs=3233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:04.936 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:04.933+0000 7fca2d138640 1 --2- 172.21.15.117:0/1684284265 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fca28109560 0x7fca2819b920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:04.937 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:04.933+0000 7fca2d138640 1 --2- 172.21.15.117:0/1684284265 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fca28103e70 0x7fca281a1ca0 unknown :-1 s=CLOSED pgs=3127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:04.937 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:04.933+0000 7fca2d138640 1 --2- 172.21.15.117:0/1684284265 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fca281034b0 0x7fca281a1760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:04.937 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:04.933+0000 7fca2d138640 1 -- 172.21.15.117:0/1684284265 >> 172.21.15.117:0/1684284265 conn(0x7fca280fef40 msgr2=0x7fca2810a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:10:04.937 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:04.933+0000 7fca2d138640 1 -- 172.21.15.117:0/1684284265 shutdown_connections 2024-03-04T16:10:04.937 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:04.933+0000 7fca2d138640 1 -- 172.21.15.117:0/1684284265 wait complete. 2024-03-04T16:10:04.955 INFO:tasks.workunit.client.0.smithi117.stderr:+ wait 132100 2024-03-04T16:10:04.955 INFO:tasks.workunit.client.0.smithi117.stderr:+ [ 0 -ne 0 ] 2024-03-04T16:10:04.955 INFO:tasks.workunit.client.0.smithi117.stderr:+ true 2024-03-04T16:10:04.955 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p 736f5599-4143-4934-af9d-08c0b0a4b1a3 put three /etc/passwd 2024-03-04T16:10:04.995 INFO:tasks.workunit.client.0.smithi117.stderr:+ uuidgen 2024-03-04T16:10:04.995 INFO:tasks.workunit.client.0.smithi117.stderr:+ pp=d70eaa68-ebfd-4a58-9771-a0ceff99a3ca 2024-03-04T16:10:04.995 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph osd pool create d70eaa68-ebfd-4a58-9771-a0ceff99a3ca 12 2024-03-04T16:10:05.112 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848e50a640 1 -- 172.21.15.117:0/3166763449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8488101eb0 msgr2=0x7f8488105f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:05.113 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848e50a640 1 --2- 172.21.15.117:0/3166763449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8488101eb0 0x7f8488105f00 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f8470009930 tx=0x7f847001c7e0 comp rx=0 tx=0).stop 2024-03-04T16:10:05.113 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848e50a640 1 -- 172.21.15.117:0/3166763449 shutdown_connections 2024-03-04T16:10:05.113 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848e50a640 1 --2- 172.21.15.117:0/3166763449 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8488101eb0 0x7f8488105f00 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:05.113 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848e50a640 1 --2- 172.21.15.117:0/3166763449 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8488106630 0x7f848810e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:05.113 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848e50a640 1 --2- 172.21.15.117:0/3166763449 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8488101500 0x7f84881018e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:05.113 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848e50a640 1 -- 172.21.15.117:0/3166763449 >> 172.21.15.117:0/3166763449 conn(0x7f84880fcd90 msgr2=0x7f84880ff1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:10:05.113 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848e50a640 1 -- 172.21.15.117:0/3166763449 shutdown_connections 2024-03-04T16:10:05.113 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848e50a640 1 -- 172.21.15.117:0/3166763449 wait complete. 2024-03-04T16:10:05.113 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848e50a640 1 Processor -- start 2024-03-04T16:10:05.113 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848e50a640 1 -- start start 2024-03-04T16:10:05.113 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848e50a640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8488101500 0x7f8488199e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:10:05.113 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848e50a640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8488101eb0 0x7f848819a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:10:05.113 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848e50a640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8488106630 0x7f848819e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:10:05.113 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848e50a640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84881138f0 con 0x7f8488106630 2024-03-04T16:10:05.114 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848e50a640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8488113770 con 0x7f8488101eb0 2024-03-04T16:10:05.114 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848e50a640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8488113a70 con 0x7f8488101500 2024-03-04T16:10:05.114 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f8487fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8488101500 0x7f8488199e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:10:05.114 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848caa7640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8488106630 0x7f848819e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:10:05.114 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f8487fff640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8488101500 0x7f8488199e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:54992/0 (socket says 172.21.15.117:54992) 2024-03-04T16:10:05.114 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f8487fff640 1 -- 172.21.15.117:0/1901470347 learned_addr learned my addr 172.21.15.117:0/1901470347 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T16:10:05.114 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f84877fe640 1 --2- 172.21.15.117:0/1901470347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8488101eb0 0x7f848819a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:10:05.114 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f8487fff640 1 -- 172.21.15.117:0/1901470347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8488101eb0 msgr2=0x7f848819a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:05.114 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f8487fff640 1 --2- 172.21.15.117:0/1901470347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8488101eb0 0x7f848819a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:05.114 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f8487fff640 1 -- 172.21.15.117:0/1901470347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8488106630 msgr2=0x7f848819e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:05.114 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f8487fff640 1 --2- 172.21.15.117:0/1901470347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8488106630 0x7f848819e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:05.114 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f8487fff640 1 -- 172.21.15.117:0/1901470347 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f848819ee30 con 0x7f8488101500 2024-03-04T16:10:05.114 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f8487fff640 1 --2- 172.21.15.117:0/1901470347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8488101500 0x7f8488199e80 secure :-1 s=READY pgs=3129 cs=0 l=1 rev1=1 crypto rx=0x7f847800d950 tx=0x7f847800de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:10:05.114 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f84857fa640 1 -- 172.21.15.117:0/1901470347 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8478017070 con 0x7f8488101500 2024-03-04T16:10:05.115 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848e50a640 1 -- 172.21.15.117:0/1901470347 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f848819f120 con 0x7f8488101500 2024-03-04T16:10:05.115 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f84857fa640 1 -- 172.21.15.117:0/1901470347 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f8478009c30 con 0x7f8488101500 2024-03-04T16:10:05.115 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f84857fa640 1 -- 172.21.15.117:0/1901470347 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f847800f3c0 con 0x7f8488101500 2024-03-04T16:10:05.115 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848e50a640 1 -- 172.21.15.117:0/1901470347 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84881a6a00 con 0x7f8488101500 2024-03-04T16:10:05.116 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.109+0000 7f848e50a640 1 -- 172.21.15.117:0/1901470347 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8488068630 con 0x7f8488101500 2024-03-04T16:10:05.116 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.113+0000 7f84857fa640 1 -- 172.21.15.117:0/1901470347 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f8478010460 con 0x7f8488101500 2024-03-04T16:10:05.116 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.113+0000 7f84857fa640 1 --2- 172.21.15.117:0/1901470347 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f847406d700 0x7f847406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:10:05.116 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.113+0000 7f84857fa640 1 -- 172.21.15.117:0/1901470347 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(782..782 src has 1..782) v4 ==== 9068+0+0 (secure 0 0 0) 0x7f847808dd30 con 0x7f8488101500 2024-03-04T16:10:05.116 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.113+0000 7f84877fe640 1 --2- 172.21.15.117:0/1901470347 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f847406d700 0x7f847406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:10:05.116 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.113+0000 7f84877fe640 1 --2- 172.21.15.117:0/1901470347 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f847406d700 0x7f847406fbc0 secure :-1 s=READY pgs=3235 cs=0 l=1 rev1=1 crypto rx=0x7f84880fefe0 tx=0x7f847001b020 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:10:05.119 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.117+0000 7f84857fa640 1 -- 172.21.15.117:0/1901470347 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f847805e990 con 0x7f8488101500 2024-03-04T16:10:05.291 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.289+0000 7f848e50a640 1 -- 172.21.15.117:0/1901470347 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pg_num": 12} v 0) v1 -- 0x7f84881a6bd0 con 0x7f8488101500 2024-03-04T16:10:05.943 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:05.941+0000 7f84857fa640 1 -- 172.21.15.117:0/1901470347 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pg_num": 12}]=0 pool 'd70eaa68-ebfd-4a58-9771-a0ceff99a3ca' created v783) v1 ==== 176+0+0 (secure 0 0 0) 0x7f8478062f50 con 0x7f8488101500 2024-03-04T16:10:06.053 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.053+0000 7f848e50a640 1 -- 172.21.15.117:0/1901470347 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pg_num": 12} v 0) v1 -- 0x7f8488061900 con 0x7f8488101500 2024-03-04T16:10:06.055 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.053+0000 7f84857fa640 1 -- 172.21.15.117:0/1901470347 <== mon.2 v2:172.21.15.117:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pg_num": 12}]=0 pool 'd70eaa68-ebfd-4a58-9771-a0ceff99a3ca' already exists v783) v1 ==== 183+0+0 (secure 0 0 0) 0x7f8478058990 con 0x7f8488101500 2024-03-04T16:10:06.055 INFO:tasks.workunit.client.0.smithi117.stderr:pool 'd70eaa68-ebfd-4a58-9771-a0ceff99a3ca' already exists 2024-03-04T16:10:06.058 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.057+0000 7f848e50a640 1 -- 172.21.15.117:0/1901470347 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f847406d700 msgr2=0x7f847406fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:06.058 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.057+0000 7f848e50a640 1 --2- 172.21.15.117:0/1901470347 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f847406d700 0x7f847406fbc0 secure :-1 s=READY pgs=3235 cs=0 l=1 rev1=1 crypto rx=0x7f84880fefe0 tx=0x7f847001b020 comp rx=0 tx=0).stop 2024-03-04T16:10:06.058 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.057+0000 7f848e50a640 1 -- 172.21.15.117:0/1901470347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8488101500 msgr2=0x7f8488199e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:06.058 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.057+0000 7f848e50a640 1 --2- 172.21.15.117:0/1901470347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8488101500 0x7f8488199e80 secure :-1 s=READY pgs=3129 cs=0 l=1 rev1=1 crypto rx=0x7f847800d950 tx=0x7f847800de10 comp rx=0 tx=0).stop 2024-03-04T16:10:06.058 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.057+0000 7f848e50a640 1 -- 172.21.15.117:0/1901470347 shutdown_connections 2024-03-04T16:10:06.058 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.057+0000 7f848e50a640 1 --2- 172.21.15.117:0/1901470347 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f847406d700 0x7f847406fbc0 unknown :-1 s=CLOSED pgs=3235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:06.058 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.057+0000 7f848e50a640 1 --2- 172.21.15.117:0/1901470347 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f8488106630 0x7f848819e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:06.058 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.057+0000 7f848e50a640 1 --2- 172.21.15.117:0/1901470347 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f8488101eb0 0x7f848819a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:06.058 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.057+0000 7f848e50a640 1 --2- 172.21.15.117:0/1901470347 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f8488101500 0x7f8488199e80 unknown :-1 s=CLOSED pgs=3129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:06.058 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.057+0000 7f848e50a640 1 -- 172.21.15.117:0/1901470347 >> 172.21.15.117:0/1901470347 conn(0x7f84880fcd90 msgr2=0x7f84880fd390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:10:06.059 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.057+0000 7f848e50a640 1 -- 172.21.15.117:0/1901470347 shutdown_connections 2024-03-04T16:10:06.059 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.057+0000 7f848e50a640 1 -- 172.21.15.117:0/1901470347 wait complete. 2024-03-04T16:10:06.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:05 smithi117 bash[21480]: audit 2024-03-04T16:10:04.436068+0000 mon.a (mon.0) 3664 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]': finished 2024-03-04T16:10:06.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:05 smithi117 bash[21480]: cluster 2024-03-04T16:10:04.444088+0000 mon.a (mon.0) 3665 : cluster 0 osdmap e781: 8 total, 8 up, 8 in 2024-03-04T16:10:06.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:05 smithi117 bash[21480]: audit 2024-03-04T16:10:04.555931+0000 mon.c (mon.2) 840 : audit 1 from='client.? 172.21.15.117:0/1684284265' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:10:06.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:05 smithi117 bash[21480]: audit 2024-03-04T16:10:04.556729+0000 mon.a (mon.0) 3666 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:10:06.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:05 smithi117 bash[21480]: cluster 2024-03-04T16:10:04.920965+0000 mon.a (mon.0) 3667 : cluster 1 pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' no longer out of quota; removing NO_QUOTA flag 2024-03-04T16:10:06.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:05 smithi117 bash[21480]: cluster 2024-03-04T16:10:04.921616+0000 mon.a (mon.0) 3668 : cluster 1 Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-04T16:10:06.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:05 smithi117 bash[21480]: audit 2024-03-04T16:10:04.923917+0000 mon.a (mon.0) 3669 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]': finished 2024-03-04T16:10:06.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:05 smithi117 bash[21480]: cluster 2024-03-04T16:10:04.934486+0000 mon.a (mon.0) 3670 : cluster 0 osdmap e782: 8 total, 8 up, 8 in 2024-03-04T16:10:06.068 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:05 smithi117 bash[28843]: audit 2024-03-04T16:10:04.436068+0000 mon.a (mon.0) 3664 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]': finished 2024-03-04T16:10:06.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:05 smithi117 bash[28843]: cluster 2024-03-04T16:10:04.444088+0000 mon.a (mon.0) 3665 : cluster 0 osdmap e781: 8 total, 8 up, 8 in 2024-03-04T16:10:06.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:05 smithi117 bash[28843]: audit 2024-03-04T16:10:04.555931+0000 mon.c (mon.2) 840 : audit 1 from='client.? 172.21.15.117:0/1684284265' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:10:06.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:05 smithi117 bash[28843]: audit 2024-03-04T16:10:04.556729+0000 mon.a (mon.0) 3666 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:10:06.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:05 smithi117 bash[28843]: cluster 2024-03-04T16:10:04.920965+0000 mon.a (mon.0) 3667 : cluster 1 pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' no longer out of quota; removing NO_QUOTA flag 2024-03-04T16:10:06.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:05 smithi117 bash[28843]: cluster 2024-03-04T16:10:04.921616+0000 mon.a (mon.0) 3668 : cluster 1 Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-04T16:10:06.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:05 smithi117 bash[28843]: audit 2024-03-04T16:10:04.923917+0000 mon.a (mon.0) 3669 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]': finished 2024-03-04T16:10:06.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:05 smithi117 bash[28843]: cluster 2024-03-04T16:10:04.934486+0000 mon.a (mon.0) 3670 : cluster 0 osdmap e782: 8 total, 8 up, 8 in 2024-03-04T16:10:06.072 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph osd pool application enable d70eaa68-ebfd-4a58-9771-a0ceff99a3ca rados 2024-03-04T16:10:06.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:05 smithi151 bash[23756]: audit 2024-03-04T16:10:04.436068+0000 mon.a (mon.0) 3664 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]': finished 2024-03-04T16:10:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:05 smithi151 bash[23756]: cluster 2024-03-04T16:10:04.444088+0000 mon.a (mon.0) 3665 : cluster 0 osdmap e781: 8 total, 8 up, 8 in 2024-03-04T16:10:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:05 smithi151 bash[23756]: audit 2024-03-04T16:10:04.555931+0000 mon.c (mon.2) 840 : audit 1 from='client.? 172.21.15.117:0/1684284265' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:10:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:05 smithi151 bash[23756]: audit 2024-03-04T16:10:04.556729+0000 mon.a (mon.0) 3666 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:10:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:05 smithi151 bash[23756]: cluster 2024-03-04T16:10:04.920965+0000 mon.a (mon.0) 3667 : cluster 1 pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' no longer out of quota; removing NO_QUOTA flag 2024-03-04T16:10:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:05 smithi151 bash[23756]: cluster 2024-03-04T16:10:04.921616+0000 mon.a (mon.0) 3668 : cluster 1 Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-03-04T16:10:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:05 smithi151 bash[23756]: audit 2024-03-04T16:10:04.923917+0000 mon.a (mon.0) 3669 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]': finished 2024-03-04T16:10:06.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:05 smithi151 bash[23756]: cluster 2024-03-04T16:10:04.934486+0000 mon.a (mon.0) 3670 : cluster 0 osdmap e782: 8 total, 8 up, 8 in 2024-03-04T16:10:06.197 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f8973c18640 1 -- 172.21.15.117:0/2486339254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f896c106600 msgr2=0x7f896c10e180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:06.197 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f8973c18640 1 --2- 172.21.15.117:0/2486339254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f896c106600 0x7f896c10e180 secure :-1 s=READY pgs=3130 cs=0 l=1 rev1=1 crypto rx=0x7f8964009cb0 tx=0x7f896401cca0 comp rx=0 tx=0).stop 2024-03-04T16:10:06.197 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f8973c18640 1 -- 172.21.15.117:0/2486339254 shutdown_connections 2024-03-04T16:10:06.197 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f8973c18640 1 --2- 172.21.15.117:0/2486339254 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f896c106600 0x7f896c10e180 unknown :-1 s=CLOSED pgs=3130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:06.197 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f8973c18640 1 --2- 172.21.15.117:0/2486339254 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f896c101e80 0x7f896c105ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:06.197 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f8973c18640 1 --2- 172.21.15.117:0/2486339254 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f896c1014d0 0x7f896c1018b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:06.197 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f8973c18640 1 -- 172.21.15.117:0/2486339254 >> 172.21.15.117:0/2486339254 conn(0x7f896c0fcdc0 msgr2=0x7f896c0ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:10:06.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f8973c18640 1 -- 172.21.15.117:0/2486339254 shutdown_connections 2024-03-04T16:10:06.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f8973c18640 1 -- 172.21.15.117:0/2486339254 wait complete. 2024-03-04T16:10:06.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f8973c18640 1 Processor -- start 2024-03-04T16:10:06.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f8973c18640 1 -- start start 2024-03-04T16:10:06.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f8973c18640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f896c1014d0 0x7f896c199dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:10:06.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f8973c18640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f896c101e80 0x7f896c19a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:10:06.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f8973c18640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f896c106600 0x7f896c19e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:10:06.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f8973c18640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c1138c0 con 0x7f896c1014d0 2024-03-04T16:10:06.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f8973c18640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f896c113740 con 0x7f896c101e80 2024-03-04T16:10:06.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f8973c18640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f896c113a40 con 0x7f896c106600 2024-03-04T16:10:06.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f89721b5640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f896c106600 0x7f896c19e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:10:06.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f89719b4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f896c1014d0 0x7f896c199dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:10:06.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f89719b4640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f896c1014d0 0x7f896c199dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:38486/0 (socket says 172.21.15.117:38486) 2024-03-04T16:10:06.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f89719b4640 1 -- 172.21.15.117:0/3503630643 learned_addr learned my addr 172.21.15.117:0/3503630643 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T16:10:06.198 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f89711b3640 1 --2- 172.21.15.117:0/3503630643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f896c101e80 0x7f896c19a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:10:06.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f89719b4640 1 -- 172.21.15.117:0/3503630643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f896c106600 msgr2=0x7f896c19e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:06.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f89719b4640 1 --2- 172.21.15.117:0/3503630643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f896c106600 0x7f896c19e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:06.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f89719b4640 1 -- 172.21.15.117:0/3503630643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f896c101e80 msgr2=0x7f896c19a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:06.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f89719b4640 1 --2- 172.21.15.117:0/3503630643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f896c101e80 0x7f896c19a310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:06.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f89719b4640 1 -- 172.21.15.117:0/3503630643 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f896c19ed80 con 0x7f896c1014d0 2024-03-04T16:10:06.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.193+0000 7f89721b5640 1 --2- 172.21.15.117:0/3503630643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f896c106600 0x7f896c19e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T16:10:06.199 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.197+0000 7f89719b4640 1 --2- 172.21.15.117:0/3503630643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f896c1014d0 0x7f896c199dd0 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f896800e9f0 tx=0x7f896800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:10:06.200 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.197+0000 7f8962ffd640 1 -- 172.21.15.117:0/3503630643 <== mon.0 v2:172.21.15.117:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f896800cd80 con 0x7f896c1014d0 2024-03-04T16:10:06.200 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.197+0000 7f8973c18640 1 -- 172.21.15.117:0/3503630643 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f896c19f070 con 0x7f896c1014d0 2024-03-04T16:10:06.200 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.197+0000 7f8962ffd640 1 -- 172.21.15.117:0/3503630643 <== mon.0 v2:172.21.15.117:3300/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f89680108b0 con 0x7f896c1014d0 2024-03-04T16:10:06.200 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.197+0000 7f8962ffd640 1 -- 172.21.15.117:0/3503630643 <== mon.0 v2:172.21.15.117:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8968018800 con 0x7f896c1014d0 2024-03-04T16:10:06.201 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.197+0000 7f8973c18640 1 -- 172.21.15.117:0/3503630643 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f896c1a6950 con 0x7f896c1014d0 2024-03-04T16:10:06.201 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.197+0000 7f8973c18640 1 -- 172.21.15.117:0/3503630643 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f896c0681e0 con 0x7f896c1014d0 2024-03-04T16:10:06.201 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.197+0000 7f8962ffd640 1 -- 172.21.15.117:0/3503630643 <== mon.0 v2:172.21.15.117:3300/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f8968010430 con 0x7f896c1014d0 2024-03-04T16:10:06.201 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.197+0000 7f8962ffd640 1 --2- 172.21.15.117:0/3503630643 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f895806d700 0x7f895806fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:10:06.201 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.197+0000 7f8962ffd640 1 -- 172.21.15.117:0/3503630643 <== mon.0 v2:172.21.15.117:3300/0 5 ==== osd_map(783..783 src has 1..783) v4 ==== 9443+0+0 (secure 0 0 0) 0x7f896808ea00 con 0x7f896c1014d0 2024-03-04T16:10:06.201 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.197+0000 7f89711b3640 1 --2- 172.21.15.117:0/3503630643 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f895806d700 0x7f895806fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:10:06.201 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.197+0000 7f89711b3640 1 --2- 172.21.15.117:0/3503630643 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f895806d700 0x7f895806fbc0 secure :-1 s=READY pgs=3236 cs=0 l=1 rev1=1 crypto rx=0x7f896c0ff6c0 tx=0x7f895c005c60 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:10:06.204 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.201+0000 7f8962ffd640 1 -- 172.21.15.117:0/3503630643 <== mon.0 v2:172.21.15.117:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f896805f620 con 0x7f896c1014d0 2024-03-04T16:10:06.374 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.373+0000 7f8973c18640 1 -- 172.21.15.117:0/3503630643 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "app": "rados"} v 0) v1 -- 0x7f896c1a6b20 con 0x7f896c1014d0 2024-03-04T16:10:06.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:10:06 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:10:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:10:06.937 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:06.933+0000 7f8962ffd640 1 -- 172.21.15.117:0/3503630643 <== mon.0 v2:172.21.15.117:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "app": "rados"}]=0 enabled application 'rados' on pool 'd70eaa68-ebfd-4a58-9771-a0ceff99a3ca' v784) v1 ==== 213+0+0 (secure 0 0 0) 0x7f8968063be0 con 0x7f896c1014d0 2024-03-04T16:10:07.056 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:07.053+0000 7f8973c18640 1 -- 172.21.15.117:0/3503630643 --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "app": "rados"} v 0) v1 -- 0x7f896c1a6d70 con 0x7f896c1014d0 2024-03-04T16:10:07.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:06 smithi117 bash[21480]: audit 2024-03-04T16:10:05.295005+0000 mon.c (mon.2) 841 : audit 1 from='client.? 172.21.15.117:0/1901470347' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pg_num": 12}]: dispatch 2024-03-04T16:10:07.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:06 smithi117 bash[21480]: audit 2024-03-04T16:10:05.295616+0000 mon.a (mon.0) 3671 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pg_num": 12}]: dispatch 2024-03-04T16:10:07.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:06 smithi117 bash[21480]: cluster 2024-03-04T16:10:05.300508+0000 mgr.y (mgr.24428) 775 : cluster 0 pgmap v1296: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:10:07.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:06 smithi117 bash[21480]: audit 2024-03-04T16:10:05.937302+0000 mon.a (mon.0) 3672 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pg_num": 12}]': finished 2024-03-04T16:10:07.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:06 smithi117 bash[21480]: cluster 2024-03-04T16:10:05.945691+0000 mon.a (mon.0) 3673 : cluster 0 osdmap e783: 8 total, 8 up, 8 in 2024-03-04T16:10:07.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:06 smithi117 bash[21480]: audit 2024-03-04T16:10:06.057730+0000 mon.c (mon.2) 842 : audit 1 from='client.? 172.21.15.117:0/1901470347' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pg_num": 12}]: dispatch 2024-03-04T16:10:07.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:06 smithi117 bash[21480]: audit 2024-03-04T16:10:06.058319+0000 mon.a (mon.0) 3674 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pg_num": 12}]: dispatch 2024-03-04T16:10:07.068 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:06 smithi117 bash[21480]: audit 2024-03-04T16:10:06.378120+0000 mon.a (mon.0) 3675 : audit 1 from='client.? 172.21.15.117:0/3503630643' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "app": "rados"}]: dispatch 2024-03-04T16:10:07.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:06 smithi117 bash[28843]: audit 2024-03-04T16:10:05.295005+0000 mon.c (mon.2) 841 : audit 1 from='client.? 172.21.15.117:0/1901470347' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pg_num": 12}]: dispatch 2024-03-04T16:10:07.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:06 smithi117 bash[28843]: audit 2024-03-04T16:10:05.295616+0000 mon.a (mon.0) 3671 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pg_num": 12}]: dispatch 2024-03-04T16:10:07.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:06 smithi117 bash[28843]: cluster 2024-03-04T16:10:05.300508+0000 mgr.y (mgr.24428) 775 : cluster 0 pgmap v1296: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:10:07.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:06 smithi117 bash[28843]: audit 2024-03-04T16:10:05.937302+0000 mon.a (mon.0) 3672 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pg_num": 12}]': finished 2024-03-04T16:10:07.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:06 smithi117 bash[28843]: cluster 2024-03-04T16:10:05.945691+0000 mon.a (mon.0) 3673 : cluster 0 osdmap e783: 8 total, 8 up, 8 in 2024-03-04T16:10:07.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:06 smithi117 bash[28843]: audit 2024-03-04T16:10:06.057730+0000 mon.c (mon.2) 842 : audit 1 from='client.? 172.21.15.117:0/1901470347' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pg_num": 12}]: dispatch 2024-03-04T16:10:07.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:06 smithi117 bash[28843]: audit 2024-03-04T16:10:06.058319+0000 mon.a (mon.0) 3674 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pg_num": 12}]: dispatch 2024-03-04T16:10:07.069 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:06 smithi117 bash[28843]: audit 2024-03-04T16:10:06.378120+0000 mon.a (mon.0) 3675 : audit 1 from='client.? 172.21.15.117:0/3503630643' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "app": "rados"}]: dispatch 2024-03-04T16:10:07.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:06 smithi151 bash[23756]: audit 2024-03-04T16:10:05.295005+0000 mon.c (mon.2) 841 : audit 1 from='client.? 172.21.15.117:0/1901470347' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pg_num": 12}]: dispatch 2024-03-04T16:10:07.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:06 smithi151 bash[23756]: audit 2024-03-04T16:10:05.295616+0000 mon.a (mon.0) 3671 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pg_num": 12}]: dispatch 2024-03-04T16:10:07.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:06 smithi151 bash[23756]: cluster 2024-03-04T16:10:05.300508+0000 mgr.y (mgr.24428) 775 : cluster 0 pgmap v1296: 176 pgs: 176 active+clean; 722 KiB data, 802 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:10:07.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:06 smithi151 bash[23756]: audit 2024-03-04T16:10:05.937302+0000 mon.a (mon.0) 3672 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pg_num": 12}]': finished 2024-03-04T16:10:07.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:06 smithi151 bash[23756]: cluster 2024-03-04T16:10:05.945691+0000 mon.a (mon.0) 3673 : cluster 0 osdmap e783: 8 total, 8 up, 8 in 2024-03-04T16:10:07.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:06 smithi151 bash[23756]: audit 2024-03-04T16:10:06.057730+0000 mon.c (mon.2) 842 : audit 1 from='client.? 172.21.15.117:0/1901470347' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pg_num": 12}]: dispatch 2024-03-04T16:10:07.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:06 smithi151 bash[23756]: audit 2024-03-04T16:10:06.058319+0000 mon.a (mon.0) 3674 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pg_num": 12}]: dispatch 2024-03-04T16:10:07.182 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:06 smithi151 bash[23756]: audit 2024-03-04T16:10:06.378120+0000 mon.a (mon.0) 3675 : audit 1 from='client.? 172.21.15.117:0/3503630643' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "app": "rados"}]: dispatch 2024-03-04T16:10:07.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:10:07 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:10:07.940 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:07.937+0000 7f8962ffd640 1 -- 172.21.15.117:0/3503630643 <== mon.0 v2:172.21.15.117:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "app": "rados"}]=0 enabled application 'rados' on pool 'd70eaa68-ebfd-4a58-9771-a0ceff99a3ca' v785) v1 ==== 213+0+0 (secure 0 0 0) 0x7f8968010d40 con 0x7f896c1014d0 2024-03-04T16:10:07.940 INFO:tasks.workunit.client.0.smithi117.stderr:enabled application 'rados' on pool 'd70eaa68-ebfd-4a58-9771-a0ceff99a3ca' 2024-03-04T16:10:07.944 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:07.941+0000 7f8973c18640 1 -- 172.21.15.117:0/3503630643 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f895806d700 msgr2=0x7f895806fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:07.944 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:07.941+0000 7f8973c18640 1 --2- 172.21.15.117:0/3503630643 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f895806d700 0x7f895806fbc0 secure :-1 s=READY pgs=3236 cs=0 l=1 rev1=1 crypto rx=0x7f896c0ff6c0 tx=0x7f895c005c60 comp rx=0 tx=0).stop 2024-03-04T16:10:07.944 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:07.941+0000 7f8973c18640 1 -- 172.21.15.117:0/3503630643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f896c1014d0 msgr2=0x7f896c199dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:07.944 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:07.941+0000 7f8973c18640 1 --2- 172.21.15.117:0/3503630643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f896c1014d0 0x7f896c199dd0 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f896800e9f0 tx=0x7f896800eeb0 comp rx=0 tx=0).stop 2024-03-04T16:10:07.945 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:07.941+0000 7f8973c18640 1 -- 172.21.15.117:0/3503630643 shutdown_connections 2024-03-04T16:10:07.945 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:07.941+0000 7f8973c18640 1 --2- 172.21.15.117:0/3503630643 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f895806d700 0x7f895806fbc0 unknown :-1 s=CLOSED pgs=3236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:07.945 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:07.941+0000 7f8973c18640 1 --2- 172.21.15.117:0/3503630643 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f896c106600 0x7f896c19e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:07.945 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:07.941+0000 7f8973c18640 1 --2- 172.21.15.117:0/3503630643 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f896c101e80 0x7f896c19a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:07.945 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:07.941+0000 7f8973c18640 1 --2- 172.21.15.117:0/3503630643 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f896c1014d0 0x7f896c199dd0 secure :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f896800e9f0 tx=0x7f896800eeb0 comp rx=0 tx=0).stop 2024-03-04T16:10:07.945 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:07.941+0000 7f8973c18640 1 -- 172.21.15.117:0/3503630643 >> 172.21.15.117:0/3503630643 conn(0x7f896c0fcdc0 msgr2=0x7f896c0fd330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:10:07.945 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:07.945+0000 7f8973c18640 1 -- 172.21.15.117:0/3503630643 shutdown_connections 2024-03-04T16:10:07.946 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:07.945+0000 7f8973c18640 1 -- 172.21.15.117:0/3503630643 wait complete. 2024-03-04T16:10:07.966 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph osd pool set-quota d70eaa68-ebfd-4a58-9771-a0ceff99a3ca max_objects 10 2024-03-04T16:10:08.088 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c9129640 1 -- 172.21.15.117:0/3979774966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4c41003e0 msgr2=0x7fa4c41007c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:08.088 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c9129640 1 --2- 172.21.15.117:0/3979774966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4c41003e0 0x7fa4c41007c0 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc009930 tx=0x7fa4bc01c7e0 comp rx=0 tx=0).stop 2024-03-04T16:10:08.088 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c9129640 1 -- 172.21.15.117:0/3979774966 shutdown_connections 2024-03-04T16:10:08.088 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c9129640 1 --2- 172.21.15.117:0/3979774966 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4c4105510 0x7fa4c410d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:08.088 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c9129640 1 --2- 172.21.15.117:0/3979774966 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4c4100d90 0x7fa4c4104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:08.088 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c9129640 1 --2- 172.21.15.117:0/3979774966 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4c41003e0 0x7fa4c41007c0 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:08.088 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c9129640 1 -- 172.21.15.117:0/3979774966 >> 172.21.15.117:0/3979774966 conn(0x7fa4c40fbcb0 msgr2=0x7fa4c40fe0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:10:08.088 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c9129640 1 -- 172.21.15.117:0/3979774966 shutdown_connections 2024-03-04T16:10:08.088 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c9129640 1 -- 172.21.15.117:0/3979774966 wait complete. 2024-03-04T16:10:08.089 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c9129640 1 Processor -- start 2024-03-04T16:10:08.089 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c9129640 1 -- start start 2024-03-04T16:10:08.089 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c9129640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4c41003e0 0x7fa4c419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:10:08.089 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c9129640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4c4100d90 0x7fa4c419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:10:08.089 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c9129640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4c4105510 0x7fa4c419ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:10:08.089 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c9129640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c4112da0 con 0x7fa4c4105510 2024-03-04T16:10:08.089 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c9129640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c4112c20 con 0x7fa4c4100d90 2024-03-04T16:10:08.089 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c9129640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4c4112f20 con 0x7fa4c41003e0 2024-03-04T16:10:08.089 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c2d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4c41003e0 0x7fa4c419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:10:08.089 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c2d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4c41003e0 0x7fa4c419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:43798/0 (socket says 172.21.15.117:43798) 2024-03-04T16:10:08.089 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c2d9d640 1 -- 172.21.15.117:0/341278432 learned_addr learned my addr 172.21.15.117:0/341278432 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T16:10:08.089 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c359e640 1 --2- 172.21.15.117:0/341278432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4c4105510 0x7fa4c419ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:10:08.089 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c2d9d640 1 -- 172.21.15.117:0/341278432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4c4100d90 msgr2=0x7fa4c419a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:08.089 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c2d9d640 1 --2- 172.21.15.117:0/341278432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4c4100d90 0x7fa4c419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:08.089 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c2d9d640 1 -- 172.21.15.117:0/341278432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4c4105510 msgr2=0x7fa4c419ed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:08.090 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c2d9d640 1 --2- 172.21.15.117:0/341278432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4c4105510 0x7fa4c419ed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:08.090 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c2d9d640 1 -- 172.21.15.117:0/341278432 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4c419f440 con 0x7fa4c41003e0 2024-03-04T16:10:08.090 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c2d9d640 1 --2- 172.21.15.117:0/341278432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4c41003e0 0x7fa4c419a490 secure :-1 s=READY pgs=3131 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc0a8020 tx=0x7fa4bc0ad6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:10:08.090 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4abfff640 1 -- 172.21.15.117:0/341278432 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc0ba020 con 0x7fa4c41003e0 2024-03-04T16:10:08.090 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c9129640 1 -- 172.21.15.117:0/341278432 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4c419f6d0 con 0x7fa4c41003e0 2024-03-04T16:10:08.090 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4abfff640 1 -- 172.21.15.117:0/341278432 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7fa4bc0a5a60 con 0x7fa4c41003e0 2024-03-04T16:10:08.090 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4abfff640 1 -- 172.21.15.117:0/341278432 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4bc0b2040 con 0x7fa4c41003e0 2024-03-04T16:10:08.090 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c9129640 1 -- 172.21.15.117:0/341278432 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4c41a7010 con 0x7fa4c41003e0 2024-03-04T16:10:08.091 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.085+0000 7fa4c9129640 1 -- 172.21.15.117:0/341278432 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4c4068630 con 0x7fa4c41003e0 2024-03-04T16:10:08.091 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.089+0000 7fa4abfff640 1 -- 172.21.15.117:0/341278432 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7fa4bc0bd020 con 0x7fa4c41003e0 2024-03-04T16:10:08.091 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.089+0000 7fa4abfff640 1 --2- 172.21.15.117:0/341278432 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fa4ac06d700 0x7fa4ac06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:10:08.091 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.089+0000 7fa4abfff640 1 -- 172.21.15.117:0/341278432 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(785..785 src has 1..785) v4 ==== 9456+0+0 (secure 0 0 0) 0x7fa4bc1289a0 con 0x7fa4c41003e0 2024-03-04T16:10:08.091 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.089+0000 7fa4c259c640 1 --2- 172.21.15.117:0/341278432 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fa4ac06d700 0x7fa4ac06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:10:08.091 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.089+0000 7fa4c259c640 1 --2- 172.21.15.117:0/341278432 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fa4ac06d700 0x7fa4ac06fbc0 secure :-1 s=READY pgs=3237 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8005d10 tx=0x7fa4b8005c60 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:10:08.095 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.093+0000 7fa4abfff640 1 -- 172.21.15.117:0/341278432 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7fa4bc0c4020 con 0x7fa4c41003e0 2024-03-04T16:10:08.269 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.265+0000 7fa4c9129640 1 -- 172.21.15.117:0/341278432 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fa4c41a71e0 con 0x7fa4c41003e0 2024-03-04T16:10:08.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:07 smithi117 bash[21480]: audit 2024-03-04T16:10:06.940268+0000 mon.a (mon.0) 3676 : audit 1 from='client.? 172.21.15.117:0/3503630643' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "app": "rados"}]': finished 2024-03-04T16:10:08.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:07 smithi117 bash[21480]: cluster 2024-03-04T16:10:06.945672+0000 mon.a (mon.0) 3677 : cluster 0 osdmap e784: 8 total, 8 up, 8 in 2024-03-04T16:10:08.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:07 smithi117 bash[21480]: audit 2024-03-04T16:10:07.059772+0000 mon.a (mon.0) 3678 : audit 1 from='client.? 172.21.15.117:0/3503630643' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "app": "rados"}]: dispatch 2024-03-04T16:10:08.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:07 smithi117 bash[21480]: cluster 2024-03-04T16:10:07.301312+0000 mgr.y (mgr.24428) 776 : cluster 0 pgmap v1299: 188 pgs: 12 creating+peering, 176 active+clean; 745 KiB data, 803 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 6.2 KiB/s wr, 1 op/s 2024-03-04T16:10:08.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:07 smithi117 bash[21480]: audit 2024-03-04T16:10:07.346566+0000 mgr.y (mgr.24428) 777 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:10:08.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:07 smithi117 bash[28843]: audit 2024-03-04T16:10:06.940268+0000 mon.a (mon.0) 3676 : audit 1 from='client.? 172.21.15.117:0/3503630643' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "app": "rados"}]': finished 2024-03-04T16:10:08.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:07 smithi117 bash[28843]: cluster 2024-03-04T16:10:06.945672+0000 mon.a (mon.0) 3677 : cluster 0 osdmap e784: 8 total, 8 up, 8 in 2024-03-04T16:10:08.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:07 smithi117 bash[28843]: audit 2024-03-04T16:10:07.059772+0000 mon.a (mon.0) 3678 : audit 1 from='client.? 172.21.15.117:0/3503630643' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "app": "rados"}]: dispatch 2024-03-04T16:10:08.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:07 smithi117 bash[28843]: cluster 2024-03-04T16:10:07.301312+0000 mgr.y (mgr.24428) 776 : cluster 0 pgmap v1299: 188 pgs: 12 creating+peering, 176 active+clean; 745 KiB data, 803 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 6.2 KiB/s wr, 1 op/s 2024-03-04T16:10:08.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:07 smithi117 bash[28843]: audit 2024-03-04T16:10:07.346566+0000 mgr.y (mgr.24428) 777 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:10:08.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:07 smithi151 bash[23756]: audit 2024-03-04T16:10:06.940268+0000 mon.a (mon.0) 3676 : audit 1 from='client.? 172.21.15.117:0/3503630643' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "app": "rados"}]': finished 2024-03-04T16:10:08.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:07 smithi151 bash[23756]: cluster 2024-03-04T16:10:06.945672+0000 mon.a (mon.0) 3677 : cluster 0 osdmap e784: 8 total, 8 up, 8 in 2024-03-04T16:10:08.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:07 smithi151 bash[23756]: audit 2024-03-04T16:10:07.059772+0000 mon.a (mon.0) 3678 : audit 1 from='client.? 172.21.15.117:0/3503630643' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "app": "rados"}]: dispatch 2024-03-04T16:10:08.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:07 smithi151 bash[23756]: cluster 2024-03-04T16:10:07.301312+0000 mgr.y (mgr.24428) 776 : cluster 0 pgmap v1299: 188 pgs: 12 creating+peering, 176 active+clean; 745 KiB data, 803 MiB used, 714 GiB / 715 GiB avail; 1.3 KiB/s rd, 6.2 KiB/s wr, 1 op/s 2024-03-04T16:10:08.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:07 smithi151 bash[23756]: audit 2024-03-04T16:10:07.346566+0000 mgr.y (mgr.24428) 777 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:10:08.968 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:08.965+0000 7fa4abfff640 1 -- 172.21.15.117:0/341278432 <== mon.2 v2:172.21.15.117:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool d70eaa68-ebfd-4a58-9771-a0ceff99a3ca v786) v1 ==== 223+0+0 (secure 0 0 0) 0x7fa4bc0164f0 con 0x7fa4c41003e0 2024-03-04T16:10:09.078 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:09.077+0000 7fa4c9129640 1 -- 172.21.15.117:0/341278432 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7fa4c41a7430 con 0x7fa4c41003e0 2024-03-04T16:10:09.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:08 smithi117 bash[21480]: audit 2024-03-04T16:10:07.943074+0000 mon.a (mon.0) 3679 : audit 1 from='client.? 172.21.15.117:0/3503630643' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "app": "rados"}]': finished 2024-03-04T16:10:09.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:08 smithi117 bash[21480]: cluster 2024-03-04T16:10:07.950046+0000 mon.a (mon.0) 3680 : cluster 0 osdmap e785: 8 total, 8 up, 8 in 2024-03-04T16:10:09.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:08 smithi117 bash[21480]: audit 2024-03-04T16:10:08.272779+0000 mon.c (mon.2) 843 : audit 1 from='client.? 172.21.15.117:0/341278432' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"}]: dispatch 2024-03-04T16:10:09.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:08 smithi117 bash[21480]: audit 2024-03-04T16:10:08.273419+0000 mon.a (mon.0) 3681 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"}]: dispatch 2024-03-04T16:10:09.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:08 smithi117 bash[28843]: audit 2024-03-04T16:10:07.943074+0000 mon.a (mon.0) 3679 : audit 1 from='client.? 172.21.15.117:0/3503630643' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "app": "rados"}]': finished 2024-03-04T16:10:09.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:08 smithi117 bash[28843]: cluster 2024-03-04T16:10:07.950046+0000 mon.a (mon.0) 3680 : cluster 0 osdmap e785: 8 total, 8 up, 8 in 2024-03-04T16:10:09.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:08 smithi117 bash[28843]: audit 2024-03-04T16:10:08.272779+0000 mon.c (mon.2) 843 : audit 1 from='client.? 172.21.15.117:0/341278432' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"}]: dispatch 2024-03-04T16:10:09.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:08 smithi117 bash[28843]: audit 2024-03-04T16:10:08.273419+0000 mon.a (mon.0) 3681 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"}]: dispatch 2024-03-04T16:10:09.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:08 smithi151 bash[23756]: audit 2024-03-04T16:10:07.943074+0000 mon.a (mon.0) 3679 : audit 1 from='client.? 172.21.15.117:0/3503630643' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "app": "rados"}]': finished 2024-03-04T16:10:09.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:08 smithi151 bash[23756]: cluster 2024-03-04T16:10:07.950046+0000 mon.a (mon.0) 3680 : cluster 0 osdmap e785: 8 total, 8 up, 8 in 2024-03-04T16:10:09.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:08 smithi151 bash[23756]: audit 2024-03-04T16:10:08.272779+0000 mon.c (mon.2) 843 : audit 1 from='client.? 172.21.15.117:0/341278432' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"}]: dispatch 2024-03-04T16:10:09.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:08 smithi151 bash[23756]: audit 2024-03-04T16:10:08.273419+0000 mon.a (mon.0) 3681 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"}]: dispatch 2024-03-04T16:10:09.985 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:09.981+0000 7fa4abfff640 1 -- 172.21.15.117:0/341278432 <== mon.2 v2:172.21.15.117:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool d70eaa68-ebfd-4a58-9771-a0ceff99a3ca v787) v1 ==== 223+0+0 (secure 0 0 0) 0x7fa4bc0f5b30 con 0x7fa4c41003e0 2024-03-04T16:10:09.985 INFO:tasks.workunit.client.0.smithi117.stderr:set-quota max_objects = 10 for pool d70eaa68-ebfd-4a58-9771-a0ceff99a3ca 2024-03-04T16:10:09.988 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:09.985+0000 7fa4c9129640 1 -- 172.21.15.117:0/341278432 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fa4ac06d700 msgr2=0x7fa4ac06fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:09.989 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:09.985+0000 7fa4c9129640 1 --2- 172.21.15.117:0/341278432 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fa4ac06d700 0x7fa4ac06fbc0 secure :-1 s=READY pgs=3237 cs=0 l=1 rev1=1 crypto rx=0x7fa4b8005d10 tx=0x7fa4b8005c60 comp rx=0 tx=0).stop 2024-03-04T16:10:09.989 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:09.985+0000 7fa4c9129640 1 -- 172.21.15.117:0/341278432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4c41003e0 msgr2=0x7fa4c419a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:10:09.989 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:09.985+0000 7fa4c9129640 1 --2- 172.21.15.117:0/341278432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4c41003e0 0x7fa4c419a490 secure :-1 s=READY pgs=3131 cs=0 l=1 rev1=1 crypto rx=0x7fa4bc0a8020 tx=0x7fa4bc0ad6f0 comp rx=0 tx=0).stop 2024-03-04T16:10:09.989 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:09.985+0000 7fa4c9129640 1 -- 172.21.15.117:0/341278432 shutdown_connections 2024-03-04T16:10:09.989 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:09.985+0000 7fa4c9129640 1 --2- 172.21.15.117:0/341278432 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7fa4ac06d700 0x7fa4ac06fbc0 unknown :-1 s=CLOSED pgs=3237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:09.989 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:09.985+0000 7fa4c9129640 1 --2- 172.21.15.117:0/341278432 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7fa4c4105510 0x7fa4c419ed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:09.989 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:09.985+0000 7fa4c9129640 1 --2- 172.21.15.117:0/341278432 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7fa4c4100d90 0x7fa4c419a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:09.989 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:09.985+0000 7fa4c9129640 1 --2- 172.21.15.117:0/341278432 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7fa4c41003e0 0x7fa4c419a490 unknown :-1 s=CLOSED pgs=3131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:10:09.989 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:09.985+0000 7fa4c9129640 1 -- 172.21.15.117:0/341278432 >> 172.21.15.117:0/341278432 conn(0x7fa4c40fbcb0 msgr2=0x7fa4c4103fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:10:09.989 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:09.985+0000 7fa4c9129640 1 -- 172.21.15.117:0/341278432 shutdown_connections 2024-03-04T16:10:09.989 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:10:09.985+0000 7fa4c9129640 1 -- 172.21.15.117:0/341278432 wait complete. 2024-03-04T16:10:10.008 INFO:tasks.workunit.client.0.smithi117.stderr:+ sleep 30 2024-03-04T16:10:10.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:09 smithi117 bash[21480]: audit 2024-03-04T16:10:08.963974+0000 mon.a (mon.0) 3682 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"}]': finished 2024-03-04T16:10:10.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:09 smithi117 bash[21480]: cluster 2024-03-04T16:10:08.971214+0000 mon.a (mon.0) 3683 : cluster 0 osdmap e786: 8 total, 8 up, 8 in 2024-03-04T16:10:10.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:09 smithi117 bash[21480]: audit 2024-03-04T16:10:09.082350+0000 mon.c (mon.2) 844 : audit 1 from='client.? 172.21.15.117:0/341278432' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"}]: dispatch 2024-03-04T16:10:10.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:09 smithi117 bash[21480]: audit 2024-03-04T16:10:09.082910+0000 mon.a (mon.0) 3684 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"}]: dispatch 2024-03-04T16:10:10.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:09 smithi117 bash[21480]: cluster 2024-03-04T16:10:09.302202+0000 mgr.y (mgr.24428) 778 : cluster 0 pgmap v1302: 188 pgs: 12 creating+peering, 176 active+clean; 745 KiB data, 803 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-04T16:10:10.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:09 smithi117 bash[28843]: audit 2024-03-04T16:10:08.963974+0000 mon.a (mon.0) 3682 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"}]': finished 2024-03-04T16:10:10.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:09 smithi117 bash[28843]: cluster 2024-03-04T16:10:08.971214+0000 mon.a (mon.0) 3683 : cluster 0 osdmap e786: 8 total, 8 up, 8 in 2024-03-04T16:10:10.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:09 smithi117 bash[28843]: audit 2024-03-04T16:10:09.082350+0000 mon.c (mon.2) 844 : audit 1 from='client.? 172.21.15.117:0/341278432' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"}]: dispatch 2024-03-04T16:10:10.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:09 smithi117 bash[28843]: audit 2024-03-04T16:10:09.082910+0000 mon.a (mon.0) 3684 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"}]: dispatch 2024-03-04T16:10:10.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:09 smithi117 bash[28843]: cluster 2024-03-04T16:10:09.302202+0000 mgr.y (mgr.24428) 778 : cluster 0 pgmap v1302: 188 pgs: 12 creating+peering, 176 active+clean; 745 KiB data, 803 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-04T16:10:10.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:09 smithi151 bash[23756]: audit 2024-03-04T16:10:08.963974+0000 mon.a (mon.0) 3682 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"}]': finished 2024-03-04T16:10:10.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:09 smithi151 bash[23756]: cluster 2024-03-04T16:10:08.971214+0000 mon.a (mon.0) 3683 : cluster 0 osdmap e786: 8 total, 8 up, 8 in 2024-03-04T16:10:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:09 smithi151 bash[23756]: audit 2024-03-04T16:10:09.082350+0000 mon.c (mon.2) 844 : audit 1 from='client.? 172.21.15.117:0/341278432' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"}]: dispatch 2024-03-04T16:10:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:09 smithi151 bash[23756]: audit 2024-03-04T16:10:09.082910+0000 mon.a (mon.0) 3684 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"}]: dispatch 2024-03-04T16:10:10.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:09 smithi151 bash[23756]: cluster 2024-03-04T16:10:09.302202+0000 mgr.y (mgr.24428) 778 : cluster 0 pgmap v1302: 188 pgs: 12 creating+peering, 176 active+clean; 745 KiB data, 803 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.0 KiB/s wr, 1 op/s 2024-03-04T16:10:11.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:10 smithi117 bash[21480]: audit 2024-03-04T16:10:09.974894+0000 mon.a (mon.0) 3685 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"}]': finished 2024-03-04T16:10:11.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:10 smithi117 bash[21480]: cluster 2024-03-04T16:10:09.987897+0000 mon.a (mon.0) 3686 : cluster 0 osdmap e787: 8 total, 8 up, 8 in 2024-03-04T16:10:11.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:10 smithi117 bash[28843]: audit 2024-03-04T16:10:09.974894+0000 mon.a (mon.0) 3685 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"}]': finished 2024-03-04T16:10:11.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:10 smithi117 bash[28843]: cluster 2024-03-04T16:10:09.987897+0000 mon.a (mon.0) 3686 : cluster 0 osdmap e787: 8 total, 8 up, 8 in 2024-03-04T16:10:11.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:10 smithi151 bash[23756]: audit 2024-03-04T16:10:09.974894+0000 mon.a (mon.0) 3685 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "field": "max_objects", "val": "10"}]': finished 2024-03-04T16:10:11.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:10 smithi151 bash[23756]: cluster 2024-03-04T16:10:09.987897+0000 mon.a (mon.0) 3686 : cluster 0 osdmap e787: 8 total, 8 up, 8 in 2024-03-04T16:10:12.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:11 smithi117 bash[21480]: cluster 2024-03-04T16:10:11.303028+0000 mgr.y (mgr.24428) 779 : cluster 0 pgmap v1304: 188 pgs: 12 creating+peering, 176 active+clean; 745 KiB data, 803 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:10:12.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:11 smithi117 bash[28843]: cluster 2024-03-04T16:10:11.303028+0000 mgr.y (mgr.24428) 779 : cluster 0 pgmap v1304: 188 pgs: 12 creating+peering, 176 active+clean; 745 KiB data, 803 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:10:12.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:11 smithi151 bash[23756]: cluster 2024-03-04T16:10:11.303028+0000 mgr.y (mgr.24428) 779 : cluster 0 pgmap v1304: 188 pgs: 12 creating+peering, 176 active+clean; 745 KiB data, 803 MiB used, 714 GiB / 715 GiB avail 2024-03-04T16:10:14.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:14 smithi151 bash[23756]: cluster 2024-03-04T16:10:13.305049+0000 mgr.y (mgr.24428) 780 : cluster 0 pgmap v1305: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:14.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:14 smithi117 bash[21480]: cluster 2024-03-04T16:10:13.305049+0000 mgr.y (mgr.24428) 780 : cluster 0 pgmap v1305: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:14.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:14 smithi117 bash[28843]: cluster 2024-03-04T16:10:13.305049+0000 mgr.y (mgr.24428) 780 : cluster 0 pgmap v1305: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:16.364 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:10:16 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:10:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:10:16.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:16 smithi151 bash[23756]: cluster 2024-03-04T16:10:15.305812+0000 mgr.y (mgr.24428) 781 : cluster 0 pgmap v1306: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 695 B/s rd, 0 op/s 2024-03-04T16:10:16.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:16 smithi151 bash[23756]: audit 2024-03-04T16:10:15.639776+0000 mon.a (mon.0) 3687 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:10:16.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:16 smithi151 bash[23756]: audit 2024-03-04T16:10:15.642720+0000 mon.c (mon.2) 845 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:10:16.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:16 smithi117 bash[21480]: cluster 2024-03-04T16:10:15.305812+0000 mgr.y (mgr.24428) 781 : cluster 0 pgmap v1306: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 695 B/s rd, 0 op/s 2024-03-04T16:10:16.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:16 smithi117 bash[21480]: audit 2024-03-04T16:10:15.639776+0000 mon.a (mon.0) 3687 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:10:16.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:16 smithi117 bash[21480]: audit 2024-03-04T16:10:15.642720+0000 mon.c (mon.2) 845 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:10:16.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:16 smithi117 bash[28843]: cluster 2024-03-04T16:10:15.305812+0000 mgr.y (mgr.24428) 781 : cluster 0 pgmap v1306: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 695 B/s rd, 0 op/s 2024-03-04T16:10:16.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:16 smithi117 bash[28843]: audit 2024-03-04T16:10:15.639776+0000 mon.a (mon.0) 3687 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:10:16.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:16 smithi117 bash[28843]: audit 2024-03-04T16:10:15.642720+0000 mon.c (mon.2) 845 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:10:17.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:10:17 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:10:18.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:18 smithi151 bash[23756]: cluster 2024-03-04T16:10:17.307827+0000 mgr.y (mgr.24428) 782 : cluster 0 pgmap v1307: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:10:18.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:18 smithi151 bash[23756]: audit 2024-03-04T16:10:17.357645+0000 mgr.y (mgr.24428) 783 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:10:18.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:18 smithi117 bash[21480]: cluster 2024-03-04T16:10:17.307827+0000 mgr.y (mgr.24428) 782 : cluster 0 pgmap v1307: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:10:18.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:18 smithi117 bash[21480]: audit 2024-03-04T16:10:17.357645+0000 mgr.y (mgr.24428) 783 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:10:18.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:18 smithi117 bash[28843]: cluster 2024-03-04T16:10:17.307827+0000 mgr.y (mgr.24428) 782 : cluster 0 pgmap v1307: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:10:18.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:18 smithi117 bash[28843]: audit 2024-03-04T16:10:17.357645+0000 mgr.y (mgr.24428) 783 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:10:20.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:20 smithi151 bash[23756]: cluster 2024-03-04T16:10:19.308642+0000 mgr.y (mgr.24428) 784 : cluster 0 pgmap v1308: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:10:20.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:20 smithi117 bash[21480]: cluster 2024-03-04T16:10:19.308642+0000 mgr.y (mgr.24428) 784 : cluster 0 pgmap v1308: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:10:20.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:20 smithi117 bash[28843]: cluster 2024-03-04T16:10:19.308642+0000 mgr.y (mgr.24428) 784 : cluster 0 pgmap v1308: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:10:22.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:22 smithi151 bash[23756]: cluster 2024-03-04T16:10:21.309486+0000 mgr.y (mgr.24428) 785 : cluster 0 pgmap v1309: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 903 B/s rd, 0 op/s 2024-03-04T16:10:22.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:22 smithi117 bash[21480]: cluster 2024-03-04T16:10:21.309486+0000 mgr.y (mgr.24428) 785 : cluster 0 pgmap v1309: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 903 B/s rd, 0 op/s 2024-03-04T16:10:22.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:22 smithi117 bash[28843]: cluster 2024-03-04T16:10:21.309486+0000 mgr.y (mgr.24428) 785 : cluster 0 pgmap v1309: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 903 B/s rd, 0 op/s 2024-03-04T16:10:24.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:24 smithi151 bash[23756]: cluster 2024-03-04T16:10:23.311442+0000 mgr.y (mgr.24428) 786 : cluster 0 pgmap v1310: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:10:24.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:24 smithi117 bash[21480]: cluster 2024-03-04T16:10:23.311442+0000 mgr.y (mgr.24428) 786 : cluster 0 pgmap v1310: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:10:24.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:24 smithi117 bash[28843]: cluster 2024-03-04T16:10:23.311442+0000 mgr.y (mgr.24428) 786 : cluster 0 pgmap v1310: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:10:26.389 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:10:26 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:10:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:10:26.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:26 smithi151 bash[23756]: cluster 2024-03-04T16:10:25.312253+0000 mgr.y (mgr.24428) 787 : cluster 0 pgmap v1311: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:26.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:26 smithi117 bash[21480]: cluster 2024-03-04T16:10:25.312253+0000 mgr.y (mgr.24428) 787 : cluster 0 pgmap v1311: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:26.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:26 smithi117 bash[28843]: cluster 2024-03-04T16:10:25.312253+0000 mgr.y (mgr.24428) 787 : cluster 0 pgmap v1311: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:27.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:10:27 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:10:28.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:28 smithi151 bash[23756]: cluster 2024-03-04T16:10:27.314359+0000 mgr.y (mgr.24428) 788 : cluster 0 pgmap v1312: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:10:28.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:28 smithi151 bash[23756]: audit 2024-03-04T16:10:27.363710+0000 mgr.y (mgr.24428) 789 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:10:28.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:28 smithi117 bash[21480]: cluster 2024-03-04T16:10:27.314359+0000 mgr.y (mgr.24428) 788 : cluster 0 pgmap v1312: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:10:28.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:28 smithi117 bash[21480]: audit 2024-03-04T16:10:27.363710+0000 mgr.y (mgr.24428) 789 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:10:28.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:28 smithi117 bash[28843]: cluster 2024-03-04T16:10:27.314359+0000 mgr.y (mgr.24428) 788 : cluster 0 pgmap v1312: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:10:28.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:28 smithi117 bash[28843]: audit 2024-03-04T16:10:27.363710+0000 mgr.y (mgr.24428) 789 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:10:30.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:30 smithi151 bash[23756]: cluster 2024-03-04T16:10:29.315304+0000 mgr.y (mgr.24428) 790 : cluster 0 pgmap v1313: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:30.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:30 smithi117 bash[21480]: cluster 2024-03-04T16:10:29.315304+0000 mgr.y (mgr.24428) 790 : cluster 0 pgmap v1313: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:30.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:30 smithi117 bash[28843]: cluster 2024-03-04T16:10:29.315304+0000 mgr.y (mgr.24428) 790 : cluster 0 pgmap v1313: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:31.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:31 smithi151 bash[23756]: audit 2024-03-04T16:10:30.675964+0000 mon.c (mon.2) 846 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:10:31.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:31 smithi117 bash[21480]: audit 2024-03-04T16:10:30.675964+0000 mon.c (mon.2) 846 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:10:31.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:31 smithi117 bash[28843]: audit 2024-03-04T16:10:30.675964+0000 mon.c (mon.2) 846 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:10:32.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:32 smithi151 bash[23756]: cluster 2024-03-04T16:10:31.316073+0000 mgr.y (mgr.24428) 791 : cluster 0 pgmap v1314: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:32.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:32 smithi117 bash[21480]: cluster 2024-03-04T16:10:31.316073+0000 mgr.y (mgr.24428) 791 : cluster 0 pgmap v1314: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:32.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:32 smithi117 bash[28843]: cluster 2024-03-04T16:10:31.316073+0000 mgr.y (mgr.24428) 791 : cluster 0 pgmap v1314: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:34.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:34 smithi151 bash[23756]: cluster 2024-03-04T16:10:33.318063+0000 mgr.y (mgr.24428) 792 : cluster 0 pgmap v1315: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:10:34.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:34 smithi117 bash[21480]: cluster 2024-03-04T16:10:33.318063+0000 mgr.y (mgr.24428) 792 : cluster 0 pgmap v1315: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:10:34.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:34 smithi117 bash[28843]: cluster 2024-03-04T16:10:33.318063+0000 mgr.y (mgr.24428) 792 : cluster 0 pgmap v1315: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:10:36.417 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:10:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:10:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:10:36.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:36 smithi151 bash[23756]: cluster 2024-03-04T16:10:35.318830+0000 mgr.y (mgr.24428) 793 : cluster 0 pgmap v1316: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:36.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:36 smithi117 bash[21480]: cluster 2024-03-04T16:10:35.318830+0000 mgr.y (mgr.24428) 793 : cluster 0 pgmap v1316: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:36.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:36 smithi117 bash[28843]: cluster 2024-03-04T16:10:35.318830+0000 mgr.y (mgr.24428) 793 : cluster 0 pgmap v1316: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:37.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:10:37 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:10:38.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:38 smithi151 bash[23756]: cluster 2024-03-04T16:10:37.320862+0000 mgr.y (mgr.24428) 794 : cluster 0 pgmap v1317: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:10:38.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:38 smithi151 bash[23756]: audit 2024-03-04T16:10:37.367224+0000 mgr.y (mgr.24428) 795 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:10:38.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:38 smithi117 bash[21480]: cluster 2024-03-04T16:10:37.320862+0000 mgr.y (mgr.24428) 794 : cluster 0 pgmap v1317: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:10:38.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:38 smithi117 bash[21480]: audit 2024-03-04T16:10:37.367224+0000 mgr.y (mgr.24428) 795 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:10:38.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:38 smithi117 bash[28843]: cluster 2024-03-04T16:10:37.320862+0000 mgr.y (mgr.24428) 794 : cluster 0 pgmap v1317: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:10:38.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:38 smithi117 bash[28843]: audit 2024-03-04T16:10:37.367224+0000 mgr.y (mgr.24428) 795 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:10:40.010 INFO:tasks.workunit.client.0.smithi117.stderr:+ seq 1 10 2024-03-04T16:10:40.011 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p d70eaa68-ebfd-4a58-9771-a0ceff99a3ca put obj1 /etc/passwd 2024-03-04T16:10:40.058 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p d70eaa68-ebfd-4a58-9771-a0ceff99a3ca put obj2 /etc/passwd 2024-03-04T16:10:40.099 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p d70eaa68-ebfd-4a58-9771-a0ceff99a3ca put obj3 /etc/passwd 2024-03-04T16:10:40.138 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p d70eaa68-ebfd-4a58-9771-a0ceff99a3ca put obj4 /etc/passwd 2024-03-04T16:10:40.177 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p d70eaa68-ebfd-4a58-9771-a0ceff99a3ca put obj5 /etc/passwd 2024-03-04T16:10:40.216 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p d70eaa68-ebfd-4a58-9771-a0ceff99a3ca put obj6 /etc/passwd 2024-03-04T16:10:40.253 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p d70eaa68-ebfd-4a58-9771-a0ceff99a3ca put obj7 /etc/passwd 2024-03-04T16:10:40.290 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p d70eaa68-ebfd-4a58-9771-a0ceff99a3ca put obj8 /etc/passwd 2024-03-04T16:10:40.328 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p d70eaa68-ebfd-4a58-9771-a0ceff99a3ca put obj9 /etc/passwd 2024-03-04T16:10:40.366 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p d70eaa68-ebfd-4a58-9771-a0ceff99a3ca put obj10 /etc/passwd 2024-03-04T16:10:40.402 INFO:tasks.workunit.client.0.smithi117.stderr:+ sleep 30 2024-03-04T16:10:40.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:40 smithi151 bash[23756]: cluster 2024-03-04T16:10:39.321691+0000 mgr.y (mgr.24428) 796 : cluster 0 pgmap v1318: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:40.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:40 smithi117 bash[21480]: cluster 2024-03-04T16:10:39.321691+0000 mgr.y (mgr.24428) 796 : cluster 0 pgmap v1318: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:40.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:40 smithi117 bash[28843]: cluster 2024-03-04T16:10:39.321691+0000 mgr.y (mgr.24428) 796 : cluster 0 pgmap v1318: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:41.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:41 smithi151 bash[23756]: cluster 2024-03-04T16:10:41.322495+0000 mgr.y (mgr.24428) 797 : cluster 0 pgmap v1319: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:41.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:41 smithi117 bash[28843]: cluster 2024-03-04T16:10:41.322495+0000 mgr.y (mgr.24428) 797 : cluster 0 pgmap v1319: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:41.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:41 smithi117 bash[21480]: cluster 2024-03-04T16:10:41.322495+0000 mgr.y (mgr.24428) 797 : cluster 0 pgmap v1319: 188 pgs: 188 active+clean; 745 KiB data, 821 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:10:44.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:44 smithi151 bash[23756]: cluster 2024-03-04T16:10:43.324471+0000 mgr.y (mgr.24428) 798 : cluster 0 pgmap v1320: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-03-04T16:10:44.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:44 smithi117 bash[28843]: cluster 2024-03-04T16:10:43.324471+0000 mgr.y (mgr.24428) 798 : cluster 0 pgmap v1320: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-03-04T16:10:44.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:44 smithi117 bash[21480]: cluster 2024-03-04T16:10:43.324471+0000 mgr.y (mgr.24428) 798 : cluster 0 pgmap v1320: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 10 KiB/s wr, 2 op/s 2024-03-04T16:10:45.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:45 smithi151 bash[23756]: cluster 2024-03-04T16:10:44.931239+0000 mon.a (mon.0) 3688 : cluster 3 pool 'd70eaa68-ebfd-4a58-9771-a0ceff99a3ca' is full (reached quota's max_objects: 10) 2024-03-04T16:10:45.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:45 smithi151 bash[23756]: cluster 2024-03-04T16:10:44.931911+0000 mon.a (mon.0) 3689 : cluster 3 Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-04T16:10:45.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:45 smithi151 bash[23756]: cluster 2024-03-04T16:10:44.942218+0000 mon.a (mon.0) 3690 : cluster 0 osdmap e788: 8 total, 8 up, 8 in 2024-03-04T16:10:45.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:45 smithi117 bash[28843]: cluster 2024-03-04T16:10:44.931239+0000 mon.a (mon.0) 3688 : cluster 3 pool 'd70eaa68-ebfd-4a58-9771-a0ceff99a3ca' is full (reached quota's max_objects: 10) 2024-03-04T16:10:45.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:45 smithi117 bash[28843]: cluster 2024-03-04T16:10:44.931911+0000 mon.a (mon.0) 3689 : cluster 3 Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-04T16:10:45.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:45 smithi117 bash[28843]: cluster 2024-03-04T16:10:44.942218+0000 mon.a (mon.0) 3690 : cluster 0 osdmap e788: 8 total, 8 up, 8 in 2024-03-04T16:10:45.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:45 smithi117 bash[21480]: cluster 2024-03-04T16:10:44.931239+0000 mon.a (mon.0) 3688 : cluster 3 pool 'd70eaa68-ebfd-4a58-9771-a0ceff99a3ca' is full (reached quota's max_objects: 10) 2024-03-04T16:10:45.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:45 smithi117 bash[21480]: cluster 2024-03-04T16:10:44.931911+0000 mon.a (mon.0) 3689 : cluster 3 Health check failed: 1 pool(s) full (POOL_FULL) 2024-03-04T16:10:45.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:45 smithi117 bash[21480]: cluster 2024-03-04T16:10:44.942218+0000 mon.a (mon.0) 3690 : cluster 0 osdmap e788: 8 total, 8 up, 8 in 2024-03-04T16:10:46.383 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:10:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:10:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:10:46.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:46 smithi151 bash[23756]: cluster 2024-03-04T16:10:45.325299+0000 mgr.y (mgr.24428) 799 : cluster 0 pgmap v1322: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-04T16:10:46.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:46 smithi151 bash[23756]: audit 2024-03-04T16:10:45.716676+0000 mon.a (mon.0) 3691 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:10:46.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:46 smithi151 bash[23756]: audit 2024-03-04T16:10:45.719606+0000 mon.c (mon.2) 847 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:10:46.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:46 smithi117 bash[28843]: cluster 2024-03-04T16:10:45.325299+0000 mgr.y (mgr.24428) 799 : cluster 0 pgmap v1322: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-04T16:10:46.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:46 smithi117 bash[28843]: audit 2024-03-04T16:10:45.716676+0000 mon.a (mon.0) 3691 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:10:46.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:46 smithi117 bash[28843]: audit 2024-03-04T16:10:45.719606+0000 mon.c (mon.2) 847 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:10:46.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:46 smithi117 bash[21480]: cluster 2024-03-04T16:10:45.325299+0000 mgr.y (mgr.24428) 799 : cluster 0 pgmap v1322: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-04T16:10:46.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:46 smithi117 bash[21480]: audit 2024-03-04T16:10:45.716676+0000 mon.a (mon.0) 3691 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:10:46.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:46 smithi117 bash[21480]: audit 2024-03-04T16:10:45.719606+0000 mon.c (mon.2) 847 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:10:47.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:10:47 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:10:48.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:48 smithi151 bash[23756]: cluster 2024-03-04T16:10:47.327813+0000 mgr.y (mgr.24428) 800 : cluster 0 pgmap v1323: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-04T16:10:48.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:48 smithi151 bash[23756]: audit 2024-03-04T16:10:47.378716+0000 mgr.y (mgr.24428) 801 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:10:48.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:48 smithi117 bash[28843]: cluster 2024-03-04T16:10:47.327813+0000 mgr.y (mgr.24428) 800 : cluster 0 pgmap v1323: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-04T16:10:48.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:48 smithi117 bash[28843]: audit 2024-03-04T16:10:47.378716+0000 mgr.y (mgr.24428) 801 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:10:48.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:48 smithi117 bash[21480]: cluster 2024-03-04T16:10:47.327813+0000 mgr.y (mgr.24428) 800 : cluster 0 pgmap v1323: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-04T16:10:48.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:48 smithi117 bash[21480]: audit 2024-03-04T16:10:47.378716+0000 mgr.y (mgr.24428) 801 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:10:50.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:50 smithi151 bash[23756]: cluster 2024-03-04T16:10:49.328673+0000 mgr.y (mgr.24428) 802 : cluster 0 pgmap v1324: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-04T16:10:50.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:50 smithi117 bash[21480]: cluster 2024-03-04T16:10:49.328673+0000 mgr.y (mgr.24428) 802 : cluster 0 pgmap v1324: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-04T16:10:50.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:50 smithi117 bash[28843]: cluster 2024-03-04T16:10:49.328673+0000 mgr.y (mgr.24428) 802 : cluster 0 pgmap v1324: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-04T16:10:52.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:52 smithi151 bash[23756]: cluster 2024-03-04T16:10:51.329516+0000 mgr.y (mgr.24428) 803 : cluster 0 pgmap v1325: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-04T16:10:52.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:52 smithi117 bash[28843]: cluster 2024-03-04T16:10:51.329516+0000 mgr.y (mgr.24428) 803 : cluster 0 pgmap v1325: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-04T16:10:52.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:52 smithi117 bash[21480]: cluster 2024-03-04T16:10:51.329516+0000 mgr.y (mgr.24428) 803 : cluster 0 pgmap v1325: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 12 KiB/s wr, 1 op/s 2024-03-04T16:10:54.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:54 smithi151 bash[23756]: cluster 2024-03-04T16:10:53.331510+0000 mgr.y (mgr.24428) 804 : cluster 0 pgmap v1326: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:10:54.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:54 smithi117 bash[21480]: cluster 2024-03-04T16:10:53.331510+0000 mgr.y (mgr.24428) 804 : cluster 0 pgmap v1326: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:10:54.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:54 smithi117 bash[28843]: cluster 2024-03-04T16:10:53.331510+0000 mgr.y (mgr.24428) 804 : cluster 0 pgmap v1326: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-03-04T16:10:56.406 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:10:56 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:10:56] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:10:56.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:56 smithi151 bash[23756]: cluster 2024-03-04T16:10:55.332322+0000 mgr.y (mgr.24428) 805 : cluster 0 pgmap v1327: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 985 B/s rd, 0 op/s 2024-03-04T16:10:56.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:56 smithi151 bash[23756]: audit 2024-03-04T16:10:56.355931+0000 mon.c (mon.2) 848 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:10:56.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:56 smithi117 bash[28843]: cluster 2024-03-04T16:10:55.332322+0000 mgr.y (mgr.24428) 805 : cluster 0 pgmap v1327: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 985 B/s rd, 0 op/s 2024-03-04T16:10:56.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:56 smithi117 bash[28843]: audit 2024-03-04T16:10:56.355931+0000 mon.c (mon.2) 848 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:10:56.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:56 smithi117 bash[21480]: cluster 2024-03-04T16:10:55.332322+0000 mgr.y (mgr.24428) 805 : cluster 0 pgmap v1327: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 985 B/s rd, 0 op/s 2024-03-04T16:10:56.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:56 smithi117 bash[21480]: audit 2024-03-04T16:10:56.355931+0000 mon.c (mon.2) 848 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-03-04T16:10:57.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:10:57 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:10:58.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:57 smithi117 bash[21480]: audit 2024-03-04T16:10:56.942702+0000 mon.a (mon.0) 3692 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:10:58.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:57 smithi117 bash[21480]: audit 2024-03-04T16:10:56.963106+0000 mon.a (mon.0) 3693 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:10:58.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:57 smithi117 bash[21480]: cluster 2024-03-04T16:10:57.334271+0000 mgr.y (mgr.24428) 806 : cluster 0 pgmap v1328: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:10:58.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:57 smithi117 bash[21480]: audit 2024-03-04T16:10:57.381816+0000 mgr.y (mgr.24428) 807 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:10:58.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:57 smithi117 bash[21480]: audit 2024-03-04T16:10:57.555814+0000 mon.c (mon.2) 849 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:10:58.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:57 smithi117 bash[21480]: audit 2024-03-04T16:10:57.557199+0000 mon.c (mon.2) 850 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:10:58.319 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:10:57 smithi117 bash[21480]: audit 2024-03-04T16:10:57.566914+0000 mon.a (mon.0) 3694 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:10:58.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:57 smithi117 bash[28843]: audit 2024-03-04T16:10:56.942702+0000 mon.a (mon.0) 3692 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:10:58.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:57 smithi117 bash[28843]: audit 2024-03-04T16:10:56.963106+0000 mon.a (mon.0) 3693 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:10:58.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:57 smithi117 bash[28843]: cluster 2024-03-04T16:10:57.334271+0000 mgr.y (mgr.24428) 806 : cluster 0 pgmap v1328: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:10:58.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:57 smithi117 bash[28843]: audit 2024-03-04T16:10:57.381816+0000 mgr.y (mgr.24428) 807 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:10:58.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:57 smithi117 bash[28843]: audit 2024-03-04T16:10:57.555814+0000 mon.c (mon.2) 849 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:10:58.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:57 smithi117 bash[28843]: audit 2024-03-04T16:10:57.557199+0000 mon.c (mon.2) 850 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:10:58.319 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:10:57 smithi117 bash[28843]: audit 2024-03-04T16:10:57.566914+0000 mon.a (mon.0) 3694 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:10:58.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:57 smithi151 bash[23756]: audit 2024-03-04T16:10:56.942702+0000 mon.a (mon.0) 3692 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:10:58.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:57 smithi151 bash[23756]: audit 2024-03-04T16:10:56.963106+0000 mon.a (mon.0) 3693 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:10:58.431 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:57 smithi151 bash[23756]: cluster 2024-03-04T16:10:57.334271+0000 mgr.y (mgr.24428) 806 : cluster 0 pgmap v1328: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:10:58.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:57 smithi151 bash[23756]: audit 2024-03-04T16:10:57.381816+0000 mgr.y (mgr.24428) 807 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:10:58.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:57 smithi151 bash[23756]: audit 2024-03-04T16:10:57.555814+0000 mon.c (mon.2) 849 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-03-04T16:10:58.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:57 smithi151 bash[23756]: audit 2024-03-04T16:10:57.557199+0000 mon.c (mon.2) 850 : audit 1 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-03-04T16:10:58.432 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:10:57 smithi151 bash[23756]: audit 2024-03-04T16:10:57.566914+0000 mon.a (mon.0) 3694 : audit 1 from='mgr.24428 ' entity='mgr.y' 2024-03-04T16:11:00.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:00 smithi151 bash[23756]: cluster 2024-03-04T16:10:59.335075+0000 mgr.y (mgr.24428) 808 : cluster 0 pgmap v1329: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:00.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:00 smithi117 bash[28843]: cluster 2024-03-04T16:10:59.335075+0000 mgr.y (mgr.24428) 808 : cluster 0 pgmap v1329: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:00.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:00 smithi117 bash[21480]: cluster 2024-03-04T16:10:59.335075+0000 mgr.y (mgr.24428) 808 : cluster 0 pgmap v1329: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:01.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:01 smithi151 bash[23756]: audit 2024-03-04T16:11:00.752731+0000 mon.c (mon.2) 851 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:11:01.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:01 smithi117 bash[28843]: audit 2024-03-04T16:11:00.752731+0000 mon.c (mon.2) 851 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:11:01.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:01 smithi117 bash[21480]: audit 2024-03-04T16:11:00.752731+0000 mon.c (mon.2) 851 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:11:02.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:02 smithi151 bash[23756]: cluster 2024-03-04T16:11:01.335870+0000 mgr.y (mgr.24428) 809 : cluster 0 pgmap v1330: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:02.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:02 smithi117 bash[21480]: cluster 2024-03-04T16:11:01.335870+0000 mgr.y (mgr.24428) 809 : cluster 0 pgmap v1330: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:02.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:02 smithi117 bash[28843]: cluster 2024-03-04T16:11:01.335870+0000 mgr.y (mgr.24428) 809 : cluster 0 pgmap v1330: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:04.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:04 smithi151 bash[23756]: cluster 2024-03-04T16:11:03.337889+0000 mgr.y (mgr.24428) 810 : cluster 0 pgmap v1331: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:11:04.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:04 smithi117 bash[21480]: cluster 2024-03-04T16:11:03.337889+0000 mgr.y (mgr.24428) 810 : cluster 0 pgmap v1331: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:11:04.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:04 smithi117 bash[28843]: cluster 2024-03-04T16:11:03.337889+0000 mgr.y (mgr.24428) 810 : cluster 0 pgmap v1331: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:11:06.408 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:11:06 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:11:06] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:11:06.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:06 smithi151 bash[23756]: cluster 2024-03-04T16:11:05.338672+0000 mgr.y (mgr.24428) 811 : cluster 0 pgmap v1332: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:06.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:06 smithi117 bash[21480]: cluster 2024-03-04T16:11:05.338672+0000 mgr.y (mgr.24428) 811 : cluster 0 pgmap v1332: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:06.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:06 smithi117 bash[28843]: cluster 2024-03-04T16:11:05.338672+0000 mgr.y (mgr.24428) 811 : cluster 0 pgmap v1332: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:07.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:11:07 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:11:08.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:08 smithi151 bash[23756]: cluster 2024-03-04T16:11:07.340650+0000 mgr.y (mgr.24428) 812 : cluster 0 pgmap v1333: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:11:08.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:08 smithi151 bash[23756]: audit 2024-03-04T16:11:07.393213+0000 mgr.y (mgr.24428) 813 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:11:08.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:08 smithi117 bash[21480]: cluster 2024-03-04T16:11:07.340650+0000 mgr.y (mgr.24428) 812 : cluster 0 pgmap v1333: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:11:08.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:08 smithi117 bash[21480]: audit 2024-03-04T16:11:07.393213+0000 mgr.y (mgr.24428) 813 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:11:08.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:08 smithi117 bash[28843]: cluster 2024-03-04T16:11:07.340650+0000 mgr.y (mgr.24428) 812 : cluster 0 pgmap v1333: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:11:08.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:08 smithi117 bash[28843]: audit 2024-03-04T16:11:07.393213+0000 mgr.y (mgr.24428) 813 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:11:10.403 INFO:tasks.workunit.client.0.smithi117.stderr:+ rados -p 736f5599-4143-4934-af9d-08c0b0a4b1a3 put threemore /etc/passwd 2024-03-04T16:11:10.451 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph osd pool set-quota 736f5599-4143-4934-af9d-08c0b0a4b1a3 max_bytes 0 2024-03-04T16:11:10.573 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b24ef6640 1 -- 172.21.15.117:0/2498740113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b201120c0 msgr2=0x7f6b201144b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:11:10.574 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b24ef6640 1 --2- 172.21.15.117:0/2498740113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b201120c0 0x7f6b201144b0 secure :-1 s=READY pgs=3145 cs=0 l=1 rev1=1 crypto rx=0x7f6b10009930 tx=0x7f6b1001c7e0 comp rx=0 tx=0).stop 2024-03-04T16:11:10.574 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b24ef6640 1 -- 172.21.15.117:0/2498740113 shutdown_connections 2024-03-04T16:11:10.574 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b24ef6640 1 --2- 172.21.15.117:0/2498740113 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b201120c0 0x7f6b201144b0 unknown :-1 s=CLOSED pgs=3145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:10.574 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b24ef6640 1 --2- 172.21.15.117:0/2498740113 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b20100550 0x7f6b20111b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:10.574 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b24ef6640 1 --2- 172.21.15.117:0/2498740113 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b200ffc30 0x7f6b20100010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:10.574 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b24ef6640 1 -- 172.21.15.117:0/2498740113 >> 172.21.15.117:0/2498740113 conn(0x7f6b20075450 msgr2=0x7f6b20075860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:11:10.574 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b24ef6640 1 -- 172.21.15.117:0/2498740113 shutdown_connections 2024-03-04T16:11:10.574 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b24ef6640 1 -- 172.21.15.117:0/2498740113 wait complete. 2024-03-04T16:11:10.574 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b24ef6640 1 Processor -- start 2024-03-04T16:11:10.574 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b24ef6640 1 -- start start 2024-03-04T16:11:10.574 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b24ef6640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b200ffc30 0x7f6b2019fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:11:10.575 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b24ef6640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b20100550 0x7f6b201a0390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:11:10.575 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b24ef6640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b201120c0 0x7f6b201a4720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:11:10.575 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b24ef6640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b201198c0 con 0x7f6b20100550 2024-03-04T16:11:10.575 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b24ef6640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b20119740 con 0x7f6b200ffc30 2024-03-04T16:11:10.575 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b24ef6640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b20119a40 con 0x7f6b201120c0 2024-03-04T16:11:10.575 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b1dd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b20100550 0x7f6b201a0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:11:10.575 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b1ed9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b201120c0 0x7f6b201a4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:11:10.575 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b1ed9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b201120c0 0x7f6b201a4720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:44752/0 (socket says 172.21.15.117:44752) 2024-03-04T16:11:10.575 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b1dd9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b20100550 0x7f6b201a0390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57096/0 (socket says 172.21.15.117:57096) 2024-03-04T16:11:10.575 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b1ed9d640 1 -- 172.21.15.117:0/3455227129 learned_addr learned my addr 172.21.15.117:0/3455227129 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T16:11:10.575 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b1e59c640 1 --2- 172.21.15.117:0/3455227129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b200ffc30 0x7f6b2019fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:11:10.575 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b1ed9d640 1 -- 172.21.15.117:0/3455227129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b200ffc30 msgr2=0x7f6b2019fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:11:10.575 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b1ed9d640 1 --2- 172.21.15.117:0/3455227129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b200ffc30 0x7f6b2019fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:10.575 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b1ed9d640 1 -- 172.21.15.117:0/3455227129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b20100550 msgr2=0x7f6b201a0390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:11:10.576 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b1ed9d640 1 --2- 172.21.15.117:0/3455227129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b20100550 0x7f6b201a0390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:10.576 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b1ed9d640 1 -- 172.21.15.117:0/3455227129 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b201a4e00 con 0x7f6b201120c0 2024-03-04T16:11:10.576 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b1dd9b640 1 --2- 172.21.15.117:0/3455227129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b20100550 0x7f6b201a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T16:11:10.576 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b1ed9d640 1 --2- 172.21.15.117:0/3455227129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b201120c0 0x7f6b201a4720 secure :-1 s=READY pgs=3146 cs=0 l=1 rev1=1 crypto rx=0x7f6b1001c610 tx=0x7f6b100ad6f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:11:10.576 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b177fe640 1 -- 172.21.15.117:0/3455227129 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b100b2040 con 0x7f6b201120c0 2024-03-04T16:11:10.576 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b177fe640 1 -- 172.21.15.117:0/3455227129 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f6b100a5a60 con 0x7f6b201120c0 2024-03-04T16:11:10.576 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b24ef6640 1 -- 172.21.15.117:0/3455227129 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b201a5090 con 0x7f6b201120c0 2024-03-04T16:11:10.576 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b177fe640 1 -- 172.21.15.117:0/3455227129 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b10005460 con 0x7f6b201120c0 2024-03-04T16:11:10.576 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.572+0000 7f6b24ef6640 1 -- 172.21.15.117:0/3455227129 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b201a54e0 con 0x7f6b201120c0 2024-03-04T16:11:10.577 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.576+0000 7f6b24ef6640 1 -- 172.21.15.117:0/3455227129 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b20068630 con 0x7f6b201120c0 2024-03-04T16:11:10.577 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.576+0000 7f6b177fe640 1 -- 172.21.15.117:0/3455227129 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f6b100bd060 con 0x7f6b201120c0 2024-03-04T16:11:10.577 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.576+0000 7f6b177fe640 1 --2- 172.21.15.117:0/3455227129 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f6b0c06d700 0x7f6b0c06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:11:10.577 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.576+0000 7f6b177fe640 1 -- 172.21.15.117:0/3455227129 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(788..788 src has 1..788) v4 ==== 9456+0+0 (secure 0 0 0) 0x7f6b10127690 con 0x7f6b201120c0 2024-03-04T16:11:10.577 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.576+0000 7f6b1e59c640 1 --2- 172.21.15.117:0/3455227129 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f6b0c06d700 0x7f6b0c06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:11:10.577 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.576+0000 7f6b177fe640 1 -- 172.21.15.117:0/3455227129 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=789}) v3 -- 0x7f6b0c0799b0 con 0x7f6b201120c0 2024-03-04T16:11:10.578 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.576+0000 7f6b1e59c640 1 --2- 172.21.15.117:0/3455227129 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f6b0c06d700 0x7f6b0c06fbc0 secure :-1 s=READY pgs=3249 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009f60 tx=0x7f6b18009500 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:11:10.580 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.580+0000 7f6b177fe640 1 -- 172.21.15.117:0/3455227129 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f6b100164f0 con 0x7f6b201120c0 2024-03-04T16:11:10.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:10 smithi151 bash[23756]: cluster 2024-03-04T16:11:09.341446+0000 mgr.y (mgr.24428) 814 : cluster 0 pgmap v1334: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:10.750 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:10.752+0000 7f6b24ef6640 1 -- 172.21.15.117:0/3455227129 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f6b201aca90 con 0x7f6b201120c0 2024-03-04T16:11:10.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:10 smithi117 bash[21480]: cluster 2024-03-04T16:11:09.341446+0000 mgr.y (mgr.24428) 814 : cluster 0 pgmap v1334: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:10.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:10 smithi117 bash[28843]: cluster 2024-03-04T16:11:09.341446+0000 mgr.y (mgr.24428) 814 : cluster 0 pgmap v1334: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:11.436 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:11.436+0000 7f6b177fe640 1 -- 172.21.15.117:0/3455227129 <== mon.2 v2:172.21.15.117:3301/0 7 ==== osd_map(789..789 src has 1..789) v4 ==== 628+0+0 (secure 0 0 0) 0x7f6b1015ad20 con 0x7f6b201120c0 2024-03-04T16:11:11.436 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:11.436+0000 7f6b177fe640 1 -- 172.21.15.117:0/3455227129 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=790}) v3 -- 0x7f6b0c07a520 con 0x7f6b201120c0 2024-03-04T16:11:11.444 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:11.444+0000 7f6b177fe640 1 -- 172.21.15.117:0/3455227129 <== mon.2 v2:172.21.15.117:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 v789) v1 ==== 217+0+0 (secure 0 0 0) 0x7f6b100f4720 con 0x7f6b201120c0 2024-03-04T16:11:11.549 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:11.548+0000 7f6b24ef6640 1 -- 172.21.15.117:0/3455227129 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f6b20061980 con 0x7f6b201120c0 2024-03-04T16:11:11.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:11 smithi151 bash[23756]: audit 2024-03-04T16:11:10.754489+0000 mon.c (mon.2) 852 : audit 1 from='client.? 172.21.15.117:0/3455227129' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:11:11.682 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:11 smithi151 bash[23756]: audit 2024-03-04T16:11:10.755218+0000 mon.a (mon.0) 3695 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:11:11.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:11 smithi117 bash[21480]: audit 2024-03-04T16:11:10.754489+0000 mon.c (mon.2) 852 : audit 1 from='client.? 172.21.15.117:0/3455227129' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:11:11.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:11 smithi117 bash[21480]: audit 2024-03-04T16:11:10.755218+0000 mon.a (mon.0) 3695 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:11:11.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:11 smithi117 bash[28843]: audit 2024-03-04T16:11:10.754489+0000 mon.c (mon.2) 852 : audit 1 from='client.? 172.21.15.117:0/3455227129' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:11:11.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:11 smithi117 bash[28843]: audit 2024-03-04T16:11:10.755218+0000 mon.a (mon.0) 3695 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:11:12.438 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.440+0000 7f6b177fe640 1 -- 172.21.15.117:0/3455227129 <== mon.2 v2:172.21.15.117:3301/0 9 ==== osd_map(790..790 src has 1..790) v4 ==== 628+0+0 (secure 0 0 0) 0x7f6b100f20c0 con 0x7f6b201120c0 2024-03-04T16:11:12.438 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.440+0000 7f6b177fe640 1 -- 172.21.15.117:0/3455227129 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=791}) v3 -- 0x7f6b0c07a990 con 0x7f6b201120c0 2024-03-04T16:11:12.447 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.448+0000 7f6b177fe640 1 -- 172.21.15.117:0/3455227129 <== mon.2 v2:172.21.15.117:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 v790) v1 ==== 217+0+0 (secure 0 0 0) 0x7f6b100f8170 con 0x7f6b201120c0 2024-03-04T16:11:12.447 INFO:tasks.workunit.client.0.smithi117.stderr:set-quota max_bytes = 0 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 2024-03-04T16:11:12.451 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.452+0000 7f6b24ef6640 1 -- 172.21.15.117:0/3455227129 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f6b0c06d700 msgr2=0x7f6b0c06fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:11:12.451 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.452+0000 7f6b24ef6640 1 --2- 172.21.15.117:0/3455227129 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f6b0c06d700 0x7f6b0c06fbc0 secure :-1 s=READY pgs=3249 cs=0 l=1 rev1=1 crypto rx=0x7f6b18009f60 tx=0x7f6b18009500 comp rx=0 tx=0).stop 2024-03-04T16:11:12.451 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.452+0000 7f6b24ef6640 1 -- 172.21.15.117:0/3455227129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b201120c0 msgr2=0x7f6b201a4720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:11:12.451 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.452+0000 7f6b24ef6640 1 --2- 172.21.15.117:0/3455227129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b201120c0 0x7f6b201a4720 secure :-1 s=READY pgs=3146 cs=0 l=1 rev1=1 crypto rx=0x7f6b1001c610 tx=0x7f6b100ad6f0 comp rx=0 tx=0).stop 2024-03-04T16:11:12.451 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.452+0000 7f6b24ef6640 1 -- 172.21.15.117:0/3455227129 shutdown_connections 2024-03-04T16:11:12.451 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.452+0000 7f6b24ef6640 1 --2- 172.21.15.117:0/3455227129 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f6b0c06d700 0x7f6b0c06fbc0 unknown :-1 s=CLOSED pgs=3249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:12.451 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.452+0000 7f6b24ef6640 1 --2- 172.21.15.117:0/3455227129 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6b20100550 0x7f6b201a0390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:12.451 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.452+0000 7f6b24ef6640 1 --2- 172.21.15.117:0/3455227129 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6b201120c0 0x7f6b201a4720 unknown :-1 s=CLOSED pgs=3146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:12.451 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.452+0000 7f6b24ef6640 1 --2- 172.21.15.117:0/3455227129 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6b200ffc30 0x7f6b2019fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:12.451 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.452+0000 7f6b24ef6640 1 -- 172.21.15.117:0/3455227129 >> 172.21.15.117:0/3455227129 conn(0x7f6b20075450 msgr2=0x7f6b2010fb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:11:12.452 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.452+0000 7f6b24ef6640 1 -- 172.21.15.117:0/3455227129 shutdown_connections 2024-03-04T16:11:12.452 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.452+0000 7f6b24ef6640 1 -- 172.21.15.117:0/3455227129 wait complete. 2024-03-04T16:11:12.472 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph osd pool set-quota 736f5599-4143-4934-af9d-08c0b0a4b1a3 max_objects 0 2024-03-04T16:11:12.597 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cfc975640 1 -- 172.21.15.117:0/598638545 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cf8112130 msgr2=0x7f7cf8114520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:11:12.597 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cfc975640 1 --2- 172.21.15.117:0/598638545 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cf8112130 0x7f7cf8114520 secure :-1 s=READY pgs=3147 cs=0 l=1 rev1=1 crypto rx=0x7f7cf0009ce0 tx=0x7f7cf001cc80 comp rx=0 tx=0).stop 2024-03-04T16:11:12.597 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cfc975640 1 -- 172.21.15.117:0/598638545 shutdown_connections 2024-03-04T16:11:12.597 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cfc975640 1 --2- 172.21.15.117:0/598638545 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cf80761f0 0x7f7cf8111bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:12.598 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cfc975640 1 --2- 172.21.15.117:0/598638545 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cf8112130 0x7f7cf8114520 unknown :-1 s=CLOSED pgs=3147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:12.598 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cfc975640 1 --2- 172.21.15.117:0/598638545 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cf80758d0 0x7f7cf8075cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:12.598 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cfc975640 1 -- 172.21.15.117:0/598638545 >> 172.21.15.117:0/598638545 conn(0x7f7cf80fef40 msgr2=0x7f7cf8101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:11:12.598 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cfc975640 1 -- 172.21.15.117:0/598638545 shutdown_connections 2024-03-04T16:11:12.598 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cfc975640 1 -- 172.21.15.117:0/598638545 wait complete. 2024-03-04T16:11:12.598 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cfc975640 1 Processor -- start 2024-03-04T16:11:12.598 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cfc975640 1 -- start start 2024-03-04T16:11:12.598 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cfc975640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cf80758d0 0x7f7cf819fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:11:12.598 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cfc975640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cf80761f0 0x7f7cf81a03b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:11:12.598 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cfc975640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cf8112130 0x7f7cf81a4740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:11:12.598 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cfc975640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf81198d0 con 0x7f7cf80761f0 2024-03-04T16:11:12.598 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cfc975640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf8119750 con 0x7f7cf80758d0 2024-03-04T16:11:12.598 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cfc975640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf8119a50 con 0x7f7cf8112130 2024-03-04T16:11:12.598 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cf6d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cf8112130 0x7f7cf81a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:11:12.599 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cf5d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cf80761f0 0x7f7cf81a03b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:11:12.599 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cf5d9b640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cf80761f0 0x7f7cf81a03b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:57122/0 (socket says 172.21.15.117:57122) 2024-03-04T16:11:12.599 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cf6d9d640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cf8112130 0x7f7cf81a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:44768/0 (socket says 172.21.15.117:44768) 2024-03-04T16:11:12.599 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cf5d9b640 1 -- 172.21.15.117:0/299485854 learned_addr learned my addr 172.21.15.117:0/299485854 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T16:11:12.599 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cf659c640 1 --2- 172.21.15.117:0/299485854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cf80758d0 0x7f7cf819fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:11:12.599 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cf6d9d640 1 -- 172.21.15.117:0/299485854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cf80758d0 msgr2=0x7f7cf819fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:11:12.599 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cf6d9d640 1 --2- 172.21.15.117:0/299485854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cf80758d0 0x7f7cf819fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:12.599 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cf6d9d640 1 -- 172.21.15.117:0/299485854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cf80761f0 msgr2=0x7f7cf81a03b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:11:12.599 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cf6d9d640 1 --2- 172.21.15.117:0/299485854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cf80761f0 0x7f7cf81a03b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:12.599 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cf6d9d640 1 -- 172.21.15.117:0/299485854 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf81a4e20 con 0x7f7cf8112130 2024-03-04T16:11:12.599 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cf5d9b640 1 --2- 172.21.15.117:0/299485854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cf80761f0 0x7f7cf81a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T16:11:12.599 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cf6d9d640 1 --2- 172.21.15.117:0/299485854 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cf8112130 0x7f7cf81a4740 secure :-1 s=READY pgs=3148 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00b0040 tx=0x7f7cf00045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:11:12.599 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7ce77fe640 1 -- 172.21.15.117:0/299485854 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf00b2040 con 0x7f7cf8112130 2024-03-04T16:11:12.599 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cfc975640 1 -- 172.21.15.117:0/299485854 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf81a50b0 con 0x7f7cf8112130 2024-03-04T16:11:12.599 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7ce77fe640 1 -- 172.21.15.117:0/299485854 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f7cf00047d0 con 0x7f7cf8112130 2024-03-04T16:11:12.600 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7ce77fe640 1 -- 172.21.15.117:0/299485854 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf00ada40 con 0x7f7cf8112130 2024-03-04T16:11:12.600 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.596+0000 7f7cfc975640 1 -- 172.21.15.117:0/299485854 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf81ac9f0 con 0x7f7cf8112130 2024-03-04T16:11:12.601 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.600+0000 7f7cfc975640 1 -- 172.21.15.117:0/299485854 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf80681e0 con 0x7f7cf8112130 2024-03-04T16:11:12.601 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.600+0000 7f7ce77fe640 1 -- 172.21.15.117:0/299485854 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f7cf00adbe0 con 0x7f7cf8112130 2024-03-04T16:11:12.601 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.600+0000 7f7ce77fe640 1 --2- 172.21.15.117:0/299485854 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f7ce006d700 0x7f7ce006fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:11:12.601 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.600+0000 7f7ce77fe640 1 -- 172.21.15.117:0/299485854 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(790..790 src has 1..790) v4 ==== 9456+0+0 (secure 0 0 0) 0x7f7cf01287d0 con 0x7f7cf8112130 2024-03-04T16:11:12.601 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.600+0000 7f7cf659c640 1 --2- 172.21.15.117:0/299485854 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f7ce006d700 0x7f7ce006fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:11:12.601 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.600+0000 7f7ce77fe640 1 -- 172.21.15.117:0/299485854 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=791}) v3 -- 0x7f7ce00799b0 con 0x7f7cf8112130 2024-03-04T16:11:12.601 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.600+0000 7f7cf659c640 1 --2- 172.21.15.117:0/299485854 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f7ce006d700 0x7f7ce006fbc0 secure :-1 s=READY pgs=3250 cs=0 l=1 rev1=1 crypto rx=0x7f7ce80058d0 tx=0x7f7ce8005e30 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:11:12.603 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.604+0000 7f7ce77fe640 1 -- 172.21.15.117:0/299485854 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f7cf0016990 con 0x7f7cf8112130 2024-03-04T16:11:12.774 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:12.776+0000 7f7cfc975640 1 -- 172.21.15.117:0/299485854 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f7cf81acbc0 con 0x7f7cf8112130 2024-03-04T16:11:12.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:12 smithi117 bash[21480]: cluster 2024-03-04T16:11:11.342172+0000 mgr.y (mgr.24428) 815 : cluster 0 pgmap v1335: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:12.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:12 smithi117 bash[21480]: audit 2024-03-04T16:11:11.435095+0000 mon.a (mon.0) 3696 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]': finished 2024-03-04T16:11:12.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:12 smithi117 bash[21480]: cluster 2024-03-04T16:11:11.445872+0000 mon.a (mon.0) 3697 : cluster 0 osdmap e789: 8 total, 8 up, 8 in 2024-03-04T16:11:12.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:12 smithi117 bash[21480]: audit 2024-03-04T16:11:11.552917+0000 mon.c (mon.2) 853 : audit 1 from='client.? 172.21.15.117:0/3455227129' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:11:12.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:12 smithi117 bash[21480]: audit 2024-03-04T16:11:11.553646+0000 mon.a (mon.0) 3698 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:11:12.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:12 smithi117 bash[28843]: cluster 2024-03-04T16:11:11.342172+0000 mgr.y (mgr.24428) 815 : cluster 0 pgmap v1335: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:12.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:12 smithi117 bash[28843]: audit 2024-03-04T16:11:11.435095+0000 mon.a (mon.0) 3696 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]': finished 2024-03-04T16:11:12.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:12 smithi117 bash[28843]: cluster 2024-03-04T16:11:11.445872+0000 mon.a (mon.0) 3697 : cluster 0 osdmap e789: 8 total, 8 up, 8 in 2024-03-04T16:11:12.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:12 smithi117 bash[28843]: audit 2024-03-04T16:11:11.552917+0000 mon.c (mon.2) 853 : audit 1 from='client.? 172.21.15.117:0/3455227129' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:11:12.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:12 smithi117 bash[28843]: audit 2024-03-04T16:11:11.553646+0000 mon.a (mon.0) 3698 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:11:12.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:12 smithi151 bash[23756]: cluster 2024-03-04T16:11:11.342172+0000 mgr.y (mgr.24428) 815 : cluster 0 pgmap v1335: 188 pgs: 188 active+clean; 862 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:12 smithi151 bash[23756]: audit 2024-03-04T16:11:11.435095+0000 mon.a (mon.0) 3696 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]': finished 2024-03-04T16:11:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:12 smithi151 bash[23756]: cluster 2024-03-04T16:11:11.445872+0000 mon.a (mon.0) 3697 : cluster 0 osdmap e789: 8 total, 8 up, 8 in 2024-03-04T16:11:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:12 smithi151 bash[23756]: audit 2024-03-04T16:11:11.552917+0000 mon.c (mon.2) 853 : audit 1 from='client.? 172.21.15.117:0/3455227129' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:11:12.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:12 smithi151 bash[23756]: audit 2024-03-04T16:11:11.553646+0000 mon.a (mon.0) 3698 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]: dispatch 2024-03-04T16:11:13.488 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:13.488+0000 7f7ce77fe640 1 -- 172.21.15.117:0/299485854 <== mon.2 v2:172.21.15.117:3301/0 7 ==== osd_map(791..791 src has 1..791) v4 ==== 628+0+0 (secure 0 0 0) 0x7f7cf015bd20 con 0x7f7cf8112130 2024-03-04T16:11:13.489 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:13.488+0000 7f7ce77fe640 1 -- 172.21.15.117:0/299485854 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=792}) v3 -- 0x7f7ce007a580 con 0x7f7cf8112130 2024-03-04T16:11:13.489 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:13.488+0000 7f7ce77fe640 1 -- 172.21.15.117:0/299485854 <== mon.2 v2:172.21.15.117:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 v791) v1 ==== 221+0+0 (secure 0 0 0) 0x7f7cf00f5860 con 0x7f7cf8112130 2024-03-04T16:11:13.595 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:13.596+0000 7f7cfc975640 1 -- 172.21.15.117:0/299485854 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f7cf8061980 con 0x7f7cf8112130 2024-03-04T16:11:13.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:13 smithi117 bash[21480]: audit 2024-03-04T16:11:12.437632+0000 mon.a (mon.0) 3699 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]': finished 2024-03-04T16:11:13.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:13 smithi117 bash[21480]: cluster 2024-03-04T16:11:12.448183+0000 mon.a (mon.0) 3700 : cluster 0 osdmap e790: 8 total, 8 up, 8 in 2024-03-04T16:11:13.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:13 smithi117 bash[21480]: audit 2024-03-04T16:11:12.777959+0000 mon.c (mon.2) 854 : audit 1 from='client.? 172.21.15.117:0/299485854' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:11:13.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:13 smithi117 bash[21480]: audit 2024-03-04T16:11:12.778525+0000 mon.a (mon.0) 3701 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:11:13.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:13 smithi117 bash[21480]: cluster 2024-03-04T16:11:13.342979+0000 mgr.y (mgr.24428) 816 : cluster 0 pgmap v1338: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-03-04T16:11:13.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:13 smithi117 bash[28843]: audit 2024-03-04T16:11:12.437632+0000 mon.a (mon.0) 3699 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]': finished 2024-03-04T16:11:13.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:13 smithi117 bash[28843]: cluster 2024-03-04T16:11:12.448183+0000 mon.a (mon.0) 3700 : cluster 0 osdmap e790: 8 total, 8 up, 8 in 2024-03-04T16:11:13.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:13 smithi117 bash[28843]: audit 2024-03-04T16:11:12.777959+0000 mon.c (mon.2) 854 : audit 1 from='client.? 172.21.15.117:0/299485854' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:11:13.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:13 smithi117 bash[28843]: audit 2024-03-04T16:11:12.778525+0000 mon.a (mon.0) 3701 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:11:13.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:13 smithi117 bash[28843]: cluster 2024-03-04T16:11:13.342979+0000 mgr.y (mgr.24428) 816 : cluster 0 pgmap v1338: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-03-04T16:11:13.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:13 smithi151 bash[23756]: audit 2024-03-04T16:11:12.437632+0000 mon.a (mon.0) 3699 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_bytes", "val": "0"}]': finished 2024-03-04T16:11:13.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:13 smithi151 bash[23756]: cluster 2024-03-04T16:11:12.448183+0000 mon.a (mon.0) 3700 : cluster 0 osdmap e790: 8 total, 8 up, 8 in 2024-03-04T16:11:13.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:13 smithi151 bash[23756]: audit 2024-03-04T16:11:12.777959+0000 mon.c (mon.2) 854 : audit 1 from='client.? 172.21.15.117:0/299485854' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:11:13.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:13 smithi151 bash[23756]: audit 2024-03-04T16:11:12.778525+0000 mon.a (mon.0) 3701 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:11:13.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:13 smithi151 bash[23756]: cluster 2024-03-04T16:11:13.342979+0000 mgr.y (mgr.24428) 816 : cluster 0 pgmap v1338: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 1 op/s 2024-03-04T16:11:14.476 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:14.476+0000 7f7ce77fe640 1 -- 172.21.15.117:0/299485854 <== mon.2 v2:172.21.15.117:3301/0 9 ==== osd_map(792..792 src has 1..792) v4 ==== 628+0+0 (secure 0 0 0) 0x7f7cf00fb030 con 0x7f7cf8112130 2024-03-04T16:11:14.479 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:14.476+0000 7f7ce77fe640 1 -- 172.21.15.117:0/299485854 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=793}) v3 -- 0x7f7ce007ab40 con 0x7f7cf8112130 2024-03-04T16:11:14.483 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:14.484+0000 7f7ce77fe640 1 -- 172.21.15.117:0/299485854 <== mon.2 v2:172.21.15.117:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 v792) v1 ==== 221+0+0 (secure 0 0 0) 0x7f7cf00f92b0 con 0x7f7cf8112130 2024-03-04T16:11:14.484 INFO:tasks.workunit.client.0.smithi117.stderr:set-quota max_objects = 0 for pool 736f5599-4143-4934-af9d-08c0b0a4b1a3 2024-03-04T16:11:14.488 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:14.488+0000 7f7cfc975640 1 -- 172.21.15.117:0/299485854 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f7ce006d700 msgr2=0x7f7ce006fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:11:14.488 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:14.488+0000 7f7cfc975640 1 --2- 172.21.15.117:0/299485854 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f7ce006d700 0x7f7ce006fbc0 secure :-1 s=READY pgs=3250 cs=0 l=1 rev1=1 crypto rx=0x7f7ce80058d0 tx=0x7f7ce8005e30 comp rx=0 tx=0).stop 2024-03-04T16:11:14.488 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:14.488+0000 7f7cfc975640 1 -- 172.21.15.117:0/299485854 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cf8112130 msgr2=0x7f7cf81a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:11:14.488 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:14.488+0000 7f7cfc975640 1 --2- 172.21.15.117:0/299485854 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cf8112130 0x7f7cf81a4740 secure :-1 s=READY pgs=3148 cs=0 l=1 rev1=1 crypto rx=0x7f7cf00b0040 tx=0x7f7cf00045e0 comp rx=0 tx=0).stop 2024-03-04T16:11:14.489 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:14.488+0000 7f7cfc975640 1 -- 172.21.15.117:0/299485854 shutdown_connections 2024-03-04T16:11:14.489 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:14.488+0000 7f7cfc975640 1 --2- 172.21.15.117:0/299485854 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f7ce006d700 0x7f7ce006fbc0 unknown :-1 s=CLOSED pgs=3250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:14.489 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:14.488+0000 7f7cfc975640 1 --2- 172.21.15.117:0/299485854 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f7cf80761f0 0x7f7cf81a03b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:14.489 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:14.488+0000 7f7cfc975640 1 --2- 172.21.15.117:0/299485854 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f7cf8112130 0x7f7cf81a4740 unknown :-1 s=CLOSED pgs=3148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:14.489 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:14.488+0000 7f7cfc975640 1 --2- 172.21.15.117:0/299485854 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f7cf80758d0 0x7f7cf819fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:14.489 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:14.488+0000 7f7cfc975640 1 -- 172.21.15.117:0/299485854 >> 172.21.15.117:0/299485854 conn(0x7f7cf80fef40 msgr2=0x7f7cf8101330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:11:14.489 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:14.488+0000 7f7cfc975640 1 -- 172.21.15.117:0/299485854 shutdown_connections 2024-03-04T16:11:14.489 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:14.488+0000 7f7cfc975640 1 -- 172.21.15.117:0/299485854 wait complete. 2024-03-04T16:11:14.509 INFO:tasks.workunit.client.0.smithi117.stderr:+ sleep 30 2024-03-04T16:11:14.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:14 smithi117 bash[28843]: audit 2024-03-04T16:11:13.474729+0000 mon.a (mon.0) 3702 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]': finished 2024-03-04T16:11:14.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:14 smithi117 bash[28843]: cluster 2024-03-04T16:11:13.482560+0000 mon.a (mon.0) 3703 : cluster 0 osdmap e791: 8 total, 8 up, 8 in 2024-03-04T16:11:14.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:14 smithi117 bash[28843]: audit 2024-03-04T16:11:13.599774+0000 mon.c (mon.2) 855 : audit 1 from='client.? 172.21.15.117:0/299485854' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:11:14.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:14 smithi117 bash[28843]: audit 2024-03-04T16:11:13.600464+0000 mon.a (mon.0) 3704 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:11:14.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:14 smithi117 bash[21480]: audit 2024-03-04T16:11:13.474729+0000 mon.a (mon.0) 3702 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]': finished 2024-03-04T16:11:14.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:14 smithi117 bash[21480]: cluster 2024-03-04T16:11:13.482560+0000 mon.a (mon.0) 3703 : cluster 0 osdmap e791: 8 total, 8 up, 8 in 2024-03-04T16:11:14.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:14 smithi117 bash[21480]: audit 2024-03-04T16:11:13.599774+0000 mon.c (mon.2) 855 : audit 1 from='client.? 172.21.15.117:0/299485854' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:11:14.819 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:14 smithi117 bash[21480]: audit 2024-03-04T16:11:13.600464+0000 mon.a (mon.0) 3704 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:11:14.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:14 smithi151 bash[23756]: audit 2024-03-04T16:11:13.474729+0000 mon.a (mon.0) 3702 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]': finished 2024-03-04T16:11:14.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:14 smithi151 bash[23756]: cluster 2024-03-04T16:11:13.482560+0000 mon.a (mon.0) 3703 : cluster 0 osdmap e791: 8 total, 8 up, 8 in 2024-03-04T16:11:14.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:14 smithi151 bash[23756]: audit 2024-03-04T16:11:13.599774+0000 mon.c (mon.2) 855 : audit 1 from='client.? 172.21.15.117:0/299485854' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:11:14.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:14 smithi151 bash[23756]: audit 2024-03-04T16:11:13.600464+0000 mon.a (mon.0) 3704 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]: dispatch 2024-03-04T16:11:15.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:15 smithi117 bash[21480]: audit 2024-03-04T16:11:14.477617+0000 mon.a (mon.0) 3705 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]': finished 2024-03-04T16:11:15.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:15 smithi117 bash[21480]: cluster 2024-03-04T16:11:14.484504+0000 mon.a (mon.0) 3706 : cluster 0 osdmap e792: 8 total, 8 up, 8 in 2024-03-04T16:11:15.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:15 smithi117 bash[21480]: cluster 2024-03-04T16:11:15.343816+0000 mgr.y (mgr.24428) 817 : cluster 0 pgmap v1341: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-04T16:11:15.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:15 smithi117 bash[28843]: audit 2024-03-04T16:11:14.477617+0000 mon.a (mon.0) 3705 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]': finished 2024-03-04T16:11:15.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:15 smithi117 bash[28843]: cluster 2024-03-04T16:11:14.484504+0000 mon.a (mon.0) 3706 : cluster 0 osdmap e792: 8 total, 8 up, 8 in 2024-03-04T16:11:15.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:15 smithi117 bash[28843]: cluster 2024-03-04T16:11:15.343816+0000 mgr.y (mgr.24428) 817 : cluster 0 pgmap v1341: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-04T16:11:15.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:15 smithi151 bash[23756]: audit 2024-03-04T16:11:14.477617+0000 mon.a (mon.0) 3705 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "field": "max_objects", "val": "0"}]': finished 2024-03-04T16:11:15.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:15 smithi151 bash[23756]: cluster 2024-03-04T16:11:14.484504+0000 mon.a (mon.0) 3706 : cluster 0 osdmap e792: 8 total, 8 up, 8 in 2024-03-04T16:11:15.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:15 smithi151 bash[23756]: cluster 2024-03-04T16:11:15.343816+0000 mgr.y (mgr.24428) 817 : cluster 0 pgmap v1341: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 3.0 KiB/s wr, 1 op/s 2024-03-04T16:11:16.501 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:11:16 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:11:16] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:11:16.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:16 smithi117 bash[21480]: audit 2024-03-04T16:11:15.785773+0000 mon.c (mon.2) 856 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:11:16.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:16 smithi117 bash[28843]: audit 2024-03-04T16:11:15.785773+0000 mon.c (mon.2) 856 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:11:16.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:16 smithi151 bash[23756]: audit 2024-03-04T16:11:15.785773+0000 mon.c (mon.2) 856 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:11:17.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:11:17 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:11:17.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:17 smithi151 bash[23756]: cluster 2024-03-04T16:11:17.345851+0000 mgr.y (mgr.24428) 818 : cluster 0 pgmap v1342: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-03-04T16:11:17.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:17 smithi151 bash[23756]: audit 2024-03-04T16:11:17.396340+0000 mgr.y (mgr.24428) 819 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:11:17.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:17 smithi117 bash[21480]: cluster 2024-03-04T16:11:17.345851+0000 mgr.y (mgr.24428) 818 : cluster 0 pgmap v1342: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-03-04T16:11:17.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:17 smithi117 bash[21480]: audit 2024-03-04T16:11:17.396340+0000 mgr.y (mgr.24428) 819 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:11:17.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:17 smithi117 bash[28843]: cluster 2024-03-04T16:11:17.345851+0000 mgr.y (mgr.24428) 818 : cluster 0 pgmap v1342: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 1 op/s 2024-03-04T16:11:17.819 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:17 smithi117 bash[28843]: audit 2024-03-04T16:11:17.396340+0000 mgr.y (mgr.24428) 819 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:11:20.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:20 smithi151 bash[23756]: cluster 2024-03-04T16:11:19.346657+0000 mgr.y (mgr.24428) 820 : cluster 0 pgmap v1343: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 741 B/s rd, 0 op/s 2024-03-04T16:11:20.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:20 smithi117 bash[21480]: cluster 2024-03-04T16:11:19.346657+0000 mgr.y (mgr.24428) 820 : cluster 0 pgmap v1343: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 741 B/s rd, 0 op/s 2024-03-04T16:11:20.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:20 smithi117 bash[28843]: cluster 2024-03-04T16:11:19.346657+0000 mgr.y (mgr.24428) 820 : cluster 0 pgmap v1343: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 741 B/s rd, 0 op/s 2024-03-04T16:11:22.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:22 smithi151 bash[23756]: cluster 2024-03-04T16:11:21.347433+0000 mgr.y (mgr.24428) 821 : cluster 0 pgmap v1344: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-04T16:11:22.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:22 smithi117 bash[21480]: cluster 2024-03-04T16:11:21.347433+0000 mgr.y (mgr.24428) 821 : cluster 0 pgmap v1344: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-04T16:11:22.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:22 smithi117 bash[28843]: cluster 2024-03-04T16:11:21.347433+0000 mgr.y (mgr.24428) 821 : cluster 0 pgmap v1344: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-03-04T16:11:24.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:24 smithi151 bash[23756]: cluster 2024-03-04T16:11:23.349467+0000 mgr.y (mgr.24428) 822 : cluster 0 pgmap v1345: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-04T16:11:24.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:24 smithi117 bash[21480]: cluster 2024-03-04T16:11:23.349467+0000 mgr.y (mgr.24428) 822 : cluster 0 pgmap v1345: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-04T16:11:24.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:24 smithi117 bash[28843]: cluster 2024-03-04T16:11:23.349467+0000 mgr.y (mgr.24428) 822 : cluster 0 pgmap v1345: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-03-04T16:11:26.414 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:11:26 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:11:26] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:11:26.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:26 smithi151 bash[23756]: cluster 2024-03-04T16:11:25.350220+0000 mgr.y (mgr.24428) 823 : cluster 0 pgmap v1346: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 942 B/s rd, 0 op/s 2024-03-04T16:11:26.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:26 smithi117 bash[21480]: cluster 2024-03-04T16:11:25.350220+0000 mgr.y (mgr.24428) 823 : cluster 0 pgmap v1346: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 942 B/s rd, 0 op/s 2024-03-04T16:11:26.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:26 smithi117 bash[28843]: cluster 2024-03-04T16:11:25.350220+0000 mgr.y (mgr.24428) 823 : cluster 0 pgmap v1346: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 942 B/s rd, 0 op/s 2024-03-04T16:11:27.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:11:27 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:11:28.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:28 smithi151 bash[23756]: cluster 2024-03-04T16:11:27.352232+0000 mgr.y (mgr.24428) 824 : cluster 0 pgmap v1347: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:11:28.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:28 smithi151 bash[23756]: audit 2024-03-04T16:11:27.407804+0000 mgr.y (mgr.24428) 825 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:11:28.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:28 smithi117 bash[21480]: cluster 2024-03-04T16:11:27.352232+0000 mgr.y (mgr.24428) 824 : cluster 0 pgmap v1347: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:11:28.820 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:28 smithi117 bash[21480]: audit 2024-03-04T16:11:27.407804+0000 mgr.y (mgr.24428) 825 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:11:28.820 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:28 smithi117 bash[28843]: cluster 2024-03-04T16:11:27.352232+0000 mgr.y (mgr.24428) 824 : cluster 0 pgmap v1347: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:11:28.821 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:28 smithi117 bash[28843]: audit 2024-03-04T16:11:27.407804+0000 mgr.y (mgr.24428) 825 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:11:29.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:29 smithi117 bash[21480]: cluster 2024-03-04T16:11:29.353083+0000 mgr.y (mgr.24428) 826 : cluster 0 pgmap v1348: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:29.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:29 smithi117 bash[28843]: cluster 2024-03-04T16:11:29.353083+0000 mgr.y (mgr.24428) 826 : cluster 0 pgmap v1348: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:29.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:29 smithi151 bash[23756]: cluster 2024-03-04T16:11:29.353083+0000 mgr.y (mgr.24428) 826 : cluster 0 pgmap v1348: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:31.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:30 smithi151 bash[23756]: audit 2024-03-04T16:11:30.819225+0000 mon.c (mon.2) 857 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:11:31.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:30 smithi117 bash[21480]: audit 2024-03-04T16:11:30.819225+0000 mon.c (mon.2) 857 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:11:31.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:30 smithi117 bash[28843]: audit 2024-03-04T16:11:30.819225+0000 mon.c (mon.2) 857 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:11:32.181 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:31 smithi151 bash[23756]: cluster 2024-03-04T16:11:31.353833+0000 mgr.y (mgr.24428) 827 : cluster 0 pgmap v1349: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:32.318 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:31 smithi117 bash[21480]: cluster 2024-03-04T16:11:31.353833+0000 mgr.y (mgr.24428) 827 : cluster 0 pgmap v1349: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:32.318 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:31 smithi117 bash[28843]: cluster 2024-03-04T16:11:31.353833+0000 mgr.y (mgr.24428) 827 : cluster 0 pgmap v1349: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:34.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:34 smithi151 bash[23756]: cluster 2024-03-04T16:11:33.355845+0000 mgr.y (mgr.24428) 828 : cluster 0 pgmap v1350: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:11:34.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:34 smithi117 bash[21480]: cluster 2024-03-04T16:11:33.355845+0000 mgr.y (mgr.24428) 828 : cluster 0 pgmap v1350: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:11:34.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:34 smithi117 bash[28843]: cluster 2024-03-04T16:11:33.355845+0000 mgr.y (mgr.24428) 828 : cluster 0 pgmap v1350: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:11:36.415 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:11:36 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:11:36] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:11:36.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:36 smithi151 bash[23756]: cluster 2024-03-04T16:11:35.356693+0000 mgr.y (mgr.24428) 829 : cluster 0 pgmap v1351: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:36.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:36 smithi117 bash[21480]: cluster 2024-03-04T16:11:35.356693+0000 mgr.y (mgr.24428) 829 : cluster 0 pgmap v1351: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:36.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:36 smithi117 bash[28843]: cluster 2024-03-04T16:11:35.356693+0000 mgr.y (mgr.24428) 829 : cluster 0 pgmap v1351: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:37.681 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:11:37 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:11:38.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:38 smithi151 bash[23756]: cluster 2024-03-04T16:11:37.358695+0000 mgr.y (mgr.24428) 830 : cluster 0 pgmap v1352: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:11:38.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:38 smithi151 bash[23756]: audit 2024-03-04T16:11:37.410364+0000 mgr.y (mgr.24428) 831 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:11:38.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:38 smithi117 bash[21480]: cluster 2024-03-04T16:11:37.358695+0000 mgr.y (mgr.24428) 830 : cluster 0 pgmap v1352: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:11:38.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:38 smithi117 bash[21480]: audit 2024-03-04T16:11:37.410364+0000 mgr.y (mgr.24428) 831 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:11:38.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:38 smithi117 bash[28843]: cluster 2024-03-04T16:11:37.358695+0000 mgr.y (mgr.24428) 830 : cluster 0 pgmap v1352: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:11:38.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:38 smithi117 bash[28843]: audit 2024-03-04T16:11:37.410364+0000 mgr.y (mgr.24428) 831 : audit 0 from='client.24442 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-03-04T16:11:40.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:40 smithi151 bash[23756]: cluster 2024-03-04T16:11:39.359502+0000 mgr.y (mgr.24428) 832 : cluster 0 pgmap v1353: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:40.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:40 smithi117 bash[21480]: cluster 2024-03-04T16:11:39.359502+0000 mgr.y (mgr.24428) 832 : cluster 0 pgmap v1353: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:40.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:40 smithi117 bash[28843]: cluster 2024-03-04T16:11:39.359502+0000 mgr.y (mgr.24428) 832 : cluster 0 pgmap v1353: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:42.681 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:42 smithi151 bash[23756]: cluster 2024-03-04T16:11:41.360447+0000 mgr.y (mgr.24428) 833 : cluster 0 pgmap v1354: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:42.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:42 smithi117 bash[21480]: cluster 2024-03-04T16:11:41.360447+0000 mgr.y (mgr.24428) 833 : cluster 0 pgmap v1354: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:42.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:42 smithi117 bash[28843]: cluster 2024-03-04T16:11:41.360447+0000 mgr.y (mgr.24428) 833 : cluster 0 pgmap v1354: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:44.511 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph osd pool delete 736f5599-4143-4934-af9d-08c0b0a4b1a3 736f5599-4143-4934-af9d-08c0b0a4b1a3 --yes-i-really-really-mean-it 2024-03-04T16:11:44.646 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f38889a2640 1 -- 172.21.15.117:0/427171905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38841034b0 msgr2=0x7f3884103930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:11:44.646 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f38889a2640 1 --2- 172.21.15.117:0/427171905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38841034b0 0x7f3884103930 secure :-1 s=READY pgs=3149 cs=0 l=1 rev1=1 crypto rx=0x7f386c009960 tx=0x7f386c01c7c0 comp rx=0 tx=0).stop 2024-03-04T16:11:44.647 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f38889a2640 1 -- 172.21.15.117:0/427171905 shutdown_connections 2024-03-04T16:11:44.647 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f38889a2640 1 --2- 172.21.15.117:0/427171905 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3884103e70 0x7f3884110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:44.647 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f38889a2640 1 --2- 172.21.15.117:0/427171905 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f38841034b0 0x7f3884103930 unknown :-1 s=CLOSED pgs=3149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:44.647 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f38889a2640 1 --2- 172.21.15.117:0/427171905 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f3884109560 0x7f3884109940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:44.647 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f38889a2640 1 -- 172.21.15.117:0/427171905 >> 172.21.15.117:0/427171905 conn(0x7f38840fef40 msgr2=0x7f3884101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:11:44.647 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f38889a2640 1 -- 172.21.15.117:0/427171905 shutdown_connections 2024-03-04T16:11:44.647 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f38889a2640 1 -- 172.21.15.117:0/427171905 wait complete. 2024-03-04T16:11:44.647 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f38889a2640 1 Processor -- start 2024-03-04T16:11:44.647 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f38889a2640 1 -- start start 2024-03-04T16:11:44.647 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f38889a2640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38841034b0 0x7f38841a16c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:11:44.647 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f38889a2640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3884103e70 0x7f38841a1c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:11:44.647 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f38889a2640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3884109560 0x7f388419b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:11:44.648 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f38889a2640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38841154b0 con 0x7f3884109560 2024-03-04T16:11:44.648 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f38889a2640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3884115330 con 0x7f38841034b0 2024-03-04T16:11:44.648 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f38889a2640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3884115630 con 0x7f3884103e70 2024-03-04T16:11:44.648 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f3882d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3884109560 0x7f388419b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:11:44.648 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f3879d9b640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3884103e70 0x7f38841a1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:11:44.648 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f3882d9d640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3884109560 0x7f388419b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:47462/0 (socket says 172.21.15.117:47462) 2024-03-04T16:11:44.648 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f3882d9d640 1 -- 172.21.15.117:0/2431304791 learned_addr learned my addr 172.21.15.117:0/2431304791 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T16:11:44.648 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f388259c640 1 --2- 172.21.15.117:0/2431304791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38841034b0 0x7f38841a16c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:11:44.648 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f3879d9b640 1 -- 172.21.15.117:0/2431304791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38841034b0 msgr2=0x7f38841a16c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:11:44.648 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f3879d9b640 1 --2- 172.21.15.117:0/2431304791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38841034b0 0x7f38841a16c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:44.648 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f3879d9b640 1 -- 172.21.15.117:0/2431304791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3884109560 msgr2=0x7f388419b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:11:44.648 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f3879d9b640 1 --2- 172.21.15.117:0/2431304791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3884109560 0x7f388419b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:44.648 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f3879d9b640 1 -- 172.21.15.117:0/2431304791 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f388419bfb0 con 0x7f3884103e70 2024-03-04T16:11:44.648 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f3882d9d640 1 --2- 172.21.15.117:0/2431304791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3884109560 0x7f388419b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T16:11:44.649 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f3879d9b640 1 --2- 172.21.15.117:0/2431304791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3884103e70 0x7f38841a1c00 secure :-1 s=READY pgs=3150 cs=0 l=1 rev1=1 crypto rx=0x7f386c01c5f0 tx=0x7f386c0adcd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:11:44.649 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f387bfff640 1 -- 172.21.15.117:0/2431304791 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c0bc020 con 0x7f3884103e70 2024-03-04T16:11:44.649 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f387bfff640 1 -- 172.21.15.117:0/2431304791 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f386c0044c0 con 0x7f3884103e70 2024-03-04T16:11:44.649 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.643+0000 7f387bfff640 1 -- 172.21.15.117:0/2431304791 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f386c0b2040 con 0x7f3884103e70 2024-03-04T16:11:44.649 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.647+0000 7f38889a2640 1 -- 172.21.15.117:0/2431304791 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f388419c240 con 0x7f3884103e70 2024-03-04T16:11:44.649 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.647+0000 7f38889a2640 1 -- 172.21.15.117:0/2431304791 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f388419c6a0 con 0x7f3884103e70 2024-03-04T16:11:44.649 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.647+0000 7f38889a2640 1 -- 172.21.15.117:0/2431304791 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38840681e0 con 0x7f3884103e70 2024-03-04T16:11:44.650 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.647+0000 7f387bfff640 1 -- 172.21.15.117:0/2431304791 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f386c0bf070 con 0x7f3884103e70 2024-03-04T16:11:44.650 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.647+0000 7f387bfff640 1 --2- 172.21.15.117:0/2431304791 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f387006d700 0x7f387006fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:11:44.650 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.647+0000 7f387bfff640 1 -- 172.21.15.117:0/2431304791 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(792..792 src has 1..792) v4 ==== 9456+0+0 (secure 0 0 0) 0x7f386c127b10 con 0x7f3884103e70 2024-03-04T16:11:44.650 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.647+0000 7f388259c640 1 --2- 172.21.15.117:0/2431304791 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f387006d700 0x7f387006fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:11:44.650 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.647+0000 7f387bfff640 1 -- 172.21.15.117:0/2431304791 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=793}) v3 -- 0x7f3870079a90 con 0x7f3884103e70 2024-03-04T16:11:44.650 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.647+0000 7f388259c640 1 --2- 172.21.15.117:0/2431304791 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f387006d700 0x7f387006fbc0 secure :-1 s=READY pgs=3251 cs=0 l=1 rev1=1 crypto rx=0x7f38740058d0 tx=0x7f3874005c90 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:11:44.652 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.651+0000 7f387bfff640 1 -- 172.21.15.117:0/2431304791 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f386c0164d0 con 0x7f3884103e70 2024-03-04T16:11:44.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:44 smithi117 bash[28843]: cluster 2024-03-04T16:11:43.362485+0000 mgr.y (mgr.24428) 834 : cluster 0 pgmap v1355: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:11:44.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:44 smithi117 bash[21480]: cluster 2024-03-04T16:11:43.362485+0000 mgr.y (mgr.24428) 834 : cluster 0 pgmap v1355: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:11:44.820 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:44.819+0000 7f38889a2640 1 -- 172.21.15.117:0/2431304791 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pool2": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f38841a8620 con 0x7f3884103e70 2024-03-04T16:11:44.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:44 smithi151 bash[23756]: cluster 2024-03-04T16:11:43.362485+0000 mgr.y (mgr.24428) 834 : cluster 0 pgmap v1355: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-03-04T16:11:45.451 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.451+0000 7f387bfff640 1 -- 172.21.15.117:0/2431304791 <== mon.2 v2:172.21.15.117:3301/0 7 ==== osd_map(793..793 src has 1..793) v4 ==== 296+0+0 (secure 0 0 0) 0x7f386c0f25c0 con 0x7f3884103e70 2024-03-04T16:11:45.451 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.451+0000 7f387bfff640 1 -- 172.21.15.117:0/2431304791 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=794}) v3 -- 0x7f387007a320 con 0x7f3884103e70 2024-03-04T16:11:45.456 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.455+0000 7f387bfff640 1 -- 172.21.15.117:0/2431304791 <== mon.2 v2:172.21.15.117:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pool2": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "yes_i_really_really_mean_it": true}]=0 pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' removed v793) v1 ==== 248+0+0 (secure 0 0 0) 0x7f386c0f4c20 con 0x7f3884103e70 2024-03-04T16:11:45.561 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.559+0000 7f38889a2640 1 -- 172.21.15.117:0/2431304791 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pool2": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f38840619b0 con 0x7f3884103e70 2024-03-04T16:11:45.563 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.563+0000 7f387bfff640 1 -- 172.21.15.117:0/2431304791 <== mon.2 v2:172.21.15.117:3301/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pool2": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "yes_i_really_really_mean_it": true}]=0 pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' does not exist v793) v1 ==== 255+0+0 (secure 0 0 0) 0x7f386c0f8670 con 0x7f3884103e70 2024-03-04T16:11:45.563 INFO:tasks.workunit.client.0.smithi117.stderr:pool '736f5599-4143-4934-af9d-08c0b0a4b1a3' does not exist 2024-03-04T16:11:45.565 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.563+0000 7f38889a2640 1 -- 172.21.15.117:0/2431304791 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f387006d700 msgr2=0x7f387006fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:11:45.565 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.563+0000 7f38889a2640 1 --2- 172.21.15.117:0/2431304791 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f387006d700 0x7f387006fbc0 secure :-1 s=READY pgs=3251 cs=0 l=1 rev1=1 crypto rx=0x7f38740058d0 tx=0x7f3874005c90 comp rx=0 tx=0).stop 2024-03-04T16:11:45.565 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.563+0000 7f38889a2640 1 -- 172.21.15.117:0/2431304791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3884103e70 msgr2=0x7f38841a1c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:11:45.566 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.563+0000 7f38889a2640 1 --2- 172.21.15.117:0/2431304791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3884103e70 0x7f38841a1c00 secure :-1 s=READY pgs=3150 cs=0 l=1 rev1=1 crypto rx=0x7f386c01c5f0 tx=0x7f386c0adcd0 comp rx=0 tx=0).stop 2024-03-04T16:11:45.566 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.563+0000 7f38889a2640 1 -- 172.21.15.117:0/2431304791 shutdown_connections 2024-03-04T16:11:45.566 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.563+0000 7f38889a2640 1 --2- 172.21.15.117:0/2431304791 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f387006d700 0x7f387006fbc0 unknown :-1 s=CLOSED pgs=3251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:45.566 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.563+0000 7f38889a2640 1 --2- 172.21.15.117:0/2431304791 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f3884109560 0x7f388419b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:45.566 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.563+0000 7f38889a2640 1 --2- 172.21.15.117:0/2431304791 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f3884103e70 0x7f38841a1c00 unknown :-1 s=CLOSED pgs=3150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:45.566 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.563+0000 7f38889a2640 1 --2- 172.21.15.117:0/2431304791 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f38841034b0 0x7f38841a16c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:45.566 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.563+0000 7f38889a2640 1 -- 172.21.15.117:0/2431304791 >> 172.21.15.117:0/2431304791 conn(0x7f38840fef40 msgr2=0x7f388410a270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:11:45.566 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.567+0000 7f38889a2640 1 -- 172.21.15.117:0/2431304791 shutdown_connections 2024-03-04T16:11:45.566 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.567+0000 7f38889a2640 1 -- 172.21.15.117:0/2431304791 wait complete. 2024-03-04T16:11:45.579 INFO:tasks.workunit.client.0.smithi117.stderr:+ ceph osd pool delete d70eaa68-ebfd-4a58-9771-a0ceff99a3ca d70eaa68-ebfd-4a58-9771-a0ceff99a3ca --yes-i-really-really-mean-it 2024-03-04T16:11:45.705 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61696ed640 1 -- 172.21.15.117:0/2728116230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6164102d50 msgr2=0x7f616410b240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:11:45.705 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61696ed640 1 --2- 172.21.15.117:0/2728116230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6164102d50 0x7f616410b240 secure :-1 s=READY pgs=3151 cs=0 l=1 rev1=1 crypto rx=0x7f6158009930 tx=0x7f615801c870 comp rx=0 tx=0).stop 2024-03-04T16:11:45.705 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61696ed640 1 -- 172.21.15.117:0/2728116230 shutdown_connections 2024-03-04T16:11:45.705 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61696ed640 1 --2- 172.21.15.117:0/2728116230 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f616410b860 0x7f616410dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:45.705 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61696ed640 1 --2- 172.21.15.117:0/2728116230 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f6164102d50 0x7f616410b240 unknown :-1 s=CLOSED pgs=3151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:45.705 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61696ed640 1 --2- 172.21.15.117:0/2728116230 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6164102430 0x7f6164102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:45.705 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61696ed640 1 -- 172.21.15.117:0/2728116230 >> 172.21.15.117:0/2728116230 conn(0x7f61640fbc60 msgr2=0x7f61640fe080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:11:45.705 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61696ed640 1 -- 172.21.15.117:0/2728116230 shutdown_connections 2024-03-04T16:11:45.705 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61696ed640 1 -- 172.21.15.117:0/2728116230 wait complete. 2024-03-04T16:11:45.705 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61696ed640 1 Processor -- start 2024-03-04T16:11:45.706 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61696ed640 1 -- start start 2024-03-04T16:11:45.706 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61696ed640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6164102430 0x7f6164199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:11:45.706 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61696ed640 1 --2- >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6164102d50 0x7f616419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:11:45.706 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61696ed640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f616410b860 0x7f616419e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:11:45.706 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61696ed640 1 -- --> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6164113900 con 0x7f6164102430 2024-03-04T16:11:45.706 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61696ed640 1 -- --> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6164113780 con 0x7f6164102d50 2024-03-04T16:11:45.706 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61696ed640 1 -- --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6164113a80 con 0x7f616410b860 2024-03-04T16:11:45.706 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f6162ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6164102430 0x7f6164199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:11:45.706 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61637fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f616410b860 0x7f616419e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:11:45.706 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61637fe640 1 --2- >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f616410b860 0x7f616419e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3301/0 says I am v2:172.21.15.117:37344/0 (socket says 172.21.15.117:37344) 2024-03-04T16:11:45.706 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f6162ffd640 1 --2- >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6164102430 0x7f6164199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.117:3300/0 says I am v2:172.21.15.117:47486/0 (socket says 172.21.15.117:47486) 2024-03-04T16:11:45.706 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61637fe640 1 -- 172.21.15.117:0/3528295947 learned_addr learned my addr 172.21.15.117:0/3528295947 (peer_addr_for_me v2:172.21.15.117:0/0) 2024-03-04T16:11:45.706 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61627fc640 1 --2- 172.21.15.117:0/3528295947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6164102d50 0x7f616419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:11:45.706 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61637fe640 1 -- 172.21.15.117:0/3528295947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6164102d50 msgr2=0x7f616419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:11:45.707 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61637fe640 1 --2- 172.21.15.117:0/3528295947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6164102d50 0x7f616419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:45.707 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61637fe640 1 -- 172.21.15.117:0/3528295947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6164102430 msgr2=0x7f6164199f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:11:45.707 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61637fe640 1 --2- 172.21.15.117:0/3528295947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6164102430 0x7f6164199f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:45.707 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61637fe640 1 -- 172.21.15.117:0/3528295947 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f616419ef10 con 0x7f616410b860 2024-03-04T16:11:45.707 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61627fc640 1 --2- 172.21.15.117:0/3528295947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6164102d50 0x7f616419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-03-04T16:11:45.707 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f6162ffd640 1 --2- 172.21.15.117:0/3528295947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6164102430 0x7f6164199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-03-04T16:11:45.707 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61637fe640 1 --2- 172.21.15.117:0/3528295947 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f616410b860 0x7f616419e830 secure :-1 s=READY pgs=3152 cs=0 l=1 rev1=1 crypto rx=0x7f615400e8c0 tx=0x7f615400ed80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:11:45.707 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f614bfff640 1 -- 172.21.15.117:0/3528295947 <== mon.2 v2:172.21.15.117:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f615400ccc0 con 0x7f616410b860 2024-03-04T16:11:45.707 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f614bfff640 1 -- 172.21.15.117:0/3528295947 <== mon.2 v2:172.21.15.117:3301/0 2 ==== config(24 keys) v1 ==== 1042+0+0 (secure 0 0 0) 0x7f615400ce60 con 0x7f616410b860 2024-03-04T16:11:45.707 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61696ed640 1 -- 172.21.15.117:0/3528295947 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f616419f200 con 0x7f616410b860 2024-03-04T16:11:45.707 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f614bfff640 1 -- 172.21.15.117:0/3528295947 <== mon.2 v2:172.21.15.117:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6154004e10 con 0x7f616410b860 2024-03-04T16:11:45.707 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61696ed640 1 -- 172.21.15.117:0/3528295947 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61641a6ae0 con 0x7f616410b860 2024-03-04T16:11:45.707 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.703+0000 7f61696ed640 1 -- 172.21.15.117:0/3528295947 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61640681e0 con 0x7f616410b860 2024-03-04T16:11:45.708 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.707+0000 7f614bfff640 1 -- 172.21.15.117:0/3528295947 <== mon.2 v2:172.21.15.117:3301/0 4 ==== mgrmap(e 21) v1 ==== 91455+0+0 (secure 0 0 0) 0x7f6154004ff0 con 0x7f616410b860 2024-03-04T16:11:45.708 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.707+0000 7f614bfff640 1 --2- 172.21.15.117:0/3528295947 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f614c06d700 0x7f614c06fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-03-04T16:11:45.709 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.707+0000 7f6162ffd640 1 --2- 172.21.15.117:0/3528295947 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f614c06d700 0x7f614c06fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-03-04T16:11:45.709 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.707+0000 7f614bfff640 1 -- 172.21.15.117:0/3528295947 <== mon.2 v2:172.21.15.117:3301/0 5 ==== osd_map(793..793 src has 1..793) v4 ==== 9068+0+0 (secure 0 0 0) 0x7f615408e480 con 0x7f616410b860 2024-03-04T16:11:45.709 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.707+0000 7f614bfff640 1 -- 172.21.15.117:0/3528295947 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=794}) v3 -- 0x7f614c079190 con 0x7f616410b860 2024-03-04T16:11:45.709 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.707+0000 7f6162ffd640 1 --2- 172.21.15.117:0/3528295947 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f614c06d700 0x7f614c06fbc0 secure :-1 s=READY pgs=3252 cs=0 l=1 rev1=1 crypto rx=0x7f615c00a7b0 tx=0x7f615c00d040 comp rx=0 tx=0).ready entity=mgr.24428 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-03-04T16:11:45.711 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.711+0000 7f614bfff640 1 -- 172.21.15.117:0/3528295947 <== mon.2 v2:172.21.15.117:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+183564 (secure 0 0 0) 0x7f615405b690 con 0x7f616410b860 2024-03-04T16:11:45.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:45 smithi117 bash[21480]: audit 2024-03-04T16:11:44.824424+0000 mon.c (mon.2) 858 : audit 1 from='client.? 172.21.15.117:0/2431304791' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pool2": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T16:11:45.818 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:45 smithi117 bash[21480]: audit 2024-03-04T16:11:44.824905+0000 mon.a (mon.0) 3707 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pool2": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T16:11:45.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:45 smithi117 bash[28843]: audit 2024-03-04T16:11:44.824424+0000 mon.c (mon.2) 858 : audit 1 from='client.? 172.21.15.117:0/2431304791' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pool2": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T16:11:45.818 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:45 smithi117 bash[28843]: audit 2024-03-04T16:11:44.824905+0000 mon.a (mon.0) 3707 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pool2": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T16:11:45.881 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:45.879+0000 7f61696ed640 1 -- 172.21.15.117:0/3528295947 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pool2": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f61641a6cb0 con 0x7f616410b860 2024-03-04T16:11:45.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:45 smithi151 bash[23756]: audit 2024-03-04T16:11:44.824424+0000 mon.c (mon.2) 858 : audit 1 from='client.? 172.21.15.117:0/2431304791' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pool2": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T16:11:45.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:45 smithi151 bash[23756]: audit 2024-03-04T16:11:44.824905+0000 mon.a (mon.0) 3707 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pool2": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T16:11:46.438 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:11:46 smithi117 bash[21734]: ::ffff:172.21.15.151 - - [04/Mar/2024:16:11:46] "GET /metrics HTTP/1.1" 503 1620 "" "Prometheus/2.43.0" 2024-03-04T16:11:46.451 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:46.451+0000 7f614bfff640 1 -- 172.21.15.117:0/3528295947 <== mon.2 v2:172.21.15.117:3301/0 7 ==== osd_map(794..794 src has 1..794) v4 ==== 296+0+0 (secure 0 0 0) 0x7f6154059030 con 0x7f616410b860 2024-03-04T16:11:46.451 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:46.451+0000 7f614bfff640 1 -- 172.21.15.117:0/3528295947 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_subscribe({osdmap=795}) v3 -- 0x7f614c079a20 con 0x7f616410b860 2024-03-04T16:11:46.458 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:46.459+0000 7f614bfff640 1 -- 172.21.15.117:0/3528295947 <== mon.2 v2:172.21.15.117:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pool2": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "yes_i_really_really_mean_it": true}]=0 pool 'd70eaa68-ebfd-4a58-9771-a0ceff99a3ca' removed v794) v1 ==== 248+0+0 (secure 0 0 0) 0x7f615405f0e0 con 0x7f616410b860 2024-03-04T16:11:46.568 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:46.567+0000 7f61696ed640 1 -- 172.21.15.117:0/3528295947 --> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] -- mon_command({"prefix": "osd pool delete", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pool2": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f61641a6f30 con 0x7f616410b860 2024-03-04T16:11:46.570 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:46.571+0000 7f614bfff640 1 -- 172.21.15.117:0/3528295947 <== mon.2 v2:172.21.15.117:3301/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pool2": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "yes_i_really_really_mean_it": true}]=0 pool 'd70eaa68-ebfd-4a58-9771-a0ceff99a3ca' does not exist v794) v1 ==== 255+0+0 (secure 0 0 0) 0x7f61540636a0 con 0x7f616410b860 2024-03-04T16:11:46.570 INFO:tasks.workunit.client.0.smithi117.stderr:pool 'd70eaa68-ebfd-4a58-9771-a0ceff99a3ca' does not exist 2024-03-04T16:11:46.573 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:46.571+0000 7f61696ed640 1 -- 172.21.15.117:0/3528295947 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f614c06d700 msgr2=0x7f614c06fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:11:46.573 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:46.571+0000 7f61696ed640 1 --2- 172.21.15.117:0/3528295947 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f614c06d700 0x7f614c06fbc0 secure :-1 s=READY pgs=3252 cs=0 l=1 rev1=1 crypto rx=0x7f615c00a7b0 tx=0x7f615c00d040 comp rx=0 tx=0).stop 2024-03-04T16:11:46.573 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:46.571+0000 7f61696ed640 1 -- 172.21.15.117:0/3528295947 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f616410b860 msgr2=0x7f616419e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-03-04T16:11:46.573 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:46.571+0000 7f61696ed640 1 --2- 172.21.15.117:0/3528295947 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f616410b860 0x7f616419e830 secure :-1 s=READY pgs=3152 cs=0 l=1 rev1=1 crypto rx=0x7f615400e8c0 tx=0x7f615400ed80 comp rx=0 tx=0).stop 2024-03-04T16:11:46.573 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:46.571+0000 7f61696ed640 1 -- 172.21.15.117:0/3528295947 shutdown_connections 2024-03-04T16:11:46.573 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:46.571+0000 7f61696ed640 1 --2- 172.21.15.117:0/3528295947 >> [v2:172.21.15.117:6800/2727720514,v1:172.21.15.117:6801/2727720514] conn(0x7f614c06d700 0x7f614c06fbc0 unknown :-1 s=CLOSED pgs=3252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:46.573 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:46.571+0000 7f61696ed640 1 --2- 172.21.15.117:0/3528295947 >> [v2:172.21.15.117:3301/0,v1:172.21.15.117:6790/0] conn(0x7f616410b860 0x7f616419e830 unknown :-1 s=CLOSED pgs=3152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:46.573 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:46.571+0000 7f61696ed640 1 --2- 172.21.15.117:0/3528295947 >> [v2:172.21.15.151:3300/0,v1:172.21.15.151:6789/0] conn(0x7f6164102d50 0x7f616419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:46.573 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:46.571+0000 7f61696ed640 1 --2- 172.21.15.117:0/3528295947 >> [v2:172.21.15.117:3300/0,v1:172.21.15.117:6789/0] conn(0x7f6164102430 0x7f6164199f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-03-04T16:11:46.573 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:46.571+0000 7f61696ed640 1 -- 172.21.15.117:0/3528295947 >> 172.21.15.117:0/3528295947 conn(0x7f61640fbc60 msgr2=0x7f61641031e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-03-04T16:11:46.573 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:46.571+0000 7f61696ed640 1 -- 172.21.15.117:0/3528295947 shutdown_connections 2024-03-04T16:11:46.573 INFO:tasks.workunit.client.0.smithi117.stderr:2024-03-04T16:11:46.571+0000 7f61696ed640 1 -- 172.21.15.117:0/3528295947 wait complete. 2024-03-04T16:11:46.588 INFO:tasks.workunit.client.0.smithi117.stdout:OK 2024-03-04T16:11:46.589 INFO:tasks.workunit.client.0.smithi117.stderr:+ echo OK 2024-03-04T16:11:46.590 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-03-04T16:11:46.590 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-03-04T16:11:46.646 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-03-04T16:11:46.646 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-03-04T16:11:46.699 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:46 smithi117 bash[21480]: cluster 2024-03-04T16:11:45.363266+0000 mgr.y (mgr.24428) 835 : cluster 0 pgmap v1356: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:46.699 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:46 smithi117 bash[21480]: audit 2024-03-04T16:11:45.449295+0000 mon.a (mon.0) 3708 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pool2": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "yes_i_really_really_mean_it": true}]': finished 2024-03-04T16:11:46.700 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:46 smithi117 bash[21480]: cluster 2024-03-04T16:11:45.459391+0000 mon.a (mon.0) 3709 : cluster 0 osdmap e793: 8 total, 8 up, 8 in 2024-03-04T16:11:46.700 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:46 smithi117 bash[21480]: audit 2024-03-04T16:11:45.565561+0000 mon.c (mon.2) 859 : audit 1 from='client.? 172.21.15.117:0/2431304791' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pool2": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T16:11:46.700 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:46 smithi117 bash[21480]: audit 2024-03-04T16:11:45.566193+0000 mon.a (mon.0) 3710 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pool2": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T16:11:46.700 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:46 smithi117 bash[21480]: audit 2024-03-04T16:11:45.852323+0000 mon.c (mon.2) 860 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:11:46.700 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:46 smithi117 bash[21480]: audit 2024-03-04T16:11:45.884984+0000 mon.c (mon.2) 861 : audit 1 from='client.? 172.21.15.117:0/3528295947' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pool2": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T16:11:46.700 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:46 smithi117 bash[21480]: audit 2024-03-04T16:11:45.885517+0000 mon.a (mon.0) 3711 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pool2": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T16:11:46.701 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:46 smithi117 bash[28843]: cluster 2024-03-04T16:11:45.363266+0000 mgr.y (mgr.24428) 835 : cluster 0 pgmap v1356: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:46.701 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:46 smithi117 bash[28843]: audit 2024-03-04T16:11:45.449295+0000 mon.a (mon.0) 3708 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pool2": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "yes_i_really_really_mean_it": true}]': finished 2024-03-04T16:11:46.701 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:46 smithi117 bash[28843]: cluster 2024-03-04T16:11:45.459391+0000 mon.a (mon.0) 3709 : cluster 0 osdmap e793: 8 total, 8 up, 8 in 2024-03-04T16:11:46.701 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:46 smithi117 bash[28843]: audit 2024-03-04T16:11:45.565561+0000 mon.c (mon.2) 859 : audit 1 from='client.? 172.21.15.117:0/2431304791' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pool2": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T16:11:46.701 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:46 smithi117 bash[28843]: audit 2024-03-04T16:11:45.566193+0000 mon.a (mon.0) 3710 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pool2": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T16:11:46.702 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:46 smithi117 bash[28843]: audit 2024-03-04T16:11:45.852323+0000 mon.c (mon.2) 860 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:11:46.702 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:46 smithi117 bash[28843]: audit 2024-03-04T16:11:45.884984+0000 mon.c (mon.2) 861 : audit 1 from='client.? 172.21.15.117:0/3528295947' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pool2": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T16:11:46.702 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:46 smithi117 bash[28843]: audit 2024-03-04T16:11:45.885517+0000 mon.a (mon.0) 3711 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pool2": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T16:11:46.931 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:46 smithi151 bash[23756]: cluster 2024-03-04T16:11:45.363266+0000 mgr.y (mgr.24428) 835 : cluster 0 pgmap v1356: 188 pgs: 188 active+clean; 874 KiB data, 822 MiB used, 714 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-03-04T16:11:46.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:46 smithi151 bash[23756]: audit 2024-03-04T16:11:45.449295+0000 mon.a (mon.0) 3708 : audit 1 from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pool2": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "yes_i_really_really_mean_it": true}]': finished 2024-03-04T16:11:46.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:46 smithi151 bash[23756]: cluster 2024-03-04T16:11:45.459391+0000 mon.a (mon.0) 3709 : cluster 0 osdmap e793: 8 total, 8 up, 8 in 2024-03-04T16:11:46.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:46 smithi151 bash[23756]: audit 2024-03-04T16:11:45.565561+0000 mon.c (mon.2) 859 : audit 1 from='client.? 172.21.15.117:0/2431304791' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pool2": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T16:11:46.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:46 smithi151 bash[23756]: audit 2024-03-04T16:11:45.566193+0000 mon.a (mon.0) 3710 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "pool2": "736f5599-4143-4934-af9d-08c0b0a4b1a3", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T16:11:46.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:46 smithi151 bash[23756]: audit 2024-03-04T16:11:45.852323+0000 mon.c (mon.2) 860 : audit 0 from='mgr.24428 172.21.15.117:0/2666392279' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-03-04T16:11:46.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:46 smithi151 bash[23756]: audit 2024-03-04T16:11:45.884984+0000 mon.c (mon.2) 861 : audit 1 from='client.? 172.21.15.117:0/3528295947' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pool2": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T16:11:46.932 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:46 smithi151 bash[23756]: audit 2024-03-04T16:11:45.885517+0000 mon.a (mon.0) 3711 : audit 1 from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "pool2": "d70eaa68-ebfd-4a58-9771-a0ceff99a3ca", "yes_i_really_really_mean_it": true}]: dispatch 2024-03-04T16:11:46.981 DEBUG:teuthology.parallel:result is None 2024-03-04T16:11:46.982 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T16:11:47.042 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T16:11:47.043 DEBUG:teuthology.orchestra.run.smithi117:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-03-04T16:11:47.102 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-03-04T16:11:47.102 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-03-04T16:11:47.118 INFO:tasks.cephadm:Teardown begin 2024-03-04T16:11:47.118 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-04T16:11:47.162 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-04T16:11:47.179 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-03-04T16:11:47.179 DEBUG:teuthology.orchestra.run.smithi117:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-04T16:11:47.210 DEBUG:teuthology.orchestra.run.smithi151:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-03-04T16:11:47.228 INFO:tasks.cephadm:Stopping all daemons... 2024-03-04T16:11:47.228 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-03-04T16:11:47.229 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mon.a 2024-03-04T16:11:47.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:11:47 smithi151 bash[48108]: debug there is no tcmu-runner data available 2024-03-04T16:11:47.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:47 smithi117 systemd[1]: Stopping Ceph mon.a for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T16:11:47.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:47 smithi117 bash[21480]: debug 2024-03-04T16:11:47.423+0000 7efe0d091700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-03-04T16:11:47.568 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:47 smithi117 bash[21480]: debug 2024-03-04T16:11:47.423+0000 7efe0d091700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-03-04T16:11:48.909 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mon.a.service' 2024-03-04T16:11:48.920 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:48 smithi117 bash[132650]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-mon-a 2024-03-04T16:11:48.920 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:48 smithi117 bash[132718]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-mon-a 2024-03-04T16:11:48.920 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:48 smithi117 systemd[1]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mon.a.service: Deactivated successfully. 2024-03-04T16:11:48.920 INFO:journalctl@ceph.mon.a.smithi117.stdout:Mar 04 16:11:48 smithi117 systemd[1]: Stopped Ceph mon.a for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T16:11:48.937 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T16:11:48.937 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-03-04T16:11:48.938 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-03-04T16:11:48.938 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mon.c 2024-03-04T16:11:49.159 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:11:49 smithi117 bash[21734]: [04/Mar/2024:16:11:49] ENGINE Bus STOPPING 2024-03-04T16:11:49.159 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:49 smithi117 systemd[1]: Stopping Ceph mon.c for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T16:11:49.159 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:49 smithi117 bash[28843]: debug 2024-03-04T16:11:49.127+0000 7f6be5ea1700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-03-04T16:11:49.159 INFO:journalctl@ceph.mon.c.smithi117.stdout:Mar 04 16:11:49 smithi117 bash[28843]: debug 2024-03-04T16:11:49.127+0000 7f6be5ea1700 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-03-04T16:11:49.261 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:11:49 smithi117 bash[21734]: [04/Mar/2024:16:11:49] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-03-04T16:11:49.261 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:11:49 smithi117 bash[21734]: [04/Mar/2024:16:11:49] ENGINE Bus STOPPED 2024-03-04T16:11:49.261 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:11:49 smithi117 bash[21734]: [04/Mar/2024:16:11:49] ENGINE Bus STARTING 2024-03-04T16:11:49.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:11:49 smithi117 bash[21734]: [04/Mar/2024:16:11:49] ENGINE Serving on http://:::9283 2024-03-04T16:11:49.568 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:11:49 smithi117 bash[21734]: [04/Mar/2024:16:11:49] ENGINE Bus STARTED 2024-03-04T16:11:50.211 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mon.c.service' 2024-03-04T16:11:50.235 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T16:11:50.236 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-03-04T16:11:50.236 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-03-04T16:11:50.236 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mon.b 2024-03-04T16:11:50.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:50 smithi151 systemd[1]: Stopping Ceph mon.b for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T16:11:50.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:50 smithi151 bash[23756]: debug 2024-03-04T16:11:50.420+0000 7f1dcb805700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-stderr=true (PID: 1) UID: 0 2024-03-04T16:11:50.674 INFO:journalctl@ceph.mon.b.smithi151.stdout:Mar 04 16:11:50 smithi151 bash[23756]: debug 2024-03-04T16:11:50.420+0000 7f1dcb805700 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-03-04T16:11:51.284 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mon.b.service' 2024-03-04T16:11:51.317 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T16:11:51.317 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-03-04T16:11:51.317 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-03-04T16:11:51.317 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mgr.y 2024-03-04T16:11:51.671 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:11:51 smithi117 systemd[1]: Stopping Ceph mgr.y for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T16:11:51.671 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:11:51 smithi117 bash[21734]: debug 2024-03-04T16:11:51.451+0000 7f0ea2a5a700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-03-04T16:11:51.671 INFO:journalctl@ceph.mgr.y.smithi117.stdout:Mar 04 16:11:51 smithi117 bash[21734]: debug 2024-03-04T16:11:51.451+0000 7f0ea2a5a700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-03-04T16:11:52.297 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mgr.y.service' 2024-03-04T16:11:52.321 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T16:11:52.321 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-03-04T16:11:52.321 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-03-04T16:11:52.321 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mgr.x 2024-03-04T16:11:52.681 INFO:journalctl@ceph.mgr.x.smithi151.stdout:Mar 04 16:11:52 smithi151 systemd[1]: Stopping Ceph mgr.x for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T16:11:53.336 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@mgr.x.service' 2024-03-04T16:11:53.360 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T16:11:53.361 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-03-04T16:11:53.361 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-03-04T16:11:53.361 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.0 2024-03-04T16:11:53.818 INFO:journalctl@ceph.osd.0.smithi117.stdout:Mar 04 16:11:53 smithi117 systemd[1]: Stopping Ceph osd.0 for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T16:11:53.818 INFO:journalctl@ceph.osd.0.smithi117.stdout:Mar 04 16:11:53 smithi117 bash[31848]: debug 2024-03-04T16:11:53.515+0000 7f1bf97f3700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-03-04T16:11:53.818 INFO:journalctl@ceph.osd.0.smithi117.stdout:Mar 04 16:11:53 smithi117 bash[31848]: debug 2024-03-04T16:11:53.515+0000 7f1bf97f3700 -1 osd.0 794 *** Got signal Terminated *** 2024-03-04T16:11:53.818 INFO:journalctl@ceph.osd.0.smithi117.stdout:Mar 04 16:11:53 smithi117 bash[31848]: debug 2024-03-04T16:11:53.515+0000 7f1bf97f3700 -1 osd.0 794 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T16:11:55.681 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 16:11:55 smithi151 bash[51061]: logger=cleanup t=2024-03-04T16:11:55.385675429Z level=info msg="Completed cleanup jobs" duration=36.116243ms 2024-03-04T16:11:59.847 INFO:journalctl@ceph.osd.0.smithi117.stdout:Mar 04 16:11:59 smithi117 bash[132969]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-osd-0 2024-03-04T16:11:59.848 INFO:journalctl@ceph.osd.0.smithi117.stdout:Mar 04 16:11:59 smithi117 bash[133032]: Error response from daemon: No such container: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-osd-0 2024-03-04T16:12:01.693 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.0.service' 2024-03-04T16:12:01.723 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T16:12:01.723 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-03-04T16:12:01.723 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-03-04T16:12:01.723 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.1 2024-03-04T16:12:01.908 INFO:journalctl@ceph.osd.1.smithi117.stdout:Mar 04 16:12:01 smithi117 systemd[1]: Stopping Ceph osd.1 for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T16:12:02.068 INFO:journalctl@ceph.osd.1.smithi117.stdout:Mar 04 16:12:01 smithi117 bash[37600]: debug 2024-03-04T16:12:01.907+0000 7f3592b5b700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-03-04T16:12:02.068 INFO:journalctl@ceph.osd.1.smithi117.stdout:Mar 04 16:12:01 smithi117 bash[37600]: debug 2024-03-04T16:12:01.907+0000 7f3592b5b700 -1 osd.1 794 *** Got signal Terminated *** 2024-03-04T16:12:02.068 INFO:journalctl@ceph.osd.1.smithi117.stdout:Mar 04 16:12:01 smithi117 bash[37600]: debug 2024-03-04T16:12:01.907+0000 7f3592b5b700 -1 osd.1 794 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T16:12:08.568 INFO:journalctl@ceph.osd.1.smithi117.stdout:Mar 04 16:12:08 smithi117 bash[133169]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-osd-1 2024-03-04T16:12:08.568 INFO:journalctl@ceph.osd.1.smithi117.stdout:Mar 04 16:12:08 smithi117 bash[133234]: Error response from daemon: No such container: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-osd-1 2024-03-04T16:12:10.361 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.1.service' 2024-03-04T16:12:10.388 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T16:12:10.388 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-03-04T16:12:10.388 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-03-04T16:12:10.389 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.2 2024-03-04T16:12:10.818 INFO:journalctl@ceph.osd.2.smithi117.stdout:Mar 04 16:12:10 smithi117 systemd[1]: Stopping Ceph osd.2 for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T16:12:10.818 INFO:journalctl@ceph.osd.2.smithi117.stdout:Mar 04 16:12:10 smithi117 bash[42991]: debug 2024-03-04T16:12:10.575+0000 7f6b990c5700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-03-04T16:12:10.818 INFO:journalctl@ceph.osd.2.smithi117.stdout:Mar 04 16:12:10 smithi117 bash[42991]: debug 2024-03-04T16:12:10.575+0000 7f6b990c5700 -1 osd.2 794 *** Got signal Terminated *** 2024-03-04T16:12:10.818 INFO:journalctl@ceph.osd.2.smithi117.stdout:Mar 04 16:12:10 smithi117 bash[42991]: debug 2024-03-04T16:12:10.575+0000 7f6b990c5700 -1 osd.2 794 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T16:12:16.814 INFO:journalctl@ceph.osd.2.smithi117.stdout:Mar 04 16:12:16 smithi117 bash[133367]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-osd-2 2024-03-04T16:12:16.814 INFO:journalctl@ceph.osd.2.smithi117.stdout:Mar 04 16:12:16 smithi117 bash[133431]: Error response from daemon: No such container: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-osd-2 2024-03-04T16:12:18.694 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.2.service' 2024-03-04T16:12:18.721 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T16:12:18.721 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-03-04T16:12:18.721 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-03-04T16:12:18.721 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.3 2024-03-04T16:12:19.068 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 04 16:12:18 smithi117 systemd[1]: Stopping Ceph osd.3 for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T16:12:19.068 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 04 16:12:18 smithi117 bash[48517]: debug 2024-03-04T16:12:18.907+0000 7f7222c83700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-03-04T16:12:19.068 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 04 16:12:18 smithi117 bash[48517]: debug 2024-03-04T16:12:18.907+0000 7f7222c83700 -1 osd.3 794 *** Got signal Terminated *** 2024-03-04T16:12:19.068 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 04 16:12:18 smithi117 bash[48517]: debug 2024-03-04T16:12:18.907+0000 7f7222c83700 -1 osd.3 794 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T16:12:19.931 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:19 smithi151 bash[38263]: debug 2024-03-04T16:12:19.575+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:20.318 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 04 16:12:19 smithi117 bash[48517]: debug 2024-03-04T16:12:19.851+0000 7f721b26f700 -1 osd.3 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:58.070479+0000 front 2024-03-04T16:11:58.070658+0000 (oldest deadline 2024-03-04T16:12:19.770760+0000) 2024-03-04T16:12:20.854 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:20 smithi151 bash[38263]: debug 2024-03-04T16:12:20.571+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:21.107 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 16:12:20 smithi151 bash[27331]: debug 2024-03-04T16:12:20.847+0000 7fd76626c700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:54.598236+0000 front 2024-03-04T16:11:54.598159+0000 (oldest deadline 2024-03-04T16:12:20.497824+0000) 2024-03-04T16:12:21.318 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 04 16:12:20 smithi117 bash[48517]: debug 2024-03-04T16:12:20.847+0000 7f721b26f700 -1 osd.3 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:58.070479+0000 front 2024-03-04T16:11:58.070658+0000 (oldest deadline 2024-03-04T16:12:19.770760+0000) 2024-03-04T16:12:21.431 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:21 smithi151 bash[43933]: debug 2024-03-04T16:12:21.107+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:21.883 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:21 smithi151 bash[38263]: debug 2024-03-04T16:12:21.583+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:21.883 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 16:12:21 smithi151 bash[27331]: debug 2024-03-04T16:12:21.815+0000 7fd76626c700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:54.598236+0000 front 2024-03-04T16:11:54.598159+0000 (oldest deadline 2024-03-04T16:12:20.497824+0000) 2024-03-04T16:12:22.143 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:21 smithi151 bash[32767]: debug 2024-03-04T16:12:21.875+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:57.729959+0000 front 2024-03-04T16:11:57.729773+0000 (oldest deadline 2024-03-04T16:12:21.229412+0000) 2024-03-04T16:12:22.318 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 04 16:12:21 smithi117 bash[48517]: debug 2024-03-04T16:12:21.879+0000 7f721b26f700 -1 osd.3 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:58.070479+0000 front 2024-03-04T16:11:58.070658+0000 (oldest deadline 2024-03-04T16:12:19.770760+0000) 2024-03-04T16:12:22.431 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:22 smithi151 bash[43933]: debug 2024-03-04T16:12:22.139+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:22.931 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:22 smithi151 bash[38263]: debug 2024-03-04T16:12:22.615+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:22.932 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 16:12:22 smithi151 bash[27331]: debug 2024-03-04T16:12:22.843+0000 7fd76626c700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:54.598236+0000 front 2024-03-04T16:11:54.598159+0000 (oldest deadline 2024-03-04T16:12:20.497824+0000) 2024-03-04T16:12:22.932 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:22 smithi151 bash[32767]: debug 2024-03-04T16:12:22.839+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:57.729959+0000 front 2024-03-04T16:11:57.729773+0000 (oldest deadline 2024-03-04T16:12:21.229412+0000) 2024-03-04T16:12:23.318 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 04 16:12:22 smithi117 bash[48517]: debug 2024-03-04T16:12:22.919+0000 7f721b26f700 -1 osd.3 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:58.070479+0000 front 2024-03-04T16:11:58.070658+0000 (oldest deadline 2024-03-04T16:12:19.770760+0000) 2024-03-04T16:12:23.431 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:23 smithi151 bash[43933]: debug 2024-03-04T16:12:23.143+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:23.931 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:23 smithi151 bash[32767]: debug 2024-03-04T16:12:23.807+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:57.729959+0000 front 2024-03-04T16:11:57.729773+0000 (oldest deadline 2024-03-04T16:12:21.229412+0000) 2024-03-04T16:12:23.932 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:23 smithi151 bash[38263]: debug 2024-03-04T16:12:23.583+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:23.932 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 16:12:23 smithi151 bash[27331]: debug 2024-03-04T16:12:23.807+0000 7fd76626c700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:54.598236+0000 front 2024-03-04T16:11:54.598159+0000 (oldest deadline 2024-03-04T16:12:20.497824+0000) 2024-03-04T16:12:24.431 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:24 smithi151 bash[43933]: debug 2024-03-04T16:12:24.107+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:24.931 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 16:12:24 smithi151 bash[27331]: debug 2024-03-04T16:12:24.839+0000 7fd76626c700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:54.598236+0000 front 2024-03-04T16:11:54.598159+0000 (oldest deadline 2024-03-04T16:12:20.497824+0000) 2024-03-04T16:12:24.931 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:24 smithi151 bash[32767]: debug 2024-03-04T16:12:24.819+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:57.729959+0000 front 2024-03-04T16:11:57.729773+0000 (oldest deadline 2024-03-04T16:12:21.229412+0000) 2024-03-04T16:12:24.931 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:24 smithi151 bash[38263]: debug 2024-03-04T16:12:24.595+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:25.318 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 04 16:12:25 smithi117 bash[133565]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-osd-3 2024-03-04T16:12:25.318 INFO:journalctl@ceph.osd.3.smithi117.stdout:Mar 04 16:12:25 smithi117 bash[133631]: Error response from daemon: No such container: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-osd-3 2024-03-04T16:12:25.431 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:25 smithi151 bash[43933]: debug 2024-03-04T16:12:25.103+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:25.931 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:25 smithi151 bash[32767]: debug 2024-03-04T16:12:25.815+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:57.729959+0000 front 2024-03-04T16:11:57.729773+0000 (oldest deadline 2024-03-04T16:12:21.229412+0000) 2024-03-04T16:12:25.932 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 16:12:25 smithi151 bash[27331]: debug 2024-03-04T16:12:25.843+0000 7fd76626c700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:54.598236+0000 front 2024-03-04T16:11:54.598159+0000 (oldest deadline 2024-03-04T16:12:20.497824+0000) 2024-03-04T16:12:25.932 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:25 smithi151 bash[38263]: debug 2024-03-04T16:12:25.607+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:26.431 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:26 smithi151 bash[43933]: debug 2024-03-04T16:12:26.111+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:26.431 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 16:12:26 smithi151 bash[51804]: ts=2024-03-04T16:12:26.330Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=node msg="Unable to refresh target groups" err="Get \"http://172.21.15.117:8765/sd/prometheus/sd-config?service=node-exporter\": dial tcp 172.21.15.117:8765: connect: connection refused" 2024-03-04T16:12:26.432 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 16:12:26 smithi151 bash[51804]: ts=2024-03-04T16:12:26.335Z caller=refresh.go:99 level=error component="discovery manager notify" discovery=http config=config-0 msg="Unable to refresh target groups" err="Get \"http://172.21.15.117:8765/sd/prometheus/sd-config?service=alertmanager\": dial tcp 172.21.15.117:8765: connect: connection refused" 2024-03-04T16:12:26.432 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 16:12:26 smithi151 bash[51804]: ts=2024-03-04T16:12:26.342Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph msg="Unable to refresh target groups" err="Get \"http://172.21.15.117:8765/sd/prometheus/sd-config?service=mgr-prometheus\": dial tcp 172.21.15.117:8765: connect: connection refused" 2024-03-04T16:12:26.432 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 16:12:26 smithi151 bash[51804]: ts=2024-03-04T16:12:26.342Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=ceph-exporter msg="Unable to refresh target groups" err="Get \"http://172.21.15.117:8765/sd/prometheus/sd-config?service=ceph-exporter\": dial tcp 172.21.15.117:8765: connect: connection refused" 2024-03-04T16:12:26.432 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 16:12:26 smithi151 bash[51804]: ts=2024-03-04T16:12:26.342Z caller=refresh.go:99 level=error component="discovery manager scrape" discovery=http config=nvmeof msg="Unable to refresh target groups" err="Get \"http://172.21.15.117:8765/sd/prometheus/sd-config?service=nvmeof\": dial tcp 172.21.15.117:8765: connect: connection refused" 2024-03-04T16:12:26.931 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:26 smithi151 bash[32767]: debug 2024-03-04T16:12:26.827+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:57.729959+0000 front 2024-03-04T16:11:57.729773+0000 (oldest deadline 2024-03-04T16:12:21.229412+0000) 2024-03-04T16:12:26.931 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 16:12:26 smithi151 bash[27331]: debug 2024-03-04T16:12:26.819+0000 7fd76626c700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:54.598236+0000 front 2024-03-04T16:11:54.598159+0000 (oldest deadline 2024-03-04T16:12:20.497824+0000) 2024-03-04T16:12:26.932 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:26 smithi151 bash[38263]: debug 2024-03-04T16:12:26.615+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:27.024 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.3.service' 2024-03-04T16:12:27.050 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T16:12:27.051 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-03-04T16:12:27.051 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-03-04T16:12:27.051 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.4 2024-03-04T16:12:27.431 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:27 smithi151 bash[43933]: debug 2024-03-04T16:12:27.071+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:27.432 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 16:12:27 smithi151 systemd[1]: Stopping Ceph osd.4 for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T16:12:27.432 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 16:12:27 smithi151 bash[27331]: debug 2024-03-04T16:12:27.199+0000 7fd76dc80700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-03-04T16:12:27.432 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 16:12:27 smithi151 bash[27331]: debug 2024-03-04T16:12:27.199+0000 7fd76dc80700 -1 osd.4 794 *** Got signal Terminated *** 2024-03-04T16:12:27.432 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 16:12:27 smithi151 bash[27331]: debug 2024-03-04T16:12:27.199+0000 7fd76dc80700 -1 osd.4 794 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T16:12:27.931 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 16:12:27 smithi151 bash[27331]: debug 2024-03-04T16:12:27.859+0000 7fd76626c700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:54.598236+0000 front 2024-03-04T16:11:54.598159+0000 (oldest deadline 2024-03-04T16:12:20.497824+0000) 2024-03-04T16:12:27.931 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:27 smithi151 bash[32767]: debug 2024-03-04T16:12:27.859+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:57.729959+0000 front 2024-03-04T16:11:57.729773+0000 (oldest deadline 2024-03-04T16:12:21.229412+0000) 2024-03-04T16:12:27.932 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:27 smithi151 bash[38263]: debug 2024-03-04T16:12:27.615+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:28.431 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:28 smithi151 bash[43933]: debug 2024-03-04T16:12:28.099+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:28.431 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:28 smithi151 bash[43933]: debug 2024-03-04T16:12:28.099+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:28.931 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 16:12:28 smithi151 bash[27331]: debug 2024-03-04T16:12:28.819+0000 7fd76626c700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:54.598236+0000 front 2024-03-04T16:11:54.598159+0000 (oldest deadline 2024-03-04T16:12:20.497824+0000) 2024-03-04T16:12:28.931 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:28 smithi151 bash[32767]: debug 2024-03-04T16:12:28.835+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:57.729959+0000 front 2024-03-04T16:11:57.729773+0000 (oldest deadline 2024-03-04T16:12:21.229412+0000) 2024-03-04T16:12:28.932 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:28 smithi151 bash[38263]: debug 2024-03-04T16:12:28.663+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:29.431 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:29 smithi151 bash[43933]: debug 2024-03-04T16:12:29.083+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:29.431 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:29 smithi151 bash[43933]: debug 2024-03-04T16:12:29.083+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:29.931 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:29 smithi151 bash[38263]: debug 2024-03-04T16:12:29.663+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:29.931 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 16:12:29 smithi151 bash[27331]: debug 2024-03-04T16:12:29.867+0000 7fd76626c700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:54.598236+0000 front 2024-03-04T16:11:54.598159+0000 (oldest deadline 2024-03-04T16:12:20.497824+0000) 2024-03-04T16:12:29.932 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:29 smithi151 bash[32767]: debug 2024-03-04T16:12:29.823+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:57.729959+0000 front 2024-03-04T16:11:57.729773+0000 (oldest deadline 2024-03-04T16:12:21.229412+0000) 2024-03-04T16:12:30.431 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:30 smithi151 bash[43933]: debug 2024-03-04T16:12:30.103+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:30.431 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:30 smithi151 bash[43933]: debug 2024-03-04T16:12:30.103+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:30.931 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 16:12:30 smithi151 bash[27331]: debug 2024-03-04T16:12:30.875+0000 7fd76626c700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:54.598236+0000 front 2024-03-04T16:11:54.598159+0000 (oldest deadline 2024-03-04T16:12:20.497824+0000) 2024-03-04T16:12:30.931 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:30 smithi151 bash[32767]: debug 2024-03-04T16:12:30.803+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:57.729959+0000 front 2024-03-04T16:11:57.729773+0000 (oldest deadline 2024-03-04T16:12:21.229412+0000) 2024-03-04T16:12:30.932 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:30 smithi151 bash[38263]: debug 2024-03-04T16:12:30.627+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:31.431 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:31 smithi151 bash[43933]: debug 2024-03-04T16:12:31.083+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:31.431 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:31 smithi151 bash[43933]: debug 2024-03-04T16:12:31.083+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:31.856 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:31 smithi151 bash[32767]: debug 2024-03-04T16:12:31.799+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:57.729959+0000 front 2024-03-04T16:11:57.729773+0000 (oldest deadline 2024-03-04T16:12:21.229412+0000) 2024-03-04T16:12:31.856 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:31 smithi151 bash[38263]: debug 2024-03-04T16:12:31.591+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:31.856 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:31 smithi151 bash[38263]: debug 2024-03-04T16:12:31.591+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:02.127955+0000 front 2024-03-04T16:12:02.127772+0000 (oldest deadline 2024-03-04T16:12:30.928257+0000) 2024-03-04T16:12:32.181 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 16:12:31 smithi151 bash[27331]: debug 2024-03-04T16:12:31.851+0000 7fd76626c700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:54.598236+0000 front 2024-03-04T16:11:54.598159+0000 (oldest deadline 2024-03-04T16:12:20.497824+0000) 2024-03-04T16:12:32.181 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 16:12:31 smithi151 bash[27331]: debug 2024-03-04T16:12:31.851+0000 7fd76626c700 -1 osd.4 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.399363+0000 front 2024-03-04T16:12:06.399736+0000 (oldest deadline 2024-03-04T16:12:31.699374+0000) 2024-03-04T16:12:32.181 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:32 smithi151 bash[43933]: debug 2024-03-04T16:12:32.075+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:32.182 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:32 smithi151 bash[43933]: debug 2024-03-04T16:12:32.075+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:32.851 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:32 smithi151 bash[38263]: debug 2024-03-04T16:12:32.623+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:32.852 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:32 smithi151 bash[38263]: debug 2024-03-04T16:12:32.623+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:02.127955+0000 front 2024-03-04T16:12:02.127772+0000 (oldest deadline 2024-03-04T16:12:30.928257+0000) 2024-03-04T16:12:33.181 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:32 smithi151 bash[32767]: debug 2024-03-04T16:12:32.847+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:57.729959+0000 front 2024-03-04T16:11:57.729773+0000 (oldest deadline 2024-03-04T16:12:21.229412+0000) 2024-03-04T16:12:33.181 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:32 smithi151 bash[32767]: debug 2024-03-04T16:12:32.847+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.833783+0000 front 2024-03-04T16:12:06.833381+0000 (oldest deadline 2024-03-04T16:12:32.733346+0000) 2024-03-04T16:12:33.181 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:33 smithi151 bash[43933]: debug 2024-03-04T16:12:33.079+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:33.182 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:33 smithi151 bash[43933]: debug 2024-03-04T16:12:33.079+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:33.442 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 16:12:33 smithi151 bash[53183]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-osd-4 2024-03-04T16:12:33.443 INFO:journalctl@ceph.osd.4.smithi151.stdout:Mar 04 16:12:33 smithi151 bash[53247]: Error response from daemon: No such container: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-osd-4 2024-03-04T16:12:33.883 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:33 smithi151 bash[38263]: debug 2024-03-04T16:12:33.643+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:33.883 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:33 smithi151 bash[38263]: debug 2024-03-04T16:12:33.643+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:02.127955+0000 front 2024-03-04T16:12:02.127772+0000 (oldest deadline 2024-03-04T16:12:30.928257+0000) 2024-03-04T16:12:34.136 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:33 smithi151 bash[32767]: debug 2024-03-04T16:12:33.879+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:57.729959+0000 front 2024-03-04T16:11:57.729773+0000 (oldest deadline 2024-03-04T16:12:21.229412+0000) 2024-03-04T16:12:34.137 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:33 smithi151 bash[32767]: debug 2024-03-04T16:12:33.879+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.833783+0000 front 2024-03-04T16:12:06.833381+0000 (oldest deadline 2024-03-04T16:12:32.733346+0000) 2024-03-04T16:12:34.387 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:34 smithi151 bash[43933]: debug 2024-03-04T16:12:34.131+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:34.387 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:34 smithi151 bash[43933]: debug 2024-03-04T16:12:34.131+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:34.681 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:34 smithi151 bash[38263]: debug 2024-03-04T16:12:34.631+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:34.681 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:34 smithi151 bash[38263]: debug 2024-03-04T16:12:34.631+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:02.127955+0000 front 2024-03-04T16:12:02.127772+0000 (oldest deadline 2024-03-04T16:12:30.928257+0000) 2024-03-04T16:12:35.181 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:34 smithi151 bash[32767]: debug 2024-03-04T16:12:34.911+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:57.729959+0000 front 2024-03-04T16:11:57.729773+0000 (oldest deadline 2024-03-04T16:12:21.229412+0000) 2024-03-04T16:12:35.181 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:34 smithi151 bash[32767]: debug 2024-03-04T16:12:34.911+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.833783+0000 front 2024-03-04T16:12:06.833381+0000 (oldest deadline 2024-03-04T16:12:32.733346+0000) 2024-03-04T16:12:35.395 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.4.service' 2024-03-04T16:12:35.424 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T16:12:35.424 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-03-04T16:12:35.424 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-03-04T16:12:35.425 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.5 2024-03-04T16:12:35.479 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:35 smithi151 bash[43933]: debug 2024-03-04T16:12:35.179+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:35.479 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:35 smithi151 bash[43933]: debug 2024-03-04T16:12:35.179+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:35.650 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:35 smithi151 systemd[1]: Stopping Ceph osd.5 for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T16:12:35.651 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:35 smithi151 bash[32767]: debug 2024-03-04T16:12:35.595+0000 7f67a283b700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-03-04T16:12:35.651 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:35 smithi151 bash[32767]: debug 2024-03-04T16:12:35.595+0000 7f67a283b700 -1 osd.5 794 *** Got signal Terminated *** 2024-03-04T16:12:35.651 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:35 smithi151 bash[32767]: debug 2024-03-04T16:12:35.595+0000 7f67a283b700 -1 osd.5 794 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T16:12:35.881 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:35 smithi151 bash[38263]: debug 2024-03-04T16:12:35.647+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:35.881 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:35 smithi151 bash[38263]: debug 2024-03-04T16:12:35.647+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:02.127955+0000 front 2024-03-04T16:12:02.127772+0000 (oldest deadline 2024-03-04T16:12:30.928257+0000) 2024-03-04T16:12:36.181 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:35 smithi151 bash[32767]: debug 2024-03-04T16:12:35.879+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:57.729959+0000 front 2024-03-04T16:11:57.729773+0000 (oldest deadline 2024-03-04T16:12:21.229412+0000) 2024-03-04T16:12:36.181 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:35 smithi151 bash[32767]: debug 2024-03-04T16:12:35.879+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.833783+0000 front 2024-03-04T16:12:06.833381+0000 (oldest deadline 2024-03-04T16:12:32.733346+0000) 2024-03-04T16:12:36.609 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:36 smithi151 bash[43933]: debug 2024-03-04T16:12:36.211+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:36.610 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:36 smithi151 bash[43933]: debug 2024-03-04T16:12:36.211+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:36.931 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:36 smithi151 bash[38263]: debug 2024-03-04T16:12:36.603+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:36.931 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:36 smithi151 bash[38263]: debug 2024-03-04T16:12:36.603+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:02.127955+0000 front 2024-03-04T16:12:02.127772+0000 (oldest deadline 2024-03-04T16:12:30.928257+0000) 2024-03-04T16:12:36.931 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:36 smithi151 bash[32767]: debug 2024-03-04T16:12:36.839+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:57.729959+0000 front 2024-03-04T16:11:57.729773+0000 (oldest deadline 2024-03-04T16:12:21.229412+0000) 2024-03-04T16:12:36.931 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:36 smithi151 bash[32767]: debug 2024-03-04T16:12:36.839+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.833783+0000 front 2024-03-04T16:12:06.833381+0000 (oldest deadline 2024-03-04T16:12:32.733346+0000) 2024-03-04T16:12:37.577 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:37 smithi151 bash[43933]: debug 2024-03-04T16:12:37.243+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:37.577 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:37 smithi151 bash[43933]: debug 2024-03-04T16:12:37.243+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:37.893 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:37 smithi151 bash[38263]: debug 2024-03-04T16:12:37.575+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:37.893 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:37 smithi151 bash[38263]: debug 2024-03-04T16:12:37.575+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:02.127955+0000 front 2024-03-04T16:12:02.127772+0000 (oldest deadline 2024-03-04T16:12:30.928257+0000) 2024-03-04T16:12:38.181 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:37 smithi151 bash[32767]: debug 2024-03-04T16:12:37.887+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:57.729959+0000 front 2024-03-04T16:11:57.729773+0000 (oldest deadline 2024-03-04T16:12:21.229412+0000) 2024-03-04T16:12:38.181 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:37 smithi151 bash[32767]: debug 2024-03-04T16:12:37.887+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.833783+0000 front 2024-03-04T16:12:06.833381+0000 (oldest deadline 2024-03-04T16:12:32.733346+0000) 2024-03-04T16:12:38.585 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:38 smithi151 bash[43933]: debug 2024-03-04T16:12:38.231+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:38.585 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:38 smithi151 bash[43933]: debug 2024-03-04T16:12:38.231+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:38.928 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:38 smithi151 bash[38263]: debug 2024-03-04T16:12:38.583+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:38.928 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:38 smithi151 bash[38263]: debug 2024-03-04T16:12:38.583+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:02.127955+0000 front 2024-03-04T16:12:02.127772+0000 (oldest deadline 2024-03-04T16:12:30.928257+0000) 2024-03-04T16:12:38.928 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:38 smithi151 bash[38263]: debug 2024-03-04T16:12:38.583+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.429639+0000 front 2024-03-04T16:12:14.429674+0000 (oldest deadline 2024-03-04T16:12:38.529350+0000) 2024-03-04T16:12:39.181 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:38 smithi151 bash[32767]: debug 2024-03-04T16:12:38.927+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:57.729959+0000 front 2024-03-04T16:11:57.729773+0000 (oldest deadline 2024-03-04T16:12:21.229412+0000) 2024-03-04T16:12:39.181 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:38 smithi151 bash[32767]: debug 2024-03-04T16:12:38.927+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.833783+0000 front 2024-03-04T16:12:06.833381+0000 (oldest deadline 2024-03-04T16:12:32.733346+0000) 2024-03-04T16:12:39.181 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:38 smithi151 bash[32767]: debug 2024-03-04T16:12:38.927+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:15.035168+0000 front 2024-03-04T16:12:15.034854+0000 (oldest deadline 2024-03-04T16:12:37.934543+0000) 2024-03-04T16:12:39.630 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:39 smithi151 bash[43933]: debug 2024-03-04T16:12:39.271+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:39.630 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:39 smithi151 bash[43933]: debug 2024-03-04T16:12:39.271+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:39.630 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:39 smithi151 bash[43933]: debug 2024-03-04T16:12:39.271+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.338573+0000 front 2024-03-04T16:12:14.338515+0000 (oldest deadline 2024-03-04T16:12:39.038226+0000) 2024-03-04T16:12:39.891 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:39 smithi151 bash[38263]: debug 2024-03-04T16:12:39.627+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:39.892 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:39 smithi151 bash[38263]: debug 2024-03-04T16:12:39.627+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:02.127955+0000 front 2024-03-04T16:12:02.127772+0000 (oldest deadline 2024-03-04T16:12:30.928257+0000) 2024-03-04T16:12:39.892 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:39 smithi151 bash[38263]: debug 2024-03-04T16:12:39.627+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.429639+0000 front 2024-03-04T16:12:14.429674+0000 (oldest deadline 2024-03-04T16:12:38.529350+0000) 2024-03-04T16:12:40.181 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:39 smithi151 bash[32767]: debug 2024-03-04T16:12:39.887+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:57.729959+0000 front 2024-03-04T16:11:57.729773+0000 (oldest deadline 2024-03-04T16:12:21.229412+0000) 2024-03-04T16:12:40.181 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:39 smithi151 bash[32767]: debug 2024-03-04T16:12:39.887+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.833783+0000 front 2024-03-04T16:12:06.833381+0000 (oldest deadline 2024-03-04T16:12:32.733346+0000) 2024-03-04T16:12:40.181 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:39 smithi151 bash[32767]: debug 2024-03-04T16:12:39.887+0000 7f679ae27700 -1 osd.5 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:15.035168+0000 front 2024-03-04T16:12:15.034854+0000 (oldest deadline 2024-03-04T16:12:37.934543+0000) 2024-03-04T16:12:40.675 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:40 smithi151 bash[43933]: debug 2024-03-04T16:12:40.235+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:40.675 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:40 smithi151 bash[43933]: debug 2024-03-04T16:12:40.235+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:40.675 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:40 smithi151 bash[43933]: debug 2024-03-04T16:12:40.235+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.338573+0000 front 2024-03-04T16:12:14.338515+0000 (oldest deadline 2024-03-04T16:12:39.038226+0000) 2024-03-04T16:12:40.931 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:40 smithi151 bash[38263]: debug 2024-03-04T16:12:40.671+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:40.931 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:40 smithi151 bash[38263]: debug 2024-03-04T16:12:40.671+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:02.127955+0000 front 2024-03-04T16:12:02.127772+0000 (oldest deadline 2024-03-04T16:12:30.928257+0000) 2024-03-04T16:12:40.931 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:40 smithi151 bash[38263]: debug 2024-03-04T16:12:40.671+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.429639+0000 front 2024-03-04T16:12:14.429674+0000 (oldest deadline 2024-03-04T16:12:38.529350+0000) 2024-03-04T16:12:41.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:41 smithi151 bash[43933]: debug 2024-03-04T16:12:41.279+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:41.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:41 smithi151 bash[43933]: debug 2024-03-04T16:12:41.279+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:41.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:41 smithi151 bash[43933]: debug 2024-03-04T16:12:41.279+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.338573+0000 front 2024-03-04T16:12:14.338515+0000 (oldest deadline 2024-03-04T16:12:39.038226+0000) 2024-03-04T16:12:41.937 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:41 smithi151 bash[53381]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-osd-5 2024-03-04T16:12:41.937 INFO:journalctl@ceph.osd.5.smithi151.stdout:Mar 04 16:12:41 smithi151 bash[53445]: Error response from daemon: No such container: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-osd-5 2024-03-04T16:12:41.937 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:41 smithi151 bash[38263]: debug 2024-03-04T16:12:41.699+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:41.937 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:41 smithi151 bash[38263]: debug 2024-03-04T16:12:41.699+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:02.127955+0000 front 2024-03-04T16:12:02.127772+0000 (oldest deadline 2024-03-04T16:12:30.928257+0000) 2024-03-04T16:12:41.937 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:41 smithi151 bash[38263]: debug 2024-03-04T16:12:41.699+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.429639+0000 front 2024-03-04T16:12:14.429674+0000 (oldest deadline 2024-03-04T16:12:38.529350+0000) 2024-03-04T16:12:42.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:42 smithi151 bash[43933]: debug 2024-03-04T16:12:42.323+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:42.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:42 smithi151 bash[43933]: debug 2024-03-04T16:12:42.323+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:42.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:42 smithi151 bash[43933]: debug 2024-03-04T16:12:42.323+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.338573+0000 front 2024-03-04T16:12:14.338515+0000 (oldest deadline 2024-03-04T16:12:39.038226+0000) 2024-03-04T16:12:43.167 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:42 smithi151 bash[38263]: debug 2024-03-04T16:12:42.719+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:43.167 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:42 smithi151 bash[38263]: debug 2024-03-04T16:12:42.719+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:02.127955+0000 front 2024-03-04T16:12:02.127772+0000 (oldest deadline 2024-03-04T16:12:30.928257+0000) 2024-03-04T16:12:43.167 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:42 smithi151 bash[38263]: debug 2024-03-04T16:12:42.719+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.429639+0000 front 2024-03-04T16:12:14.429674+0000 (oldest deadline 2024-03-04T16:12:38.529350+0000) 2024-03-04T16:12:43.437 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:43 smithi151 bash[43933]: debug 2024-03-04T16:12:43.363+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:43.438 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:43 smithi151 bash[43933]: debug 2024-03-04T16:12:43.363+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:43.438 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:43 smithi151 bash[43933]: debug 2024-03-04T16:12:43.363+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.338573+0000 front 2024-03-04T16:12:14.338515+0000 (oldest deadline 2024-03-04T16:12:39.038226+0000) 2024-03-04T16:12:43.841 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.5.service' 2024-03-04T16:12:43.909 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T16:12:43.909 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-03-04T16:12:43.909 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-03-04T16:12:43.909 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.6 2024-03-04T16:12:44.113 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:43 smithi151 bash[38263]: debug 2024-03-04T16:12:43.731+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:44.113 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:43 smithi151 bash[38263]: debug 2024-03-04T16:12:43.731+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:02.127955+0000 front 2024-03-04T16:12:02.127772+0000 (oldest deadline 2024-03-04T16:12:30.928257+0000) 2024-03-04T16:12:44.113 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:43 smithi151 bash[38263]: debug 2024-03-04T16:12:43.731+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.429639+0000 front 2024-03-04T16:12:14.429674+0000 (oldest deadline 2024-03-04T16:12:38.529350+0000) 2024-03-04T16:12:44.113 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:43 smithi151 systemd[1]: Stopping Ceph osd.6 for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T16:12:44.113 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:44 smithi151 bash[38263]: debug 2024-03-04T16:12:44.107+0000 7f12d6bd8700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-03-04T16:12:44.410 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:44 smithi151 bash[38263]: debug 2024-03-04T16:12:44.107+0000 7f12d6bd8700 -1 osd.6 794 *** Got signal Terminated *** 2024-03-04T16:12:44.410 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:44 smithi151 bash[38263]: debug 2024-03-04T16:12:44.107+0000 7f12d6bd8700 -1 osd.6 794 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T16:12:44.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:44 smithi151 bash[43933]: debug 2024-03-04T16:12:44.407+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:44.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:44 smithi151 bash[43933]: debug 2024-03-04T16:12:44.407+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:44.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:44 smithi151 bash[43933]: debug 2024-03-04T16:12:44.407+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.338573+0000 front 2024-03-04T16:12:14.338515+0000 (oldest deadline 2024-03-04T16:12:39.038226+0000) 2024-03-04T16:12:45.181 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:44 smithi151 bash[38263]: debug 2024-03-04T16:12:44.783+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:45.181 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:44 smithi151 bash[38263]: debug 2024-03-04T16:12:44.783+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:02.127955+0000 front 2024-03-04T16:12:02.127772+0000 (oldest deadline 2024-03-04T16:12:30.928257+0000) 2024-03-04T16:12:45.181 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:44 smithi151 bash[38263]: debug 2024-03-04T16:12:44.783+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.429639+0000 front 2024-03-04T16:12:14.429674+0000 (oldest deadline 2024-03-04T16:12:38.529350+0000) 2024-03-04T16:12:45.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:45 smithi151 bash[43933]: debug 2024-03-04T16:12:45.399+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:45.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:45 smithi151 bash[43933]: debug 2024-03-04T16:12:45.399+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:45.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:45 smithi151 bash[43933]: debug 2024-03-04T16:12:45.399+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.338573+0000 front 2024-03-04T16:12:14.338515+0000 (oldest deadline 2024-03-04T16:12:39.038226+0000) 2024-03-04T16:12:46.181 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:45 smithi151 bash[38263]: debug 2024-03-04T16:12:45.759+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:46.181 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:45 smithi151 bash[38263]: debug 2024-03-04T16:12:45.759+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:02.127955+0000 front 2024-03-04T16:12:02.127772+0000 (oldest deadline 2024-03-04T16:12:30.928257+0000) 2024-03-04T16:12:46.181 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:45 smithi151 bash[38263]: debug 2024-03-04T16:12:45.759+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.429639+0000 front 2024-03-04T16:12:14.429674+0000 (oldest deadline 2024-03-04T16:12:38.529350+0000) 2024-03-04T16:12:46.810 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:46 smithi151 bash[43933]: debug 2024-03-04T16:12:46.447+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:46.810 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:46 smithi151 bash[43933]: debug 2024-03-04T16:12:46.447+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:46.810 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:46 smithi151 bash[43933]: debug 2024-03-04T16:12:46.447+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.338573+0000 front 2024-03-04T16:12:14.338515+0000 (oldest deadline 2024-03-04T16:12:39.038226+0000) 2024-03-04T16:12:46.810 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:46 smithi151 bash[43933]: debug 2024-03-04T16:12:46.447+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6830 osd.3 since back 2024-03-04T16:12:21.939543+0000 front 2024-03-04T16:12:21.939635+0000 (oldest deadline 2024-03-04T16:12:45.439385+0000) 2024-03-04T16:12:47.181 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:46 smithi151 bash[38263]: debug 2024-03-04T16:12:46.807+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:47.181 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:46 smithi151 bash[38263]: debug 2024-03-04T16:12:46.807+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:02.127955+0000 front 2024-03-04T16:12:02.127772+0000 (oldest deadline 2024-03-04T16:12:30.928257+0000) 2024-03-04T16:12:47.181 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:46 smithi151 bash[38263]: debug 2024-03-04T16:12:46.807+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.429639+0000 front 2024-03-04T16:12:14.429674+0000 (oldest deadline 2024-03-04T16:12:38.529350+0000) 2024-03-04T16:12:47.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:47 smithi151 bash[43933]: debug 2024-03-04T16:12:47.419+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:47.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:47 smithi151 bash[43933]: debug 2024-03-04T16:12:47.419+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:47.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:47 smithi151 bash[43933]: debug 2024-03-04T16:12:47.419+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.338573+0000 front 2024-03-04T16:12:14.338515+0000 (oldest deadline 2024-03-04T16:12:39.038226+0000) 2024-03-04T16:12:47.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:47 smithi151 bash[43933]: debug 2024-03-04T16:12:47.419+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6830 osd.3 since back 2024-03-04T16:12:21.939543+0000 front 2024-03-04T16:12:21.939635+0000 (oldest deadline 2024-03-04T16:12:45.439385+0000) 2024-03-04T16:12:48.181 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:47 smithi151 bash[38263]: debug 2024-03-04T16:12:47.759+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:48.181 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:47 smithi151 bash[38263]: debug 2024-03-04T16:12:47.759+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:02.127955+0000 front 2024-03-04T16:12:02.127772+0000 (oldest deadline 2024-03-04T16:12:30.928257+0000) 2024-03-04T16:12:48.181 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:47 smithi151 bash[38263]: debug 2024-03-04T16:12:47.759+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.429639+0000 front 2024-03-04T16:12:14.429674+0000 (oldest deadline 2024-03-04T16:12:38.529350+0000) 2024-03-04T16:12:48.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:48 smithi151 bash[43933]: debug 2024-03-04T16:12:48.407+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:48.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:48 smithi151 bash[43933]: debug 2024-03-04T16:12:48.407+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:48.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:48 smithi151 bash[43933]: debug 2024-03-04T16:12:48.407+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.338573+0000 front 2024-03-04T16:12:14.338515+0000 (oldest deadline 2024-03-04T16:12:39.038226+0000) 2024-03-04T16:12:48.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:48 smithi151 bash[43933]: debug 2024-03-04T16:12:48.407+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6830 osd.3 since back 2024-03-04T16:12:21.939543+0000 front 2024-03-04T16:12:21.939635+0000 (oldest deadline 2024-03-04T16:12:45.439385+0000) 2024-03-04T16:12:49.181 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:48 smithi151 bash[38263]: debug 2024-03-04T16:12:48.727+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.726370+0000 front 2024-03-04T16:11:55.726715+0000 (oldest deadline 2024-03-04T16:12:19.226286+0000) 2024-03-04T16:12:49.181 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:48 smithi151 bash[38263]: debug 2024-03-04T16:12:48.727+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:02.127955+0000 front 2024-03-04T16:12:02.127772+0000 (oldest deadline 2024-03-04T16:12:30.928257+0000) 2024-03-04T16:12:49.181 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:48 smithi151 bash[38263]: debug 2024-03-04T16:12:48.727+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.429639+0000 front 2024-03-04T16:12:14.429674+0000 (oldest deadline 2024-03-04T16:12:38.529350+0000) 2024-03-04T16:12:49.182 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:48 smithi151 bash[38263]: debug 2024-03-04T16:12:48.727+0000 7f12cf1c4700 -1 osd.6 794 heartbeat_check: no reply from 172.21.15.117:6830 osd.3 since back 2024-03-04T16:12:22.531758+0000 front 2024-03-04T16:12:22.531500+0000 (oldest deadline 2024-03-04T16:12:47.831411+0000) 2024-03-04T16:12:49.708 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:49 smithi151 bash[43933]: debug 2024-03-04T16:12:49.439+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:49.708 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:49 smithi151 bash[43933]: debug 2024-03-04T16:12:49.439+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:49.708 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:49 smithi151 bash[43933]: debug 2024-03-04T16:12:49.439+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.338573+0000 front 2024-03-04T16:12:14.338515+0000 (oldest deadline 2024-03-04T16:12:39.038226+0000) 2024-03-04T16:12:49.708 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:49 smithi151 bash[43933]: debug 2024-03-04T16:12:49.439+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6830 osd.3 since back 2024-03-04T16:12:21.939543+0000 front 2024-03-04T16:12:21.939635+0000 (oldest deadline 2024-03-04T16:12:45.439385+0000) 2024-03-04T16:12:50.681 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:50 smithi151 bash[53574]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-osd-6 2024-03-04T16:12:50.681 INFO:journalctl@ceph.osd.6.smithi151.stdout:Mar 04 16:12:50 smithi151 bash[53639]: Error response from daemon: No such container: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-osd-6 2024-03-04T16:12:50.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:50 smithi151 bash[43933]: debug 2024-03-04T16:12:50.427+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:50.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:50 smithi151 bash[43933]: debug 2024-03-04T16:12:50.427+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:50.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:50 smithi151 bash[43933]: debug 2024-03-04T16:12:50.427+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.338573+0000 front 2024-03-04T16:12:14.338515+0000 (oldest deadline 2024-03-04T16:12:39.038226+0000) 2024-03-04T16:12:50.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:50 smithi151 bash[43933]: debug 2024-03-04T16:12:50.427+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6830 osd.3 since back 2024-03-04T16:12:21.939543+0000 front 2024-03-04T16:12:21.939635+0000 (oldest deadline 2024-03-04T16:12:45.439385+0000) 2024-03-04T16:12:51.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:51 smithi151 bash[43933]: debug 2024-03-04T16:12:51.463+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:51.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:51 smithi151 bash[43933]: debug 2024-03-04T16:12:51.463+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:51.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:51 smithi151 bash[43933]: debug 2024-03-04T16:12:51.463+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.338573+0000 front 2024-03-04T16:12:14.338515+0000 (oldest deadline 2024-03-04T16:12:39.038226+0000) 2024-03-04T16:12:51.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:51 smithi151 bash[43933]: debug 2024-03-04T16:12:51.463+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6830 osd.3 since back 2024-03-04T16:12:21.939543+0000 front 2024-03-04T16:12:21.939635+0000 (oldest deadline 2024-03-04T16:12:45.439385+0000) 2024-03-04T16:12:52.493 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.6.service' 2024-03-04T16:12:52.520 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T16:12:52.521 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-03-04T16:12:52.521 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-03-04T16:12:52.521 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.7 2024-03-04T16:12:52.932 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:52 smithi151 bash[43933]: debug 2024-03-04T16:12:52.459+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:52.932 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:52 smithi151 bash[43933]: debug 2024-03-04T16:12:52.459+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:52.932 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:52 smithi151 bash[43933]: debug 2024-03-04T16:12:52.459+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.338573+0000 front 2024-03-04T16:12:14.338515+0000 (oldest deadline 2024-03-04T16:12:39.038226+0000) 2024-03-04T16:12:52.932 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:52 smithi151 bash[43933]: debug 2024-03-04T16:12:52.459+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6830 osd.3 since back 2024-03-04T16:12:21.939543+0000 front 2024-03-04T16:12:21.939635+0000 (oldest deadline 2024-03-04T16:12:45.439385+0000) 2024-03-04T16:12:52.932 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:52 smithi151 systemd[1]: Stopping Ceph osd.7 for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T16:12:52.932 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:52 smithi151 bash[43933]: debug 2024-03-04T16:12:52.703+0000 7f00d3eba700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-03-04T16:12:52.932 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:52 smithi151 bash[43933]: debug 2024-03-04T16:12:52.703+0000 7f00d3eba700 -1 osd.7 794 *** Got signal Terminated *** 2024-03-04T16:12:52.932 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:52 smithi151 bash[43933]: debug 2024-03-04T16:12:52.703+0000 7f00d3eba700 -1 osd.7 794 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-03-04T16:12:53.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:53 smithi151 bash[43933]: debug 2024-03-04T16:12:53.427+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:53.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:53 smithi151 bash[43933]: debug 2024-03-04T16:12:53.427+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:53.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:53 smithi151 bash[43933]: debug 2024-03-04T16:12:53.427+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.338573+0000 front 2024-03-04T16:12:14.338515+0000 (oldest deadline 2024-03-04T16:12:39.038226+0000) 2024-03-04T16:12:53.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:53 smithi151 bash[43933]: debug 2024-03-04T16:12:53.427+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6830 osd.3 since back 2024-03-04T16:12:21.939543+0000 front 2024-03-04T16:12:21.939635+0000 (oldest deadline 2024-03-04T16:12:45.439385+0000) 2024-03-04T16:12:53.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:53 smithi151 bash[43933]: debug 2024-03-04T16:12:53.427+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.151:6804 osd.4 since back 2024-03-04T16:12:28.849142+0000 front 2024-03-04T16:12:28.849089+0000 (oldest deadline 2024-03-04T16:12:52.948849+0000) 2024-03-04T16:12:54.931 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:54 smithi151 bash[43933]: debug 2024-03-04T16:12:54.463+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:54.931 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:54 smithi151 bash[43933]: debug 2024-03-04T16:12:54.463+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:54.931 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:54 smithi151 bash[43933]: debug 2024-03-04T16:12:54.463+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.338573+0000 front 2024-03-04T16:12:14.338515+0000 (oldest deadline 2024-03-04T16:12:39.038226+0000) 2024-03-04T16:12:54.931 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:54 smithi151 bash[43933]: debug 2024-03-04T16:12:54.463+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6830 osd.3 since back 2024-03-04T16:12:21.939543+0000 front 2024-03-04T16:12:21.939635+0000 (oldest deadline 2024-03-04T16:12:45.439385+0000) 2024-03-04T16:12:54.932 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:54 smithi151 bash[43933]: debug 2024-03-04T16:12:54.463+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.151:6804 osd.4 since back 2024-03-04T16:12:28.849142+0000 front 2024-03-04T16:12:28.849089+0000 (oldest deadline 2024-03-04T16:12:52.948849+0000) 2024-03-04T16:12:55.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:55 smithi151 bash[43933]: debug 2024-03-04T16:12:55.427+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:55.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:55 smithi151 bash[43933]: debug 2024-03-04T16:12:55.427+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:55.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:55 smithi151 bash[43933]: debug 2024-03-04T16:12:55.427+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.338573+0000 front 2024-03-04T16:12:14.338515+0000 (oldest deadline 2024-03-04T16:12:39.038226+0000) 2024-03-04T16:12:55.681 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:55 smithi151 bash[43933]: debug 2024-03-04T16:12:55.427+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6830 osd.3 since back 2024-03-04T16:12:21.939543+0000 front 2024-03-04T16:12:21.939635+0000 (oldest deadline 2024-03-04T16:12:45.439385+0000) 2024-03-04T16:12:55.682 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:55 smithi151 bash[43933]: debug 2024-03-04T16:12:55.427+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.151:6804 osd.4 since back 2024-03-04T16:12:28.849142+0000 front 2024-03-04T16:12:28.849089+0000 (oldest deadline 2024-03-04T16:12:52.948849+0000) 2024-03-04T16:12:56.931 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:56 smithi151 bash[43933]: debug 2024-03-04T16:12:56.459+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:56.932 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:56 smithi151 bash[43933]: debug 2024-03-04T16:12:56.459+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:56.932 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:56 smithi151 bash[43933]: debug 2024-03-04T16:12:56.459+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.338573+0000 front 2024-03-04T16:12:14.338515+0000 (oldest deadline 2024-03-04T16:12:39.038226+0000) 2024-03-04T16:12:56.932 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:56 smithi151 bash[43933]: debug 2024-03-04T16:12:56.459+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6830 osd.3 since back 2024-03-04T16:12:21.939543+0000 front 2024-03-04T16:12:21.939635+0000 (oldest deadline 2024-03-04T16:12:45.439385+0000) 2024-03-04T16:12:56.932 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:56 smithi151 bash[43933]: debug 2024-03-04T16:12:56.459+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.151:6804 osd.4 since back 2024-03-04T16:12:28.849142+0000 front 2024-03-04T16:12:28.849089+0000 (oldest deadline 2024-03-04T16:12:52.948849+0000) 2024-03-04T16:12:57.931 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:57 smithi151 bash[43933]: debug 2024-03-04T16:12:57.507+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6806 osd.0 since back 2024-03-04T16:11:55.735083+0000 front 2024-03-04T16:11:55.734622+0000 (oldest deadline 2024-03-04T16:12:20.434336+0000) 2024-03-04T16:12:57.931 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:57 smithi151 bash[43933]: debug 2024-03-04T16:12:57.507+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6814 osd.1 since back 2024-03-04T16:12:06.836479+0000 front 2024-03-04T16:12:06.836722+0000 (oldest deadline 2024-03-04T16:12:27.936324+0000) 2024-03-04T16:12:57.932 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:57 smithi151 bash[43933]: debug 2024-03-04T16:12:57.507+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6822 osd.2 since back 2024-03-04T16:12:14.338573+0000 front 2024-03-04T16:12:14.338515+0000 (oldest deadline 2024-03-04T16:12:39.038226+0000) 2024-03-04T16:12:57.932 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:57 smithi151 bash[43933]: debug 2024-03-04T16:12:57.507+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.117:6830 osd.3 since back 2024-03-04T16:12:21.939543+0000 front 2024-03-04T16:12:21.939635+0000 (oldest deadline 2024-03-04T16:12:45.439385+0000) 2024-03-04T16:12:57.932 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:57 smithi151 bash[43933]: debug 2024-03-04T16:12:57.507+0000 7f00cc4a6700 -1 osd.7 794 heartbeat_check: no reply from 172.21.15.151:6804 osd.4 since back 2024-03-04T16:12:28.849142+0000 front 2024-03-04T16:12:28.849089+0000 (oldest deadline 2024-03-04T16:12:52.948849+0000) 2024-03-04T16:12:58.932 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:58 smithi151 bash[53772]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-osd-7 2024-03-04T16:12:58.932 INFO:journalctl@ceph.osd.7.smithi151.stdout:Mar 04 16:12:58 smithi151 bash[53835]: Error response from daemon: No such container: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-osd-7 2024-03-04T16:13:00.621 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@osd.7.service' 2024-03-04T16:13:00.648 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T16:13:00.649 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-03-04T16:13:00.649 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-03-04T16:13:00.649 DEBUG:teuthology.orchestra.run.smithi117:> sudo systemctl stop ceph-e40b5504-da3c-11ee-95c6-87774f69a715@rgw.foo.a 2024-03-04T16:13:01.068 INFO:journalctl@ceph.rgw.foo.a.smithi117.stdout:Mar 04 16:13:00 smithi117 systemd[1]: Stopping Ceph rgw.foo.a for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T16:13:01.068 INFO:journalctl@ceph.rgw.foo.a.smithi117.stdout:Mar 04 16:13:00 smithi117 bash[52907]: debug 2024-03-04T16:13:00.778+0000 7f839ece9700 -1 received signal: Terminated from /sbin/docker-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-stderr=true --default-log-stderr-prefix=debug (PID: 1) UID: 0 2024-03-04T16:13:01.068 INFO:journalctl@ceph.rgw.foo.a.smithi117.stdout:Mar 04 16:13:00 smithi117 bash[52907]: debug 2024-03-04T16:13:00.778+0000 7f83ad561ac0 -1 shutting down 2024-03-04T16:13:11.540 INFO:journalctl@ceph.rgw.foo.a.smithi117.stdout:Mar 04 16:13:11 smithi117 bash[133767]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-rgw-foo-a 2024-03-04T16:13:11.555 DEBUG:teuthology.orchestra.run.smithi117:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@rgw.foo.a.service' 2024-03-04T16:13:11.576 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T16:13:11.576 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-03-04T16:13:11.576 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-03-04T16:13:11.577 DEBUG:teuthology.orchestra.run.smithi151:> sudo systemctl stop ceph-e40b5504-da3c-11ee-95c6-87774f69a715@prometheus.a 2024-03-04T16:13:11.931 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 16:13:11 smithi151 systemd[1]: Stopping Ceph prometheus.a for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T16:13:11.931 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 16:13:11 smithi151 bash[51804]: ts=2024-03-04T16:13:11.733Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-03-04T16:13:11.931 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 16:13:11 smithi151 bash[51804]: ts=2024-03-04T16:13:11.733Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-03-04T16:13:11.931 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 16:13:11 smithi151 bash[51804]: ts=2024-03-04T16:13:11.733Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-03-04T16:13:11.931 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 16:13:11 smithi151 bash[51804]: ts=2024-03-04T16:13:11.733Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-03-04T16:13:11.931 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 16:13:11 smithi151 bash[51804]: ts=2024-03-04T16:13:11.733Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-03-04T16:13:11.932 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 16:13:11 smithi151 bash[51804]: ts=2024-03-04T16:13:11.733Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-03-04T16:13:11.932 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 16:13:11 smithi151 bash[51804]: ts=2024-03-04T16:13:11.733Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-03-04T16:13:11.932 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 16:13:11 smithi151 bash[51804]: ts=2024-03-04T16:13:11.733Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-03-04T16:13:11.932 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 16:13:11 smithi151 bash[51804]: ts=2024-03-04T16:13:11.733Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-03-04T16:13:11.932 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 16:13:11 smithi151 bash[51804]: ts=2024-03-04T16:13:11.734Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-03-04T16:13:11.932 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 16:13:11 smithi151 bash[51804]: ts=2024-03-04T16:13:11.734Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-03-04T16:13:11.932 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 16:13:11 smithi151 bash[51804]: ts=2024-03-04T16:13:11.734Z caller=main.go:1147 level=info msg="See you next time!" 2024-03-04T16:13:12.374 INFO:journalctl@ceph.prometheus.a.smithi151.stdout:Mar 04 16:13:12 smithi151 bash[53972]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-prometheus-a 2024-03-04T16:13:12.387 DEBUG:teuthology.orchestra.run.smithi151:> sudo pkill -f 'journalctl -f -n 0 -u ceph-e40b5504-da3c-11ee-95c6-87774f69a715@prometheus.a.service' 2024-03-04T16:13:12.407 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-03-04T16:13:12.408 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-03-04T16:13:12.408 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm rm-cluster --fsid e40b5504-da3c-11ee-95c6-87774f69a715 --force --keep-logs 2024-03-04T16:13:12.573 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T16:13:17.402 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 16:13:17 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:17.402 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 16:13:17 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:17.653 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 16:13:17 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:17.654 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 16:13:17 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:18.004 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 16:13:17 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:18.004 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 16:13:17 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:18.283 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 16:13:18 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:18.283 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 16:13:18 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:18.284 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 16:13:18 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:18.284 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 16:13:18 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:18.534 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 16:13:18 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:18.534 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 16:13:18 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:18.817 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 16:13:18 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:18.818 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 16:13:18 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:19.174 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 16:13:18 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:19.174 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 16:13:18 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:19.174 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 16:13:19 smithi117 systemd[1]: Stopping Ceph node-exporter.a for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T16:13:20.065 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 16:13:19 smithi117 bash[134222]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-node-exporter-a 2024-03-04T16:13:20.065 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 16:13:19 smithi117 systemd[1]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-03-04T16:13:20.065 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 16:13:19 smithi117 bash[134283]: Error response from daemon: No such container: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-node-exporter-a 2024-03-04T16:13:20.065 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 16:13:19 smithi117 systemd[1]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715@node-exporter.a.service: Failed with result 'exit-code'. 2024-03-04T16:13:20.065 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 16:13:19 smithi117 systemd[1]: Stopped Ceph node-exporter.a for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T16:13:20.318 INFO:journalctl@ceph.node-exporter.a.smithi117.stdout:Mar 04 16:13:20 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:20.318 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 16:13:20 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:20.318 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 16:13:20 smithi117 systemd[1]: Stopping Ceph alertmanager.a for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T16:13:20.318 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 16:13:20 smithi117 bash[62773]: ts=2024-03-04T16:13:20.286Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-03-04T16:13:21.267 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 16:13:20 smithi117 bash[134352]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-alertmanager-a 2024-03-04T16:13:21.267 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 16:13:21 smithi117 bash[134413]: Error response from daemon: No such container: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-alertmanager-a 2024-03-04T16:13:21.267 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 16:13:21 smithi117 systemd[1]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715@alertmanager.a.service: Deactivated successfully. 2024-03-04T16:13:21.267 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 16:13:21 smithi117 systemd[1]: Stopped Ceph alertmanager.a for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T16:13:21.536 INFO:journalctl@ceph.alertmanager.a.smithi117.stdout:Mar 04 16:13:21 smithi117 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:21.806 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm rm-cluster --fsid e40b5504-da3c-11ee-95c6-87774f69a715 --force --keep-logs 2024-03-04T16:13:21.975 INFO:teuthology.orchestra.run.smithi151.stdout:Deleting cluster with fsid: e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T16:13:26.815 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:13:26 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:26.815 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 16:13:26 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:26.815 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 16:13:26 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:27.166 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:13:26 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:27.167 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 16:13:26 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:27.167 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 16:13:26 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:27.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:13:27 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:27.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:13:27 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:27.431 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 16:13:27 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:27.431 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 16:13:27 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:27.431 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 16:13:27 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:27.432 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 16:13:27 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:27.770 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:13:27 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:27.770 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 16:13:27 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:27.770 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 16:13:27 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:28.092 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 16:13:27 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:28.092 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 16:13:27 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:28.093 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:13:27 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:28.093 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:13:27 smithi151 systemd[1]: Stopping Ceph iscsi.iscsi.a for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T16:13:28.431 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:13:28 smithi151 bash[48108]: debug Shutdown received 2024-03-04T16:13:38.860 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:13:38 smithi151 bash[54339]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-iscsi-iscsi-a 2024-03-04T16:13:39.117 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 16:13:39 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:39.118 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 16:13:39 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:39.118 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:13:38 smithi151 systemd[1]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-03-04T16:13:39.118 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:13:38 smithi151 bash[54436]: Error response from daemon: No such container: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-iscsi-iscsi-a 2024-03-04T16:13:39.118 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:13:38 smithi151 systemd[1]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-03-04T16:13:39.118 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:13:38 smithi151 systemd[1]: Stopped Ceph iscsi.iscsi.a for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T16:13:39.118 INFO:journalctl@ceph.iscsi.iscsi.a.smithi151.stdout:Mar 04 16:13:39 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:40.278 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 16:13:39 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:40.278 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 16:13:40 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:40.278 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 16:13:39 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:40.278 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 16:13:40 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:40.612 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 16:13:40 smithi151 systemd[1]: Stopping Ceph node-exporter.b for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T16:13:41.181 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 16:13:40 smithi151 bash[54637]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-node-exporter-b 2024-03-04T16:13:41.181 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 16:13:41 smithi151 bash[54698]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-node-exporter-b 2024-03-04T16:13:41.181 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 16:13:41 smithi151 systemd[1]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-03-04T16:13:41.181 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 16:13:41 smithi151 systemd[1]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715@node-exporter.b.service: Failed with result 'exit-code'. 2024-03-04T16:13:41.181 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 16:13:41 smithi151 systemd[1]: Stopped Ceph node-exporter.b for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T16:13:41.497 INFO:journalctl@ceph.node-exporter.b.smithi151.stdout:Mar 04 16:13:41 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:41.498 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 16:13:41 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:41.498 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 16:13:41 smithi151 systemd[1]: Stopping Ceph grafana.a for e40b5504-da3c-11ee-95c6-87774f69a715... 2024-03-04T16:13:42.174 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 16:13:41 smithi151 bash[54766]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-grafana-a 2024-03-04T16:13:42.175 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 16:13:41 smithi151 systemd[1]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-03-04T16:13:42.175 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 16:13:42 smithi151 bash[54827]: Error response from daemon: No such container: ceph-e40b5504-da3c-11ee-95c6-87774f69a715-grafana-a 2024-03-04T16:13:42.175 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 16:13:42 smithi151 systemd[1]: ceph-e40b5504-da3c-11ee-95c6-87774f69a715@grafana.a.service: Failed with result 'exit-code'. 2024-03-04T16:13:42.175 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 16:13:42 smithi151 systemd[1]: Stopped Ceph grafana.a for e40b5504-da3c-11ee-95c6-87774f69a715. 2024-03-04T16:13:42.432 INFO:journalctl@ceph.grafana.a.smithi151.stdout:Mar 04 16:13:42 smithi151 systemd[1]: /etc/systemd/system/ceph-e40b5504-da3c-11ee-95c6-87774f69a715@.service:23: Unit configured to use KillMode=none. This is unsafe, as it disables systemd's process lifecycle management for the service. Please update your service to use a safer KillMode=, such as 'mixed' or 'control-group'. Support for KillMode=none is deprecated and will eventually be removed. 2024-03-04T16:13:42.695 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-04T16:13:42.710 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-03-04T16:13:42.722 INFO:tasks.cephadm:Archiving crash dumps... 2024-03-04T16:13:42.725 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/crash to /home/teuthworker/archive/tonay-2024-03-04_15:07:38-orch:cephadm-wip-asm0deuz-testing-2024-03-04-1211-distro-default-smithi/7581221/remote/smithi117/crash 2024-03-04T16:13:42.726 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/crash -- . 2024-03-04T16:13:42.764 INFO:teuthology.orchestra.run.smithi117.stderr:tar: /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/crash: Cannot open: No such file or directory 2024-03-04T16:13:42.764 INFO:teuthology.orchestra.run.smithi117.stderr:tar: Error is not recoverable: exiting now 2024-03-04T16:13:42.766 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/crash to /home/teuthworker/archive/tonay-2024-03-04_15:07:38-orch:cephadm-wip-asm0deuz-testing-2024-03-04-1211-distro-default-smithi/7581221/remote/smithi151/crash 2024-03-04T16:13:42.767 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/crash -- . 2024-03-04T16:13:42.777 INFO:teuthology.orchestra.run.smithi151.stderr:tar: /var/lib/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/crash: Cannot open: No such file or directory 2024-03-04T16:13:42.777 INFO:teuthology.orchestra.run.smithi151.stderr:tar: Error is not recoverable: exiting now 2024-03-04T16:13:42.778 INFO:tasks.cephadm:Checking cluster log for badness... 2024-03-04T16:13:42.778 DEBUG:teuthology.orchestra.run.smithi117:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-04T16:13:42.825 INFO:teuthology.orchestra.run.smithi117.stdout:2024-03-04T15:41:41.931377+0000 mon.a (mon.0) 179 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-03-04T16:13:42.825 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-03-04T16:13:42.826 DEBUG:teuthology.orchestra.run.smithi117:> sudo egrep '\[SEC\]' /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-04T16:13:42.882 DEBUG:teuthology.orchestra.run.smithi117:> sudo egrep '\[ERR\]' /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-04T16:13:42.938 DEBUG:teuthology.orchestra.run.smithi117:> sudo egrep '\[WRN\]' /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-03-04T16:13:42.994 INFO:teuthology.orchestra.run.smithi117.stdout:2024-03-04T15:41:41.931377+0000 mon.a (mon.0) 179 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-03-04T16:13:42.994 INFO:tasks.cephadm:Compressing logs... 2024-03-04T16:13:42.995 DEBUG:teuthology.orchestra.run.smithi117:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-04T16:13:43.038 DEBUG:teuthology.orchestra.run.smithi151:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-03-04T16:13:43.046 INFO:teuthology.orchestra.run.smithi117.stderr:find: '/var/log/rbd-target-api': No such file or directory 2024-03-04T16:13:43.053 INFO:teuthology.orchestra.run.smithi151.stderr:find: '/var/log/rbd-target-api': No such file or directory 2024-03-04T16:13:43.060 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-04T16:13:43.060 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-mgr.y.log 2024-03-04T16:13:43.061 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.0.log 2024-03-04T16:13:43.061 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.2.log 2024-03-04T16:13:43.062 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-mon.a.log 2024-03-04T16:13:43.062 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.cephadm.log 2024-03-04T16:13:43.064 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.3.log 2024-03-04T16:13:43.064 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.1.log 2024-03-04T16:13:43.064 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.3.log: 88.9% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.cephadm.log.gz 2024-03-04T16:13:43.064 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.log 2024-03-04T16:13:43.064 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-volume.log 2024-03-04T16:13:43.071 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.1.log: /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-04T16:13:43.072 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-mon.c.log 2024-03-04T16:13:43.073 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-volume.log: 92.8% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.log.gz 2024-03-04T16:13:43.073 INFO:teuthology.orchestra.run.smithi117.stderr:gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.audit.log 2024-03-04T16:13:43.080 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-03-04T16:13:43.080 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.7.log 2024-03-04T16:13:43.081 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/cephadm.log: gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.6.log 2024-03-04T16:13:43.082 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.cephadm.log 2024-03-04T16:13:43.082 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-client.rgw.foo.a.log 2024-03-04T16:13:43.083 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.5.log 2024-03-04T16:13:43.083 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.6.log: /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/tcmu-runner.log 2024-03-04T16:13:43.083 INFO:teuthology.orchestra.run.smithi151.stderr: 81.0% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.cephadm.log.gz 2024-03-04T16:13:43.083 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-mgr.x.log 2024-03-04T16:13:43.085 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.log 2024-03-04T16:13:43.085 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/tcmu-runner.log: /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-mgr.x.log: 67.8% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/tcmu-runner.log.gz 2024-03-04T16:13:43.085 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-volume.log 2024-03-04T16:13:43.085 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.log: gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.audit.log 2024-03-04T16:13:43.085 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-mon.b.log 2024-03-04T16:13:43.088 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.audit.log: 85.2% -- replaced with /var/log/ceph/cephadm.log.gz 2024-03-04T16:13:43.089 INFO:teuthology.orchestra.run.smithi151.stderr:gzip -5 --verbose -- /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.4.log 2024-03-04T16:13:43.091 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-mon.b.log: 87.2% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.log.gz 2024-03-04T16:13:43.097 INFO:teuthology.orchestra.run.smithi151.stderr:/var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.4.log: 92.5% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-mgr.x.log.gz 2024-03-04T16:13:43.098 INFO:teuthology.orchestra.run.smithi151.stderr: 92.3% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.audit.log.gz 2024-03-04T16:13:43.102 INFO:teuthology.orchestra.run.smithi117.stderr:/var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.audit.log: /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-client.rgw.foo.a.log: 91.0% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-volume.log.gz 2024-03-04T16:13:43.102 INFO:teuthology.orchestra.run.smithi151.stderr: 91.0% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-volume.log.gz 2024-03-04T16:13:43.129 INFO:teuthology.orchestra.run.smithi117.stderr: 95.2% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph.audit.log.gz 2024-03-04T16:13:43.147 INFO:teuthology.orchestra.run.smithi117.stderr: 94.4% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-client.rgw.foo.a.log.gz 2024-03-04T16:13:43.521 INFO:teuthology.orchestra.run.smithi117.stderr: 91.0% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-mgr.y.log.gz 2024-03-04T16:13:44.118 INFO:teuthology.orchestra.run.smithi117.stderr: 92.8% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-mon.c.log.gz 2024-03-04T16:13:44.578 INFO:teuthology.orchestra.run.smithi151.stderr: 93.2% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-mon.b.log.gz 2024-03-04T16:13:45.164 INFO:teuthology.orchestra.run.smithi117.stderr: 92.4% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-mon.a.log.gz 2024-03-04T16:13:49.676 INFO:teuthology.orchestra.run.smithi151.stderr: 93.8% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.6.log.gz 2024-03-04T16:13:49.936 INFO:teuthology.orchestra.run.smithi151.stderr: 93.9% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.7.log.gz 2024-03-04T16:13:50.055 INFO:teuthology.orchestra.run.smithi117.stderr: 93.9% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.0.log.gz 2024-03-04T16:13:50.107 INFO:teuthology.orchestra.run.smithi117.stderr: 93.9% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.2.log.gz 2024-03-04T16:13:50.185 INFO:teuthology.orchestra.run.smithi151.stderr: 93.7% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.5.log.gz 2024-03-04T16:13:50.442 INFO:teuthology.orchestra.run.smithi117.stderr: 93.9% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.1.log.gz 2024-03-04T16:13:50.606 INFO:teuthology.orchestra.run.smithi151.stderr: 93.9% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.4.log.gz 2024-03-04T16:13:50.607 INFO:teuthology.orchestra.run.smithi151.stderr: 2024-03-04T16:13:50.607 INFO:teuthology.orchestra.run.smithi151.stderr:real 0m7.564s 2024-03-04T16:13:50.607 INFO:teuthology.orchestra.run.smithi151.stderr:user 0m28.610s 2024-03-04T16:13:50.608 INFO:teuthology.orchestra.run.smithi151.stderr:sys 0m1.027s 2024-03-04T16:13:51.196 INFO:teuthology.orchestra.run.smithi117.stderr: 93.9% -- replaced with /var/log/ceph/e40b5504-da3c-11ee-95c6-87774f69a715/ceph-osd.3.log.gz 2024-03-04T16:13:51.197 INFO:teuthology.orchestra.run.smithi117.stderr: 2024-03-04T16:13:51.197 INFO:teuthology.orchestra.run.smithi117.stderr:real 0m8.157s 2024-03-04T16:13:51.198 INFO:teuthology.orchestra.run.smithi117.stderr:user 0m32.103s 2024-03-04T16:13:51.198 INFO:teuthology.orchestra.run.smithi117.stderr:sys 0m1.168s 2024-03-04T16:13:51.198 INFO:tasks.cephadm:Archiving logs... 2024-03-04T16:13:51.198 DEBUG:teuthology.misc:Transferring archived files from smithi117:/var/log/ceph to /home/teuthworker/archive/tonay-2024-03-04_15:07:38-orch:cephadm-wip-asm0deuz-testing-2024-03-04-1211-distro-default-smithi/7581221/remote/smithi117/log 2024-03-04T16:13:51.199 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-04T16:13:55.103 DEBUG:teuthology.misc:Transferring archived files from smithi151:/var/log/ceph to /home/teuthworker/archive/tonay-2024-03-04_15:07:38-orch:cephadm-wip-asm0deuz-testing-2024-03-04-1211-distro-default-smithi/7581221/remote/smithi151/log 2024-03-04T16:13:55.106 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /var/log/ceph -- . 2024-03-04T16:13:58.522 INFO:tasks.cephadm:Removing cluster... 2024-03-04T16:13:58.523 DEBUG:teuthology.orchestra.run.smithi117:> sudo cephadm rm-cluster --fsid e40b5504-da3c-11ee-95c6-87774f69a715 --force 2024-03-04T16:13:58.691 INFO:teuthology.orchestra.run.smithi117.stdout:Deleting cluster with fsid: e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T16:14:00.076 DEBUG:teuthology.orchestra.run.smithi151:> sudo cephadm rm-cluster --fsid e40b5504-da3c-11ee-95c6-87774f69a715 --force 2024-03-04T16:14:00.248 INFO:teuthology.orchestra.run.smithi151.stdout:Deleting cluster with fsid: e40b5504-da3c-11ee-95c6-87774f69a715 2024-03-04T16:14:01.633 INFO:tasks.cephadm:Teardown complete 2024-03-04T16:14:01.633 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-03-04T16:14:01.643 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-03-04T16:14:01.643 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-04T16:14:01.646 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-03-04T16:14:01.711 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-03-04T16:14:01.721 INFO:teuthology.task.clock:Checking final clock skew... 2024-03-04T16:14:01.722 DEBUG:teuthology.orchestra.run.smithi117:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T16:14:01.724 DEBUG:teuthology.orchestra.run.smithi151:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-03-04T16:14:01.735 INFO:teuthology.orchestra.run.smithi117.stdout: remote refid st t when poll reach delay offset jitter 2024-03-04T16:14:01.736 INFO:teuthology.orchestra.run.smithi117.stdout:============================================================================== 2024-03-04T16:14:01.736 INFO:teuthology.orchestra.run.smithi117.stdout:*hv01.front.sepi 44.4.53.2 2 u 45 64 377 0.078 -2.780 1.772 2024-03-04T16:14:01.736 INFO:teuthology.orchestra.run.smithi117.stdout:+hv02.front.sepi 63.231.80.2 3 u 40 64 377 0.064 -2.922 0.770 2024-03-04T16:14:01.736 INFO:teuthology.orchestra.run.smithi117.stdout:+hv03.front.sepi 74.6.168.72 3 u 47 64 377 0.090 -7.487 2.161 2024-03-04T16:14:01.736 INFO:teuthology.orchestra.run.smithi117.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-03-04T16:14:01.737 INFO:teuthology.orchestra.run.smithi151.stdout: remote refid st t when poll reach delay offset jitter 2024-03-04T16:14:01.737 INFO:teuthology.orchestra.run.smithi151.stdout:============================================================================== 2024-03-04T16:14:01.737 INFO:teuthology.orchestra.run.smithi151.stdout:*hv01.front.sepi 44.4.53.2 2 u 38 128 377 0.108 +1.692 0.135 2024-03-04T16:14:01.738 INFO:teuthology.orchestra.run.smithi151.stdout:+hv02.front.sepi 63.231.80.2 3 u 31 128 377 0.121 -0.144 0.350 2024-03-04T16:14:01.738 INFO:teuthology.orchestra.run.smithi151.stdout:+hv03.front.sepi 74.6.168.72 3 u 95 128 377 0.094 -3.672 0.878 2024-03-04T16:14:01.738 INFO:teuthology.orchestra.run.smithi151.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2024-03-04T16:14:01.738 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-03-04T16:14:01.747 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-03-04T16:14:01.747 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-03-04T16:14:01.756 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-03-04T16:14:01.765 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-03-04T16:14:01.773 INFO:teuthology.task.internal:Duration was 2830.129056 seconds 2024-03-04T16:14:01.773 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-03-04T16:14:01.782 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-03-04T16:14:01.782 DEBUG:teuthology.orchestra.run.smithi117:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-04T16:14:01.787 DEBUG:teuthology.orchestra.run.smithi151:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-03-04T16:14:01.823 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-03-04T16:14:01.823 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi117.front.sepia.ceph.com 2024-03-04T16:14:01.824 DEBUG:teuthology.orchestra.run.smithi117:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-04T16:14:01.870 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi151.front.sepia.ceph.com 2024-03-04T16:14:01.870 DEBUG:teuthology.orchestra.run.smithi151:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-03-04T16:14:01.882 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-03-04T16:14:01.882 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-04T16:14:01.915 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-03-04T16:14:01.984 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-03-04T16:14:01.984 DEBUG:teuthology.orchestra.run.smithi117:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-04T16:14:01.986 DEBUG:teuthology.orchestra.run.smithi151:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-03-04T16:14:02.189 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-03-04T16:14:02.200 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-03-04T16:14:02.200 DEBUG:teuthology.orchestra.run.smithi117:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-04T16:14:02.211 DEBUG:teuthology.orchestra.run.smithi151:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-03-04T16:14:02.224 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-03-04T16:14:02.249 DEBUG:teuthology.orchestra.run.smithi117:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-04T16:14:02.254 DEBUG:teuthology.orchestra.run.smithi151:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-03-04T16:14:02.263 INFO:teuthology.orchestra.run.smithi117.stdout:kernel.core_pattern = core 2024-03-04T16:14:02.277 INFO:teuthology.orchestra.run.smithi151.stdout:kernel.core_pattern = core 2024-03-04T16:14:02.298 DEBUG:teuthology.orchestra.run.smithi117:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-04T16:14:02.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T16:14:02.321 DEBUG:teuthology.orchestra.run.smithi151:> test -e /home/ubuntu/cephtest/archive/coredump 2024-03-04T16:14:02.347 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-03-04T16:14:02.348 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-03-04T16:14:02.359 INFO:teuthology.task.internal:Transferring archived files... 2024-03-04T16:14:02.362 DEBUG:teuthology.misc:Transferring archived files from smithi117:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/tonay-2024-03-04_15:07:38-orch:cephadm-wip-asm0deuz-testing-2024-03-04-1211-distro-default-smithi/7581221/remote/smithi117 2024-03-04T16:14:02.362 DEBUG:teuthology.orchestra.run.smithi117:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-04T16:14:02.414 DEBUG:teuthology.misc:Transferring archived files from smithi151:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/tonay-2024-03-04_15:07:38-orch:cephadm-wip-asm0deuz-testing-2024-03-04-1211-distro-default-smithi/7581221/remote/smithi151 2024-03-04T16:14:02.414 DEBUG:teuthology.orchestra.run.smithi151:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-03-04T16:14:02.441 INFO:teuthology.task.internal:Removing archive directory... 2024-03-04T16:14:02.441 DEBUG:teuthology.orchestra.run.smithi117:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-04T16:14:02.444 DEBUG:teuthology.orchestra.run.smithi151:> rm -rf -- /home/ubuntu/cephtest/archive 2024-03-04T16:14:02.487 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-03-04T16:14:02.500 INFO:teuthology.task.internal:Not uploading archives. 2024-03-04T16:14:02.500 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-03-04T16:14:02.511 INFO:teuthology.task.internal:Tidying up after the test... 2024-03-04T16:14:02.512 DEBUG:teuthology.orchestra.run.smithi117:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-04T16:14:02.514 DEBUG:teuthology.orchestra.run.smithi151:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-03-04T16:14:02.518 INFO:teuthology.orchestra.run.smithi117.stdout: 658641 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 4 16:14 /home/ubuntu/cephtest 2024-03-04T16:14:02.534 INFO:teuthology.orchestra.run.smithi151.stdout: 658459 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Mar 4 16:14 /home/ubuntu/cephtest 2024-03-04T16:14:02.536 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-03-04T16:14:02.548 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-03-04T16:14:02.573 INFO:teuthology.nuke:Checking targets against current locks 2024-03-04T16:14:02.602 DEBUG:teuthology.nuke:shortname: smithi117 2024-03-04T16:14:02.603 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-04T16:14:02.627 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi117.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/tonay-2024-03-04_15:07:38-orch:cephadm-wip-asm0deuz-testing-2024-03-04-1211-distro-default-smithi/7581221', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 15:17:30.128623', 'locked_by': 'scheduled_tonay@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHf1ehnhjHc10KhtKu4jwsfchliAUjFZXiiJi8z1Q9fdLAxIiu8KDLw6bdOoDiteRYYzxWxxnkYpz5bCyVgptCs='} 2024-03-04T16:14:02.631 DEBUG:teuthology.nuke:shortname: smithi151 2024-03-04T16:14:02.631 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-03-04T16:14:02.652 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi151.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/tonay-2024-03-04_15:07:38-orch:cephadm-wip-asm0deuz-testing-2024-03-04-1211-distro-default-smithi/7581221', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-03-04 15:17:30.130277', 'locked_by': 'scheduled_tonay@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGnhs4B90sAmKtOUf5PPUHN5Bh0v5nKFlRDlAbSotJEf7zPCbtBZT2URm7gLcEtfK8Trbx0k8X847cqw8GxI7sI='} 2024-03-04T16:14:02.689 INFO:teuthology.orchestra.console.smithi117:Power off 2024-03-04T16:14:02.689 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-04T16:14:02.704 INFO:teuthology.orchestra.console.smithi151:Power off 2024-03-04T16:14:02.704 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-03-04T16:14:02.720 DEBUG:teuthology.orchestra.console.smithi117:power off output: Chassis Power Control: Down/Off 2024-03-04T16:14:02.720 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T16:14:02.739 DEBUG:teuthology.orchestra.console.smithi151:power off output: Chassis Power Control: Down/Off 2024-03-04T16:14:02.739 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T16:14:02.753 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is on 2024-03-04T16:14:02.770 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on 2024-03-04T16:14:06.754 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T16:14:06.771 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T16:14:06.865 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is on 2024-03-04T16:14:06.882 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on 2024-03-04T16:14:10.870 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T16:14:10.883 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T16:14:10.980 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is on 2024-03-04T16:14:10.993 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is on 2024-03-04T16:14:14.984 DEBUG:teuthology.orchestra.console.smithi117:pexpect command: ipmitool -H smithi117.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T16:14:14.995 DEBUG:teuthology.orchestra.console.smithi151:pexpect command: ipmitool -H smithi151.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-03-04T16:14:15.097 DEBUG:teuthology.orchestra.console.smithi117:check power output: Chassis Power is off 2024-03-04T16:14:15.123 DEBUG:teuthology.orchestra.console.smithi151:check power output: Chassis Power is off 2024-03-04T16:14:15.198 INFO:teuthology.orchestra.console.smithi117:Power off completed 2024-03-04T16:14:15.224 INFO:teuthology.orchestra.console.smithi151:Power off completed 2024-03-04T16:14:15.328 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/ubuntu_22.04 fixed-2 mode/packaged mon_election/classic msgr/async start tasks/rados_api_tests} duration: 2830.129055738449 failure_reason: '"2024-03-04T15:41:41.931377+0000 mon.a (mon.0) 179 : cluster 3 [WRN] MON_DOWN: 1/3 mons down, quorum a,b" in cluster log' flavor: default owner: scheduled_tonay@teuthology success: false 2024-03-04T16:14:15.329 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-03-04T16:14:15.415 INFO:teuthology.run:FAIL